WSL1: git-annex-add fails in DrvFs filesystem

Please describe the problem.

git-annex fails to add file to the repository because of permission problem (probably faulty permission handling in WSL). Interestingly, it is possible to add a file anyway, by executing git annex add twice. Unfortunately, files added this way are writeable, when they shouldn't.

It's probably not in the scope of git-annex developing, but I think it's good to keep trace on the problem.

What steps will reproduce the problem?

cd /mnt/c
git init test
cd test
git annex init test
init test
touch file
git annex add file

What version of git-annex are you using? On what operating system?

Windows 10 Pro version 1909 build 18363.476 - WSL (Arch)

git-annex version: 7.20191114-ga95efcbc55
build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.3 feed-1.2.0.1 ghc-8.6.5 http-client-0.6.4 persistent-sqlite-2.10.5 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
operating system: linux x86_64
supported repository versions: 7
upgrade supported from repository versions: 0 1 2 3 4 5 6
local repository version: 7

Please provide any additional information below.

$ git annex init test
init test
  Detected a filesystem without fifo support.

  Disabling ssh connection caching.
(scanning for unlocked files...)
ok
(recording state in git...)
$ touch file
$ git annex add file --debug
[2019-11-28 11:52:53.048398] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
[2019-11-28 11:52:53.0570463] process done ExitSuccess
[2019-11-28 11:52:53.0573639] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
[2019-11-28 11:52:53.0656397] process done ExitFailure 1
[2019-11-28 11:52:53.0660529] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--others","--exclude-standard","-z","--","file"]
[2019-11-28 11:52:53.0742999] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
[2019-11-28 11:52:53.0822627] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2019-11-28 11:52:53.0853736] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
add file [2019-11-28 11:52:53.0949002] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
[2019-11-28 11:52:53.1027361] process done ExitSuccess
[2019-11-28 11:52:53.1030132] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
[2019-11-28 11:52:53.1122577] process done ExitFailure 1

[2019-11-28 11:52:53.1232169] call: cp ["--reflink=auto","--preserve=timestamps",".git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","file"]
[2019-11-28 11:52:53.1606206] process done ExitSuccess

git-annex: .git/annex/othertmp/file.0/file: rename: permission denied (Permission denied)
failed
[2019-11-28 11:52:53.1617248] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--modified","-z","--","file"]
[2019-11-28 11:52:53.1693198] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","diff","--name-only","--diff-filter=T","-z","--cached","--","file"]
[2019-11-28 11:52:53.1825925] process done ExitSuccess
[2019-11-28 11:52:53.1835521] process done ExitSuccess
[2019-11-28 11:52:53.1844047] process done ExitSuccess
git-annex: add: 1 failed

Second attempt:

$ git annex add file --debug
[2019-11-28 11:57:56.4029726] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
[2019-11-28 11:57:56.4114361] process done ExitSuccess
[2019-11-28 11:57:56.411681] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
[2019-11-28 11:57:56.4201317] process done ExitFailure 1
[2019-11-28 11:57:56.420548] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--others","--exclude-standard","-z","--","file"]
[2019-11-28 11:57:56.4316368] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
[2019-11-28 11:57:56.4416827] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2019-11-28 11:57:56.4452357] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
add file [2019-11-28 11:57:56.4545013] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
[2019-11-28 11:57:56.4626846] process done ExitSuccess
[2019-11-28 11:57:56.4629866] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
[2019-11-28 11:57:56.4735385] process done ExitFailure 1

[2019-11-28 11:57:56.4848163] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2019-11-28 11:57:56.488706] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
ok
[2019-11-28 11:57:56.4964438] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--modified","-z","--","file"]
[2019-11-28 11:57:56.5043041] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","diff","--name-only","--diff-filter=T","-z","--cached","--","file"]
(recording state in git...)
[2019-11-28 11:57:56.5152453] feed: xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--"]
[2019-11-28 11:57:56.5426207] process done ExitSuccess
[2019-11-28 11:57:56.5438586] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","hash-object","-w","--stdin-paths","--no-filters"]
[2019-11-28 11:57:56.5478542] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","update-index","-z","--index-info"]
[2019-11-28 11:57:56.5713] process done ExitSuccess
[2019-11-28 11:57:56.5716027] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2019-11-28 11:57:56.5803067] process done ExitSuccess
[2019-11-28 11:57:56.5807703] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","write-tree"]
[2019-11-28 11:57:56.6111405] process done ExitSuccess
[2019-11-28 11:57:56.6115303] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","commit-tree","ffa5a12eba0b2ea9bc5b529278597615f70c901c","--no-gpg-sign","-p","refs/heads/git-annex"]
[2019-11-28 11:57:56.6269742] process done ExitSuccess
[2019-11-28 11:57:56.6272697] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","update-ref","refs/heads/git-annex","0ece4a3a069693ea12cb61168cfb701040c8a7a7"]
[2019-11-28 11:57:56.6465065] process done ExitSuccess
[2019-11-28 11:57:56.6506175] process done ExitSuccess
[2019-11-28 11:57:56.651426] process done ExitSuccess
[2019-11-28 11:57:56.6520969] process done ExitSuccess
[2019-11-28 11:57:56.6527282] process done ExitSuccess
[2019-11-28 11:57:56.6536136] process done ExitSuccess
[2019-11-28 11:57:56.6554327] process done ExitSuccess
$ echo "this should fail" > file
$ cat file
this should fail

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Yes! Thank you very much Joey for your hard work and digging into WSL bugs

RSS Atom

Using -o metadata?

I'm having the same issue, though I can add files if I set annex.addunlocked=true.

Comment by annex2384 — Tue Feb 4 03:22:45 2020

Remove comment

comment 2

These look like very similar problems: https://git-annex.branchable.com/bugs/__34__rename__58___permission_denied__34__/ https://git-annex.branchable.com/bugs/still_seeing_errors_with_parallel_git-annex-add/

Comment by joey — Wed Oct 20 17:31:07 2021

comment 3

This problem as it happens in WSL1 is due to calling withhardlink in lockDown'

lockDown' cfg file = tryNonAsync $ ifM crippledFileSystem
    ( nohardlink
    , case hardlinkFileTmpDir cfg of
        Nothing -> nohardlink
        Just tmpdir -> withhardlink tmpdir
    )

With nohardlink, the write bits are removed from the file in the working tree, moved into .git/annex/objects, a symlink is created in a temp dir and moved into the working tree. No problems here.

With withhardlink, the write bits are removed from the file in the working tree, hardlinked into a temp dir, moved into .git/annex/objects, a symlink is created in a temp dir and moved into the working tree replacing the original file. However, Windows will not allow you to do this because when the write bits are removed, the file is set to read only and cannot be replaced in this manner. The only way to get around this is to delete the file in the working tree first.

Comment by asakurareiko — Tue Oct 26 20:13:18 2021

comment 4

Interesting, thanks for that analysis.

I don't like the prospect of adding hacks to git-annex on linux to deal with infelicities of WSL1. WSL should emulate linux well, and to the extent it does not, that seems Microsoft's problem, not mine.

However, I wonder what that analysis implies about the other linked bugs with similar error messages, which are on OSX and (probably) Linux.

What could be done to detect this drvfs problem is to add this case to the probing that's done for crippled filesystems. Then it would take the nohardlink path.

But drvfs is used for /mnt/c/ etc, so that would make most all repos on WSL1 be treated as in a crippled filesystem, using unlocked adjusted branches, etc.

Also, I seem to remember using git-annex in WSL1 in /mnt/c/ and git-annex add presumably working. So I wonder what has changed.

Comment by joey — Wed Oct 27 18:19:58 2021