Missing file content in secondary worktree (win)

Please describe the problem.

In a secondary worktree on Windows, pointer files remain even with keys being available. A git annex get fails with "not available", even though "whereis" reports "[here]".

I would expect the behavior to be internally consistent (keys being available in the worktree when they are found to be available in the repository).

This report assumes that secondary worktrees are, in principle, a supported use case, based on the statement at https://git-annex.branchable.com/tips/Using_git-worktree_with_annex/: "Getting, dropping and syncing content works fine in a worktree".

What steps will reproduce the problem?

C:\Users\mih>md demo
C:\Users\mih>cd demo
C:\Users\mih\demo>git init
Initialized empty Git repository in C:/Users/mih/demo/.git/

C:\Users\mih\demo>git annex init
init
  Detected a filesystem without fifo support.
  Disabling ssh connection caching.
  Detected a crippled filesystem.
  Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
Switched to branch 'adjusted/master(unlocked)'
ok
(recording state in git...)

C:\Users\mih\demo>echo "onetwothree" > 123.txt
C:\Users\mih\demo>git annex add 123.txt
add 123.txt
ok
(recording state in git...)
C:\Users\mih\demo>git commit -m Demo
[adjusted/master(unlocked) abf6dc4] Demo
 1 file changed, 1 insertion(+)
 create mode 100644 123.txt

C:\Users\mih\demo>git status
On branch adjusted/master(unlocked)
nothing to commit, working tree clean

C:\Users\mih\demo>git worktree add ..\demo-wt2
Preparing worktree (new branch 'demo-wt2')
HEAD is now at abf6dc4 Demo

C:\Users\mih\demo>cd ..\demo-wt2
C:\Users\mih\demo-wt2>dir
 Volume in drive C is Windows
 Volume Serial Number is ECB5-B3C0

 Directory of C:\Users\mih\demo-wt2

08/07/2025  12:30    <DIR>          .
08/07/2025  12:30    <DIR>          ..
08/07/2025  12:30                98 123.txt
               1 File(s)             98 bytes
               2 Dir(s)  384.871.727.104 bytes free

C:\Users\mih\demo-wt2>type 123.txt
/annex/objects/SHA256E-s16--7091a3bb554a96356db798ae528b2eb2ec9ca6ef99daa0263e6f0af65b17bd5c.txt

C:\Users\mih\demo-wt2>git annex get 123.txt
get 123.txt (not available)
  No other repository is known to contain the file.
failed
get: 1 failed

C:\Users\mih\demo-wt2>git annex whereis 123.txt
whereis 123.txt (1 copy)
        a4127935-16c9-4d81-98c6-9fd65461293c -- mih@bnbnb67:~/demo [here]
ok

What version of git-annex are you using? On what operating system?

git-annex version: 10.20250630-gc6b6be2eab17fd5d8921f3af9376d15f2cf917f5
build flags: Assistant Webapp Pairing TorrentParser MagicMime Servant Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.24.4 bloomfilter-2.0.1.2 crypton-1.0.4 DAV-1.3.4 feed-1.3.2.1 ghc-9.10.1 http-client-0.7.19 persistent-sqlite-2.13.3.0 torrent-10000.1.3 uuid-1.3.16 yesod-1.6.2.1
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL GITBUNDLE GITMANIFEST VURL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg rclone hook external compute mask
operating system: mingw32 x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10
local repository version: 10

This is on win11.

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Absolutely! Approaching 15 years of bigger, better, faster, more ;-)

RSS Atom

comment 1

I reproduced the same behavior on linux, when using a FAT filesystem.

So, this has something to do with automatic entry of an unlocked adjusted branch on a crippled filesystem.

Interestingly, doing the same on ext4, and manually using git-annex unlock on the file and committing before checking out the worktree does not replicate the problem. The unlocked file is automatically populated on worktree checkout there. And manual git-annex adjust --unlock before worktree creation also doesn't have the problem, even though the worktree does end up in an adjusted unlocked branch.

(The output of git-annex get is also weird. I think what's happening is that, since the unlocked file is not populated, it is enumerated as a file that get can operate on. But then when it runs, since there is no other location, it displays that message. The command does not have anything to handle this unusual case of the file being a pointer file but its content being present in the repisitory. And, usually there is no way that can happen, eg even writing a pointer file manually followed by git add of it populates it. So I think this unusual behavior of git-annex get doesn't need to change, once this bug is fixed it should not be possible to see that behavior.)

Comment by joey — Fri Jul 11 16:05:53 2025

comment 2

In a FAT filesystem after reproducing this bug with initial file foo, the following thing also happens:

joey@darkstar:~/mnt/demo-wt3#demo-wt3>cat foo
/annex/objects/SHA256E-s30--dcf81122854db210a12a47851a3430b6ab000e3f981b5266f0873b94d130c999
joey@darkstar:~/mnt/demo-wt3#demo-wt3>cp foo bar
joey@darkstar:~/mnt/demo-wt3#demo-wt3>git add bar
joey@darkstar:~/mnt/demo-wt3#demo-wt3>cat bar
/annex/objects/SHA256E-s30--dcf81122854db210a12a47851a3430b6ab000e3f981b5266f0873b94d130c999

This seems to be another case of the bug, because the content of the object is present in the repository, so usually git add of a pointer file should result in the smudge filter populating it.

git-annex add behaves the same as well.

Comment by joey — Fri Jul 11 16:18:16 2025

comment 3

I've verified that populatePointerFile is not getting called in this case, and does get called in the same situation on ext4. And that call is made by reconcileStaged, which is getting called. So I would look in there for the bug.

Except, interestingly, some percent of the time, on ext4, manually populating the pointer file followed by git-annex add also does not call populatePointerFile. The pointer file remains unpopulated until another process calls reconcileStaged, and it gets populated then. This seems like also a bug, possibly another case of the same bug?

Comment by joey — Fri Jul 11 16:42:26 2025

comment 4

Apparently in the FAT case gitAnnexLocation is returning something like ../demo/.git/worktrees/demo-wt3/annex/objects/d13/2dd/SHA256E-s30--dcf81122854db210a12a47851a3430b6ab000e3f981b5266f0873b94d130c999/SHA256E-s30--dcf81122854db210a12a47851a3430b6ab000e3f981b5266f0873b94d130c999 which is not the right path to the object file. Should be ../demo/.git/annex/objects/d13/2dd/SHA256E-s30--dcf81122854db210a12a47851a3430b6ab000e3f981b5266f0873b94d130c999/SHA256E-s30--dcf81122854db210a12a47851a3430b6ab000e3f981b5266f0873b94d130c999

(In the ext4 case that does not happen, instead the reconcileStaged git diff does not include the new file. So that is a different problem.)

Turns out that .git/worktrees/foo/annex is a symlink when the filesystem supports symlinks. But, when symlinks are not supported, that symlink is not made. And so it looks for objects there, but they're not there. This could also cause other behavior differences, since other state files that go in the annex directory get written there, so git-annex inside and outside the worktree, or in different worktrees, can have different states.

That symlink is needed to make annex symlinks point to the object files. But git-annex shouldn't rely on the symlink in things like gitAnnexLocation.

Luckily, annexDir exists, and I've checked and it is the only thing that produces "annex" as a path to the annex directory. So annexDir could be made into a function that is passed the git repository and handles this special case, by returning a path like "../../annex", which when combined with the git directory in a linked worktree, ends up pointing to the main repository's ".git/annex".

Except, annexDir is not only used to find the paths to object files. It's also used to generate the symlink target. When git-annex add is run in a linked worktree, and symlinks are supported, the symlink target needs to be of the form ".git/annex/". With this annexDir change, it would not be right.

So, it seems that annexDir, and some functions that call it need to behave differently when they're generating a path into the annex directory, vs when they're generating a symlink target or other similar thing. Which is a subtle distinction to introduce.

Comment by joey — Fri Jul 11 17:41:03 2025

comment 5

When a secondary worktree is used on a filesystem not supporting symlinks, it would be possible for git-annex move to move an object from another repository. And store it to the wrong location, under .git/worktrees/foo/annex/objects/. The object would still be accessible, and a later git-annex copy --to remote, if run in the same worktree, would be able to send the object on to a remote.

But if this bug gets fixed, then the misplaced object file will be left, and won't be used any longer. Which could appear to the user as data loss in some situations. Eg, the copy to the remote would fail. (There might be situations where the populated worktree file would be used as a copy of the object, but that assumes the worktree file is still populated.)

Also, git-annex drop would not delete such misplaced object files, so the user would be left with bloated repository.

So, git-annex fsck will need to be made to search out such misplaced object files and move them to the correct objects directory.

Comment by joey — Fri Jul 11 18:38:59 2025

Add a comment