"git worktree remove" fails if git-annex has operated on the worktree, because git expects .git in a worktree to be a file, while git-annex changes it to a symlink (Using git-worktree with annex). Also, git-annex-test does not currently seem to test git-annex operation on worktrees added by git-worktree?
This seems like something the git developers might consider a bug in git worktree, since generally git doesn't care if .git is a symlink or a pointer file.
Although perhaps the reason it checks is to avoid deleting something it didn't set up.
I don't see any way to improve this in git-annex though, if .git is not a symlink then every git-annex symlink in the repository will need to change to point to the real path to .git/annex/objects, which is not practical.
Maybe git considers .git an internal detail, so they might not guarantee things working correctly if it's changed to a symlink? A clean way to handle this would be to add a 'git annex worktree' command, which normally just passes the rest of the command line to 'git worktree', but handles any git-annex-specific details as needed: e.g. 'git annex worktree add' calls 'git worktree add' then replaces the .git pointer file with symlink; 'git annex worktree remove' puts the .git pointer back then calls 'git worktree remove', etc.
Here's how to remove a worktree:
as done inside the worktree itself. Update paths if you want to remove the worktree from outside of it. So long as you don't run another
git annexcommand after replacing the symlink with a file,worktree removeshould work!