Please describe the problem.
Wanted to use metadata (to annotate anatomical T1s with metadata), and then tried get on a pathspec.
git annex then incorrectly claims that no files patch although I show with git ls-files on the same pathspec that there are files:
❯ git annex version
git-annex version: 10.20240731+git17-g6d1592f857-1~ndall+1
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
...
❯ git ls-files '**/*.nii.gz' | head -n 1
sub-0001/ses-01/anat/sub-0001_ses-01_acq-MPRAGEXp3X08mm_T1w.nii.gz
❯ git annex metadata '**/*.nii.gz'
error: pathspec './**/*.nii.gz' did not match any file(s) known to git
Did you forget to 'git add'?
metadata: 1 failed
# git-annex changed pathspec to have leading ./ -- let's try with that too:
❯ git ls-files './**/*.nii.gz' | head -n 1
sub-0001/ses-01/anat/sub-0001_ses-01_acq-MPRAGEXp3X08mm_T1w.nii.gz
# annex get -- the same story
❯ git annex get '**/*.nii.gz'
error: pathspec './**/*.nii.gz' did not match any file(s) known to git
Did you forget to 'git add'?
(merging typhon/git-annex into git-annex...)
(recording state in git...)
get: 1 failed
From annex --debug we can see that annex unconditionally uses --literal-pathspecs
❯ git annex --debug get '**/*.nii.gz'
[2024-08-23 21:29:36.951044831] (Utility.Process) process [3889124] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--error-unmatch","--","./**/*.nii.gz"]
so, I think then annex should have at least used "literal" in the error, e.g.
error: literal pathspec './**/*.nii.gz' did not match any file(s) known to git
and ideally also hinted on how to disable such behavior (if possible) and do allow for "magical" etc pathspecs there.
FWIW, I have tried with GIT_GLOB_PATHSPECS=1 env var but that didn't help.... not sure if possible at all looking at the code
fixupRepo :: Repo -> GitConfig -> IO Repo
fixupRepo r c = do
let r' = disableWildcardExpansion r
r'' <- fixupUnusualRepos r' c
if annexDirect c
then return (fixupDirect r'')
else return r''
{- Disable git's built-in wildcard expansion, which is not wanted
- when using it as plumbing by git-annex. -}
disableWildcardExpansion :: Repo -> Repo
disableWildcardExpansion r = r
{ gitGlobalOpts = gitGlobalOpts r ++ [Param "--literal-pathspecs"] }
The "did not match any file(s) known to git" message is output by git, not git-annex.
I tracked down why git-annex uses --literal-pathspecs. In f35d0bf4b2d7ada897b66620ff94d4068badd90b, a particular problem mentioned is that
git-annex add *.jpeg, in a case where there are no such files, would addfoo/bar.jpegdue to git ls-files default behavior:Which was very surprising and did not seem desirable for
git-annex add. (Let alone for a command likegit-annex drop --force!)Although
git adddoes in fact behave that way, which surprised me:git addis documented to behave that way, as are some other commands likegit rm(!). But git-annex is not, its commands are documented to operate on filenames or paths. So I don't think this is really a bug.As to providing a way to enable non-literal pathspecs, since git has
GIT_GLOB_PATHSPECSandGIT_ICASE_PATHSPECS, checking for those and removing --literal-pathspecs would be one way. But then it risks unexpected behavior if the git-annex version is too old. So a command-line option seems maybe better.But, I do consider it an implementation detail that git-annex uses
git ls-filesfor some commands. Who knows, there may eventually be a reason to change that. Making this configurable would lock in use of ls-files.There are also situations where git-annex does not use ls-files, which would all need to be covered in the documentation when implementing this. The one that comes to mind is
--batchwhich doesn't recurse trees at all.Of course,
git ls-files <pathspec> | git-annex foo --batchis a way you can operate on a pathspec without any changes.You can accomplish the same thing as that pathspec with
--include=*.nii.gzand it works on all git-annex commands and is a much richer language than git's pathspecs that can do a lot more besides.So, I think it would be redundant to support git's pathspecs, and am going to close this bug.