Please describe the problem.
Here is a reproducer using the place where I encountered it -- using git-annex-remote-rclone (tiny bash script; needs fixing though for recent rclone -- take from my fork where I am tuning testing etc) https://github.com/DanielDent/git-annex-remote-rclone/
this is a reproducer which creates repo and commits to annex of size 0
#!/bin/bash
cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)"
set -eux
# provide versioning information to possibly ease troubleshooting
git annex version
rclone --version
export HOME=$PWD
echo -e '[local]\ntype = local\nnounc =' > ~/.rclone.conf
# to pacify git/git-annex
git config --global user.name Me
git config --global user.email me@example.com
git config --global init.defaultBranch master
# Prepare rclone remote local store
mkdir rclone-local
export RCLONE_PREFIX=$PWD/rclone-local
git-annex version
mkdir testrepo
cd testrepo
git init .
git-annex init
git-annex initremote GA-rclone-CI type=external externaltype=rclone target=local prefix=$RCLONE_PREFIX chunk=100MiB encryption=shared mac=HMACSHA512
# Rudimentary test, spaces in the filename must be ok, 0 length files should be ok
touch "test 0"
#echo 1 > "test 1"
git-annex add *
git-annex copy * --to GA-rclone-CI
git-annex drop *
git-annex get --debug *
where it ends with
+ git-annex get --debug 'test 0'
[2022-06-09 13:16:20.668338168] (Utility.Process) process [1022440] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--error-unmatch","--","test 0"]
[2022-06-09 13:16:20.668784836] (Utility.Process) process [1022441] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2022-06-09 13:16:20.669111286] (Utility.Process) process [1022442] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2022-06-09 13:16:20.66965775] (Utility.Process) process [1022443] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2022-06-09 13:16:20.670318546] (Utility.Process) process [1022443] done ExitSuccess
[2022-06-09 13:16:20.67074703] (Utility.Process) process [1022444] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2022-06-09 13:16:20.671504449] (Utility.Process) process [1022444] done ExitSuccess
[2022-06-09 13:16:20.672125442] (Utility.Process) process [1022445] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..0da4aa78e4397feb9906400d1e08aaebccac1be1","--pretty=%H","-n1"]
[2022-06-09 13:16:20.673019668] (Utility.Process) process [1022445] done ExitSuccess
[2022-06-09 13:16:20.673440353] (Utility.Process) process [1022446] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
get test 0 [2022-06-09 13:16:20.674594171] (Utility.Process) process [1022448] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
(from GA-rclone-CI...)
[2022-06-09 13:16:20.682205613] (Annex.Perms) freezing content directory .git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
git-annex: .git/annex/tmp/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855: rename: does not exist (No such file or directory)
failed
[2022-06-09 13:16:20.682296739] (Utility.Process) process [1022446] done ExitSuccess
[2022-06-09 13:16:20.682333234] (Utility.Process) process [1022442] done ExitSuccess
[2022-06-09 13:16:20.682384256] (Utility.Process) process [1022441] done ExitSuccess
[2022-06-09 13:16:20.682408227] (Utility.Process) process [1022440] done ExitSuccess
[2022-06-09 13:16:20.682701178] (Utility.Process) process [1022448] done ExitSuccess
get: 1 failed
- you can see that there is not even an attempt to talk to the remote. git-annex crashes before then with that " rename: does not exist (No such file or directory)"
so I do not think it is an issue of the remote (which might also have some size 0 issues from looking at some conditions)
What version of git-annex are you using? On what operating system?
tried both 8.20211123 and 10.20220504 from debian
I can reproduce this with a directory special remote, initialized with
The chunking is what causes the problem.
This was broken by dad4be97c2057db1ef3a13bb983d1701a90c9069.
For a key of size zero,
addspeculative
adds on a[]
to the list of chunk key. But that causes retrieveChunks to think that it's already retrieved all the chunks, so it avoids doing any retrieval, so the file is not written. It needs to retrieve the empty key even though it's empty, so it can decrypt it when it's encrypted; git-annex does not special case empty file retrieval.Odd that testremote did not detect this. It does test with empty keys, and with chunking.
PS I wish there was a similar but automated annotation of those
[done]
to know actually "when/with what"