What steps will reproduce the problem?
ask annex get in parallel files which point to the same key
What version of git-annex are you using? On what operating system?
6.20170815+gitg22da64d0f-1~ndall+1
Please provide any additional information below.
# works in serial mode
$> git annex get rh.white{,_avg}
get rh.white (from web...)
/mnt/btrfs/scrap/tmp/ds0001 100%[===========================================>] 360.31K --.-KB/s in 0.1s
2017-08-30 10:08:02 URL:https://dl.dropboxusercontent.com/s/0lww4tomnwfanwd/rh.white_avg?dl=0 [368962/368962] -> "/mnt/btrfs/scrap/tmp/ds000114/derivatives/freesurfer/.git/annex/tmp/MD5E-s368962--99a4db61cedffee686aef99b2d197794" [1]
(checksum...) ok
(recording state in git...)
(dev)2 10016.....................................:Wed 30 Aug 2017 10:08:02 AM EDT:.
(git)smaug:…/btrfs/scrap/tmp/ds000114/derivatives/freesurfer[master]fsaverage5/surf
$> git annex drop --fast rh.white{,_avg}
drop rh.white (checking https://dl.dropbox.com/s/0lww4tomnwfanwd/rh.white_avg?dl=0...) ok
(recording state in git...)
# "fails" in parallel
$> git annex get -J2 rh.white{,_avg}
get rh.white get rh.white_avg (transfer already in progress, or unable to take transfer lock)
Unable to access these remotes: web
(from web...)
Try making some of these repositories available:
00000000-0000-0000-0000-000000000001 -- web
5e47b3f3-f09c-4969-8885-920a49ff8a45 -- yoh@smaug:/mnt/btrfs/datasets/datalad/crawl/workshops/nih-workshop-2017/ds000114/derivatives/freesurfer
failed
/mnt/btrfs/scrap/tmp/ds0001 100%[===========================================>] 360.31K 1.63MB/s in 0.2s
2017-08-30 10:08:21 URL:https://dl.dropboxusercontent.com/s/0lww4tomnwfanwd/rh.white_avg?dl=0 [368962/368962] -> "/mnt/btrfs/scrap/tmp/ds000114/derivatives/freesurfer/.git/annex/tmp/MD5E-s368962--99a4db61cedffee686aef99b2d197794" [1]
(checksum...) ok
(recording state in git...)
git-annex: get: 1 failed
(dev)2 10018 ->1.....................................:Wed 30 Aug 2017 10:08:21 AM EDT:.
so at the end we get a run of git-annex which exits with error 1... and in json mode also the error(s) reported etc. I wondered if annex should first analyze passed paths to get actual keys to be fetched?
fixed; also fixed for several other commands, but the final fix needed each command that could have the problem to be modified, so there could possibly be some I missed.. --Joey
The only way I can see to improve this would be to keep track of which keys already have a thread working on them, and avoid a second thread working on the same key.
I've started this in the avoid-dup-threads branch.
Getting key information to commandAction would be quite the plumbing job; there are something like 50 call sites.
More difficult, the key is not known yet when commandAction is called in a lot of cases, and looking up the key redundantly will slow down all git-annex scanning. Seems that nontrivial changes would be needed to every command.
Hi Joey
I did remember that we had something similar but forgot that it was for "get" -- now I was getting similar problem for "add --jobs". Will this fix also work for it as well?
here, caught one for you for add (git annex version is tiny bit dated: 6.20170815+gitg22da64d0f-1~ndall+1 )
so it does report success in json, but complains in stderr that file is not found... I guess some race condition between workers so it manages to catch the moment when file is moved into a key or smth like that?
The add problem is clearly an entirely unrelated problem. Opened this bug report for it:
Another way to approach the problem would be, when the transfer of the same key is already in progress by another thread of the same process, wait for that thread to complete before running the requested transfer action.
The assistant has a TransferMap of all transfers the process is running. That would need to be moved from the DaemonStatus to Annex state.
To wait on the thread that's doing the transfer, would need to store a MVar or something in the TransferInfo; the ThreadId can't be waited on by itself.
This seems much less intrusive, and just as fast as my initial approach.