Is it "safe" to run git-annex addurl or git-annex registerurl in batch mode on a git-annex repository while also occasionally executing git rm on that repository while the batch process is running? By "safe", I mean "the git rm process will never crash due to the git lock file already existing." Is there a way to ensure that a git rm invocation will not fail due to locking issues while keeping the batch process running?
The only relevant information I can find on this is https://git-annex.branchable.com/forum/locks_during_--batch_operations/, which states that a batch process will not hold a single lock throughout its entire run but says nothing about whether a batch process uses a lock at all.
(I am asking about this because we have a program that runs batch processes concurrently with git rm invocations, and very rarely there are crashes due to locking issues with git rm. My boss now wants me to address these issues by shutting down any running batch processes before running git rm, but this will be a giant hassle that I wish to avoid if at all possible.)
The lock you need to worry about here is for
.git/index, which bothgit rmandgit-annex addurlneed to modify (when the latter is adding new files, not adding urls to existing files). Unfortunately git's handling of that lock file doesn't play well with multiple processes (unlike git-annex's other lock files that it uses).With --batch, git-annex reads a line, handles it, outputs a response, and waits for the next line. It should not keep any lock held while it's waiting for the line. So, you should be able to schedule your
git rmto happen at that point and not have problems with locks colliding.registerurlalso subject to the same locking or onlyaddurl?registerurldoes not touch the git index so uses the usual git-annex locking that supports concurrency.Interestingly,
git commitdoes not hold the index.lock while waiting for the editor anymore. Despite git's lock failure message still suggesting that as the most common reason for a lock to fail. So the window for this race is narrower than it used to be. (Also this surprisingly means that you cangit add(orgit-annex addwhilegit commitis in the editor and stage files that will be included in the ongoing commit!)Also the index lock is just the git lock that I've seen be a problem before. git does have some other locks that could in some situation be a problem with concurrency. See core.filesreflocktimeout for one example.
git-annex's locking method is not without problems either, see withExclusiveLock blocking issue.