Ran git-annex copy --to origin -J4 in a repo with 100k files. origin was a local git repo. After all the files were transferred, it died:

copy bar (to origin...) (checksum...) ok
git-annex: git: createProcess: fork: resource exhausted (Resource temporarily unavailable)
-exit 1

.git/annex/journal/ had 76k files in it. git-annex merge dealt with it without a problem. --Joey

This is reproducible, and the second time was with slightly fewer files, probably around 60k. Some tries with around 10-20k files did not show any problem, never more than 5-10 processes, let alone a full fork bomb worth.

I did not see any evidence of processes building up shortly before it crashed, while it was still copying files. --Joey

Saw it again, after copying 10k files. --Joey

--debug shows a 8000+ git hash-object -w --stdin-paths --no-filters just before. Each process is logged as having exited successfully though, before the next starts, so how could this be a problem? However, it is surprising, it's supposed to write all the changes to just 1 process.

Even getting only 1000 files shows these extra processes, though not enough to crash it. Utility.CoProcess is restarting git hash-object due to an IO exception: "fd:12: hPutStr: illegal operation (handle is closed) I can't see anything that would close the handle early.

Reverted 7013798df5a161f00962985ffaea613a87cc4fe4 on a hunch, and that seems to have fixed it. Which is very weird, because AFAICS it was not getting an async exception. Even removing the catchAsync forcerestart is enough to avoid the problem though.

fixed though without full understanding of what that commit did that caused such strange behavior. --Joey