forum/external special remote protocol and threadsgit-annexhttp://git-annex.branchable.com/forum/external_special_remote_protocol_and_threads/git-annexikiwiki2015-10-04T07:49:02Zcomment 1http://git-annex.branchable.com/forum/external_special_remote_protocol_and_threads/comment_1_f29c2b0f0f5e349e36eb47176e33498f/joey2015-10-02T16:19:20Z2015-10-02T16:14:20Z
<p>Well, I can confirm that only one instance of any
external special remote will be run per git-annex process currently.</p>
<p>Actually, it's a bit more than that -- If multiple threads try to use the
same external special remote at the same time, everything is necessarily
serialized, and so using -J won't let multiple downloads happen at once
from a single external special remote, although it may still usefuly
parallelize amoung several remotes.</p>
<p>Since that seems like something worth dealing with, perhaps by having a
pool of external special remote processes, I don't feel comfortable making
any promises about the behavior. And as you note, it's easy to get multiple
procesess by running multiple git-annex commands, so that is something an
external special remote needs to deal with anyway.</p>
comment 2http://git-annex.branchable.com/forum/external_special_remote_protocol_and_threads/comment_2_df97223aad2c8c95ce109944d64a59fe/Carl2015-10-04T07:49:02Z2015-10-04T07:49:02Z
<p>Thanks for a clear answer. It seems that I am still in the sweet spot for using -J4. I.e transport everything to multiple remotes in parallel.</p>
<p>I do however take it as a recommendation to be careful, and will give it another consideration to be careful in case of doing more development work and releasing it.</p>