Please describe the problem.

While download content from dropbox (via regular urls associated with the files, so "web" remote), some fail some time with error code returned 500 (internal server failure without explicit reason), which causes git-annex get also to fail if that was the only source for the file to try.

What steps will reproduce the problem?

This is all happening with http://datasets.datalad.org/workshops/nih-2017/ds000114/derivatives/freesurfer/ repository (on eg https://dl.dropboxusercontent.com/s/sn4et1e3d2run9g/rh.aparc.dktatlas.annot?dl=0 url ), but for testing wget/curl I just created http://www.onerussian.com/tmp/errors/500 which would always return 500.

What version of git-annex are you using? On what operating system?

6.20180316+gitg308f3ecf6

Please provide any additional information below.

here are options for wget and curl which could help us out here:

# to make wget retry
> wget --retry-on-http-error=500  http://www.onerussian.com/tmp/errors/500

# to make curl retry, just needs --retry 10  which then considers 5xx errors intermittent
> curl --retry 10 http://www.onerussian.com/tmp/errors/500

Could git-annex add those options to curl/wget invocations for more robust access to the web remote?

done --Joey