Copying a large file from windows (msysgit 1.9.5.msysgit.1, git-annex 20150406) to a bare repository on debian wheezy (git 1.9.1 git-annex 20141024) I can only get around 2 MB/s transfer speed. I tested using normal windows copy (smb) and got around 10 MB/s, while the git-annex copy was still going on. I tried plain ssh and got 9 MB/s. So, something to do with git-annex is being slow it seems. Any ideas on how to remedy this, or is it a known issue?
Note: Problem not seen going from linux to linux... waaaayyy faster (40 MB/s) to put it lightly (10 min vs all day) The destination server is the same.
There's not a lot of places where git-annex could make this slower. git-annex copy is just running rsync to the ssh server.
Have you tried benchmarking rsync of a large file to the server w/o git-annex? rsync does do considerably more client-side work than does scp, in order to support resuming, so that might be it.
Well, git-annex bundles a copy of rsync, and the one that it includes is the one from cygwin.
It sounds like there's a newer version, also from cygwin, that is faster?
I've pinged the git-annex Windows autobuilder admin to see if we can upgrade that.
https://www.itefix.net/cwrsync
Hmm, ok, so that's not the cygwin rsync and it has a gui and bundles its own ssh that won't behave the same as the one bundled with git-annex.
Maybe at least some of the speed improvements could be had by just upgrading the rsync and ssh from cygwin though..