ATM git-annex passes sftp:// urls to be handled with curl. But that one does not (re)use established ssh control sockets and/or credentials
E.g.
$> git annex addurl --debug sftp://localhost/tmp/123
[2018-01-12 09:53:35.195801836] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2018-01-12 09:53:35.199709648] process done ExitSuccess
[2018-01-12 09:53:35.199781091] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2018-01-12 09:53:35.203370157] process done ExitSuccess
[2018-01-12 09:53:35.204373706] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2018-01-12 09:53:35.204819259] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
addurl sftp://localhost/tmp/123 [2018-01-12 09:53:35.208782745] read: curl ["-s","--head","-L","sftp://localhost/tmp/123","-w","%{http_code}","--user-agent","git-annex/6.20171211+gitgba511f4de-1~ndall+1"]
[2018-01-12 09:53:35.290168768] process done ExitFailure 51
[2018-01-12 09:53:35.290835362] call: wget ["-nv","--show-progress","--clobber","-c","-O","/tmp/bu/.git/annex/tmp/URL--sftp&c%%localhost%tmp%123","sftp://localhost/tmp/123","--user-agent","git-annex/6.20171211+gitgba511f4de-1~ndall+1"]
sftp://localhost/tmp/123: Unsupported scheme ‘sftp’.
[2018-01-12 09:53:35.299065531] process done ExitFailure 1
failed
[2018-01-12 09:53:35.299781408] process done ExitSuccess
[2018-01-12 09:53:35.299993431] process done ExitSuccess
git-annex: addurl: 1 failed
So far we haven't managed to make curl behave sanely with sftp urls without us passing explicitly the login name and then a password and then it still failed:
$> curl sftp://localhost/tmp/123 -u yoh
Enter host password for user 'yoh':
curl: (51) SSL peer certificate or SSH remote key was not OK
I wondered if it would be feasible for git annex natively support ssh (scp) and/or sftp urls? All the machinery is there since it deals with copying files over ssh already.
I don't think that all the machinery is there, because for urls there needs to be a way to check the file size, and it's not clear how to do that with a sftp or ssh url.
what do you mean? why something as simple as "ssh host stat filepath" wouldn't work? or it seems e.g. libssh2 provides file size information via sftp: https://curl.haxx.se/mail/lib-2011-01/0094.html
Well, depends on coreutils and needs a stat(1) parser. Also, will sftp servers necessarily let arbitrary commands be run over ssh? Driving sftp interactive and parsing its ls -l would add more complexity to getting file size.
The url security fixes also mean these uris can't be used without relaxing the security policy.. Makes me wonder how much special-casing makes sense for such an edge feature.