Please describe the problem.

If a git-annex client from Debian buster (currently 6.20180913-1) tries to talk with a server running Debian stretch (currently 6.20170101-1+deb9u2), it will fail mysteriously:

copy bar (fd:18: hClose: resource vanished (Broken pipe)) failed

What steps will reproduce the problem?

On the stretch machine, setup a test git-annex repository.

On the buster machine, setup a test repository, add the stretch machine as a remote, then add a file in git-annex, and git-annex-sync-content to the stretch machine.

What version of git-annex are you using? On what operating system?

Debian buster and stretch standard Debian packages.

Please provide any additional information below.

On the buster machine:

# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

(test) [1265]anarcat@curie:test130$ git init test
Dépôt Git vide initialisé dans /home/anarcat/test/test/.git/
(test) [1266]anarcat@curie:test$ cd test
(test) [1267]anarcat@curie:test$ git annex init 
init  ok
(recording state in git...)
(test) [1268]anarcat@curie:test$ echo foo > bar
(test) [1269]anarcat@curie:test$ git annex add bar
add bar ok
(recording state in git...)
(test) [1270]anarcat@curie:test$ git remote add sal sal:test
(test) [1271]anarcat@curie:test$ git annex sync sal 
commit 
[master (commit racine) 05b6d24] git-annex in anarcat@curie:~/test/test
 1 file changed, 1 insertion(+)
 create mode 120000 bar
ok
pull sal 
warning: pas de commit commun
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 5 (delta 0), reused 0 (delta 0)
Dépaquetage des objets: 100% (5/5), fait.
Depuis sal:test
 * [nouvelle branche] git-annex  -> sal/git-annex
ok
(merging sal/git-annex into git-annex...)
(recording state in git...)
push sal 
Énumération des objets: 18, fait.
Décompte des objets: 100% (18/18), fait.
Compression par delta en utilisant jusqu'à 4 fils d'exécution
Compression des objets: 100% (12/12), fait.
Écriture des objets: 100% (16/16), 1.92 KiB | 654.00 KiB/s, fait.
Total 16 (delta 1), réutilisés 0 (delta 0)
To sal:test
 * [new branch]      git-annex -> synced/git-annex
 * [new branch]      master -> synced/master
ok
(test) [1272]anarcat@curie:test$ git annex sync --content sal
commit 
Sur la branche master
rien à valider, la copie de travail est propre
ok
pull sal 
ok
copy bar (fd:18: hClose: resource vanished (Broken pipe)) failed
git-annex: sync: 1 failed

# End of transcript or log.

The stretch repo is just git init test && git -C test annex init. SSH communication is setup with a passwordless SSH key in both. Debugging shows us the problem is with the p2pstdio command:

$ git annex sync --content sal --debug
[2018-11-03 15:25:35.018955156] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2018-11-03 15:25:35.022443588] process done ExitSuccess
[2018-11-03 15:25:35.022611687] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2018-11-03 15:25:35.02578792] process done ExitSuccess
[2018-11-03 15:25:35.026234842] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..24d62bcb5e253326bde98283338a855c9bfbe983","--pretty=%H","-n1"]
[2018-11-03 15:25:35.030089742] process done ExitSuccess
[2018-11-03 15:25:35.041384604] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2018-11-03 15:25:35.042220799] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
commit 
[2018-11-03 15:25:35.045907106] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","commit","-a","-m","git-annex in anarcat@curie:~/test/test"]
Sur la branche master
rien à valider, la copie de travail est propre
[2018-11-03 15:25:35.077421642] process done ExitFailure 1
ok
[2018-11-03 15:25:35.077523489] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
[2018-11-03 15:25:35.079008549] process done ExitSuccess
[2018-11-03 15:25:35.079088278] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
[2018-11-03 15:25:35.080639101] process done ExitSuccess
[2018-11-03 15:25:35.080766018] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/heads/synced/master"]
[2018-11-03 15:25:35.08225223] process done ExitSuccess
[2018-11-03 15:25:35.082343323] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/master..refs/heads/synced/master","--pretty=%H","-n1"]
[2018-11-03 15:25:35.084234147] process done ExitSuccess
pull sal 
[2018-11-03 15:25:35.084663095] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","fetch","sal"]
[2018-11-03 15:25:37.678014829] process done ExitSuccess
[2018-11-03 15:25:37.678131075] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","branch","-f","synced/master","refs/heads/master"]
[2018-11-03 15:25:37.68042014] process done ExitSuccess
[2018-11-03 15:25:37.680521541] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/sal/master"]
[2018-11-03 15:25:37.682164837] process done ExitFailure 1
[2018-11-03 15:25:37.682259432] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/sal/synced/master"]
[2018-11-03 15:25:37.683835717] process done ExitSuccess
[2018-11-03 15:25:37.683934116] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/synced/master..refs/remotes/sal/synced/master","--pretty=%H","-n1"]
[2018-11-03 15:25:37.686212308] process done ExitSuccess
ok
[2018-11-03 15:25:37.686343742] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2018-11-03 15:25:37.688215586] process done ExitSuccess
[2018-11-03 15:25:37.688305585] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2018-11-03 15:25:37.6901916] process done ExitSuccess
[2018-11-03 15:25:37.690393283] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..24d62bcb5e253326bde98283338a855c9bfbe983","--pretty=%H","-n1"]
[2018-11-03 15:25:37.692726875] process done ExitSuccess
[2018-11-03 15:25:37.692906438] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--cached","-z","--"]
copy bar [2018-11-03 15:25:37.696621566] chat: ssh ["sal","-S",".git/annex/ssh/sal","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","git-annex-shell 'p2pstdio' '/~/test' '--debug' 'f73778b3-17a4-4fee-9890-402bedf02456' --uuid 02ef3aa6-34d9-42b9-814e-f9ebc47c59e1"]
[2018-11-03 15:25:37.96931512] P2P > ERROR auth failed
(fd:18: hClose: resource vanished (Broken pipe)) failed
[2018-11-03 15:25:37.96973331] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
[2018-11-03 15:25:37.970341712] read: git ["--version"]
[2018-11-03 15:25:37.97296492] process done ExitSuccess
[2018-11-03 15:25:37.974463782] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","branch","-f","synced/master","refs/heads/master"]
[2018-11-03 15:25:37.978567382] process done ExitSuccess
[2018-11-03 15:25:37.979111982] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/sal/synced/master"]
[2018-11-03 15:25:37.982068735] process done ExitSuccess
[2018-11-03 15:25:37.982213309] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/remotes/sal/synced/master..refs/heads/synced/master","--pretty=%H","-n1"]
[2018-11-03 15:25:37.986086966] process done ExitSuccess
[2018-11-03 15:25:37.986239042] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--verify","-q","refs/remotes/sal/git-annex"]
[2018-11-03 15:25:37.988954838] process done ExitSuccess
[2018-11-03 15:25:37.989110109] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/remotes/sal/git-annex..git-annex","--pretty=%H","-n1"]
[2018-11-03 15:25:37.993956367] process done ExitSuccess
[2018-11-03 15:25:37.994340874] read: ssh ["-O","stop","-S","sal","-o","ControlMaster=auto","-o","ControlPersist=yes","localhost"]
[2018-11-03 15:25:38.00909442] process done ExitSuccess
[2018-11-03 15:25:38.009887194] process done ExitSuccess
[2018-11-03 15:25:38.010571056] process done ExitSuccess
[2018-11-03 15:25:38.011368991] process done ExitSuccess
git-annex: sync: 1 failed

A manual invocation confirms the p2p protocol not supported by the remote, probably because it's too old:

$ ssh sal git-annex-shell p2pstdio
fatal: unrecognized command 'p2pstdio'
git-annex-shell: git-shell failed

I understand the need for this command (which has no manpage, for what that's worth ;), but we should be able to fallback gracefully here. In the worst case, just treat this as a remote without git-annex-shell and revert to rsync to transfer files...

The workaround is to install the (working) backport in stretch, of course.

I wonder if this could be related to new git-annex-shell protocol hides remote error messages.

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Yes! I'm using git-annex like a maniac and I'm using it to archive Brazil right now. -- anarcat

presumed done --Joey