Please describe the problem.
Since a couple of days I can't move files to or get files from a remote over ssh.
I'm guessing that it has something to do with the new release 10.20220127 release that I'm using locally, as I haven't changed a single thing in the setup, and now it doesn't work anymore. I also have a different computer with version 6.20180227, and that one can access the remote without any problems.
Maybe it's also that I have an older git-annex version installed on it (again 6.20180227). The remote server is basically an old small ARM processor running on Ubuntu 18.04 using an SSD card, and that's the newest version that's available.
What steps will reproduce the problem?
The same effect happens when I'm creating a new repository. Everything, including syncing works. But as soon as I'm moving files, errors pop up.
$ git init
$ git annex init
$ echo 123 > file
$ git annex add file
$ git commit -m "added file"
$ # remotely at host rock64 (details specified in .ssh/config) create a a directory at /mnt/wd8ext/remote-annex and git init it
$ git remote add wd8ext ssh://rock64/mnt/wd8ext/remote-annex
$ git annex sync
$ git annex --verbose --debug move --to wd8ext file
[2022-02-08 21:34:58.211337603] (Utility.Process) process [426973] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2022-02-08 21:34:58.212514973] (Utility.Process) process [426973] done ExitSuccess
[2022-02-08 21:34:58.212846926] (Utility.Process) process [426974] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2022-02-08 21:34:58.21382918] (Utility.Process) process [426974] done ExitSuccess
[2022-02-08 21:34:58.214613297] (Utility.Process) process [426975] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..81758130587469be97e82c11cb4fc39ebcab2c35","--pretty=%H","-n1"]
[2022-02-08 21:34:58.215981386] (Utility.Process) process [426975] done ExitSuccess
[2022-02-08 21:34:58.217035304] (Utility.Process) process [426976] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
[2022-02-08 21:34:58.223715442] (Utility.Process) process [426977] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","symbolic-ref","-q","HEAD"]
[2022-02-08 21:34:58.22483633] (Utility.Process) process [426977] done ExitSuccess
[2022-02-08 21:34:58.225122771] (Utility.Process) process [426978] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","refs/heads/master"]
[2022-02-08 21:34:58.226399383] (Utility.Process) process [426978] done ExitSuccess
[2022-02-08 21:34:58.226786503] (Utility.Process) process [426979] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--","file"]
[2022-02-08 21:34:58.22728136] (Utility.Process) process [426980] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2022-02-08 21:34:58.227992286] (Utility.Process) process [426981] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2022-02-08 21:34:58.228323732] (Utility.Process) process [426976] done ExitSuccess
[2022-02-08 21:34:58.229063353] (Utility.Process) process [426982] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
move file
You have enabled concurrency, but git-annex is not able to use ssh connection caching. This may result in multiple ssh processes prompting for passwords at the same time.
annex.sshcaching is not set to true
[2022-02-08 21:34:58.232369632] (Utility.Process) process [426984] chat: ssh ["rock64","-T","git-annex-shell 'p2pstdio' '/mnt/wd8ext/remote-annex' '--debug' '73dd11e5-3911-4269-9ef9-5f2796f68dd4' --uuid eea8c03e-1fa2-4998-8c1b-a47fbad2e37e"]
fatal: Run with no arguments or with -c cmd
git-annex-shell: git-shell failed
[2022-02-08 21:34:59.990281774] (Utility.Process) process [426984] done ExitFailure 1
(unable to check wd8ext) failed
[2022-02-08 21:34:59.990887594] (Utility.Process) process [426982] done ExitSuccess
[2022-02-08 21:34:59.991067464] (Utility.Process) process [426981] done ExitSuccess
[2022-02-08 21:34:59.991183308] (Utility.Process) process [426980] done ExitSuccess
[2022-02-08 21:34:59.991280532] (Utility.Process) process [426979] done ExitSuccess
move: 1 failed
What version of git-annex are you using? On what operating system?
Locally
Operating system: Manjaro Linux 21.2.3
$ git annex version
git-annex version: 10.20220127-ge6e60b644
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-9.0.2 http-client-0.7.10 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
local repository version: 8
Remote at rock64
Operating System: Ubuntu 18.04.5 LTS
$ git annex version
git-annex version: 6.20180227
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.7.1 persistent-sqlite-2.6.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
Please provide any additional information below.
Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes, everything worked perfectly (using git-annex since at least 2 years).
From the NEWS file:
So you will need to upgrade your remote.
I'm leaving this bug report open because it currently fails without explaining why, and it would be good to indicate that the remote is too old if possible. (I am not, however, unsure if it's actually possible to do that.)
Both local and remote git annex has version 10.20220127-ge6e60b644.
Initially both my local and remote repos had version 8. I used
git annex upgradeto upgrade them to 9 then 10, but the problem persisted.I tried running the
git-annex-shell p2pstdiocommand in the server shell after sshing in and it worked. I also tried a different git-annex-shell command (configlist) remotely from local withssh -T, and it worked too.But when I do p2pstdio remotely with
ssh -TI get@ycp it is very possible that you also have an older version of git-annex installed on the server, in such a way that the newer version is in PATH when you log in interactively, but the older version is when you log in non-interactively.
See https://git-annex.branchable.com/tips/get_git-annex-shell_into_PATH/
Note that "Run with no arguments or with -c cmd" is not a git-annex message. It is a message output by git-shell, eg:
This probably means that git-annex-shell, not understanding the "p2pstdio" command, has passed it on to git-shell, which failed like that. Which hints very strongly that you do indeed have an old git-annex-shell in PATH at that point.
Same problem here: on my synology NAS I'm stuck at git-annex version 6.20161212 (later releases fail to run) and I recently (blindly) updated git-annex on my macbook with homebrew to 10.20220624 (I don't remember what previous version was installed).
Because upgrading git-annex at the NAS-side is not an option at the moment, I had to downgrade git-annex locally to 8.20211011. This was quite a challenge unfortunately, and I documented it on my blog (in case anybody else might face the same uphill battle)
I'm sorry this caused you that trouble. But 6.20161212 is really extremely old, and supporting the quirks of old versions of git-annex forever in current versions is not something I want to take on.
No worries, @joey, I fully understand that you sometimes have to make breaking changes. 2016 is indeed really ages ago, but my NAS is currently stuck there. My blog post probably comes across more "complainy" than I intended. My point is actually that thanks to the "open" in open source, I eventually managed to cook up a solution.