Please describe the problem.
For many collaborative workflows it is desired to provide read-only access to a git-annex repository on a remote ssh host.
But apparently, even if user does have read access to the content, git-annex (shell?) tries to lock the entire remote repository leading to a crash:
lena:/tmp
$> git clone localhost:/tmp/testroot testroot-clone
Cloning into 'testroot-clone'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 19 (delta 1), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (19/19), done.
Resolving deltas: 100% (1/1), done.
$> git -C testroot-clone annex get .
get 123 (from origin...)
git-annex: /tmp/testroot/.git/annex: openTempFile: permission denied (Permission denied)
p2pstdio: 1 failed
Lost connection (fd:27: hGetChar: end of file)
Transfer failed
Unable to access these remotes: origin
No other repository is known to contain the file.
failed
get: 1 failed
and original user reported a little different error (I have no access to see the details of that repository):
SENSORED's password:
[2021-08-25 15:20:24.14821] (Utility.Process) process [2554781] done ExitSuccess
[2021-08-25 15:20:24.1493] (Utility.Process) process [2555432] chat: ssh ["SENSORED","-S",".git/annex/ssh/SENSORED","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","git-annex-shell 'p2pstdio' '/SENSORED' '--debug' '94ed180c-3f09-4b5b-a4c6-68b3a8acf900' --uuid d3e54e78-143b-4cee-89b6-217439bdb2e3"]
[2021-08-25 15:20:24.883791] (P2P.IO) [ssh connection Just 2555432] [ThreadId 4] P2P < AUTH-SUCCESS d3e54e78-143b-4cee-89b6-217439bdb2e3
[2021-08-25 15:20:24.883883] (P2P.IO) [ssh connection Just 2555432] [ThreadId 4] P2P > VERSION 1
[2021-08-25 15:20:24.904358] (P2P.IO) [ssh connection Just 2555432] [ThreadId 4] P2P < VERSION 1
[2021-08-25 15:20:24.904441] (P2P.IO) [ssh connection Just 2555432] [ThreadId 4] P2P > GET 0 2635065_20217_2_0.zip MD5E-s229347386--c8ba7095fa69c25cffeeaddf68124ea9.zip
[2021-08-25 15:20:24.91475] (P2P.IO) [ThreadId 4] P2P < GET 0 2635065_20217_2_0.zip MD5E-s229347386--c8ba7095fa69c25cffeeaddf68124ea9.zip
git-annex: /SENSORED/.git/annex/keysdb.lck: openFd: permission denied (Permission denied)
p2pstdio: 1 failed
Lost connection (fd:22: hGetChar: end of file)
[2021-08-25 15:20:25.019767] (Utility.Process) process [2555432] done ExitFailure 1
Transfer failed
[2021-08-25 15:20:25.02038] (Utility.Process) process [2553967] done ExitSuccess
[2021-08-25 15:20:25.020634] (Utility.Process) process [2553968] done ExitSuccess
...
this severely limits git-annex usability for many collaboration scenarios. I just wonder how come we did not run into this before...
I guess the git-annex-shell must not lock anything until some write operation is really necessary.
What steps will reproduce the problem?
create a git-annex repo owned by another user, provide permissions for others to read, try to get any annexed content via ssh
What version of git-annex are you using? On what operating system?
8.20210803-g07dcb9c
This did used to work, see read-only removable drives which ended with it working in 2016.
There is a way to do it that's certian to work: Make a ssh key that logs into the host as the user who owns the repository, and lock that ssh key down to run git-annex-shell with
GIT_ANNEX_SHELL_READONLY=1 GIT_ANNEX_SHELL_LIMITED=1Still this should be investigated. Since one failure is certianly due to the keys database update, it seems likely that's what's changed.
I can reproduce the keysdb.lck failure by simply making a new repository, owned by root, cloning it over ssh, running
git-annex init(succeeds) and thengit-annex get(fails).keysdb error was a reversion in f46e4c9b7c, which reversed some logic and made database reads be treated as database writes, so it failed on permission denied, rather than the right behavior of returning DbUnavailable when it cannot read or the database does not exist yet.
Fixed that. It seems to work for me now, both get and drop, and over ssh or direct access to the readonly repository.
I have not been able to reproduce the other error involving opening a temp file. All I can tell from the message is that it's writing to some file in .git/annex via a temp file in the same directory, but I don't know what file it is trying to write. I've made a change that will include more information in that error message. So either update your build of git-annex, or find a better reproduction recipe for that..
if of any help -- output from --debug: ``` [2021-08-30 13:59:19.423059134] (Utility.Process) process [2353815] chat: ssh ["localhost","-S",".git/annex/ssh/localhost","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","git-annex-shell 'p2pstdio' '/tmp/testroot' '--debug' 'b438aaf1-df6e-4dcd-8342-6b4c3490ddc2' --uuid e5fb275a-f2d6-45ec-89e0-be9c5737a985"] [2021-08-30 13:59:19.468653141] (P2P.IO) [ssh connection Just 2353815] [ThreadId 4] P2P < AUTH-SUCCESS e5fb275a-f2d6-45ec-89e0-be9c5737a985 [2021-08-30 13:59:19.468876656] (P2P.IO) [ssh connection Just 2353815] [ThreadId 4] P2P > VERSION 1 [2021-08-30 13:59:19.469785069] (P2P.IO) [ssh connection Just 2353815] [ThreadId 4] P2P < VERSION 1 [2021-08-30 13:59:19.470001878] (P2P.IO) [ssh connection Just 2353815] [ThreadId 4] P2P > GET 0 123 MD5E-s4--ba1f2511fc30423bdbb183fe33f3dd0f [2021-08-30 13:59:19.470733309] (P2P.IO) [ThreadId 4] P2P < GET 0 123 MD5E-s4--ba1f2511fc30423bdbb183fe33f3dd0f git-annex: /tmp/testroot/.git/annex: openTempFile: permission denied (Permission denied)
!/bin/bash
export PS4='> ' set -eu
set -x
d="$(sudo mktemp -d ${TMPDIR:-/tmp}/ann-XXXXXXX)" dclone="$d-clone"
umask 022
sudo git -C $d init sudo git -C $d annex init sudo bash -c "echo 123 > $d/123" sudo git -C $d annex add $d/123 sudo git -C $d commit -m 123 sudo chmod go+rX -R "$d"
git clone "$d" "$dclone" git -C "$dclone" annex get .
```
Well I've already fixed their problem AFAICS. Just perhaps not the problem you are seeing.
Your script does not reproduce any problem here. Are you running it on NFS or something?
Anyway, if you can build current master the improved error message will probably help me find the problem.
I'm going to close this, since I was able to replicate and fix one of the errors, and was not able to replicate the other one. If the other one still happens, follow up with the output of the current version of git-annex, as I've requested 3 times now.
There is a test case now that tests the same thing, too. It has passed on all the builders. (Except windows. I've learned git-annex cannot support readonly repos on Windows, because it has to be able to create lock files there.)
(There are still situations where git-annex needs to write to a repo in order to use it. The main one is if the it needs to upgrade the repo. I think failing in that specific situation is ok, and it fails with an informative error message IIRC. It's also possible there's some other situation of some kind where the readonly repo is in a state where git-annex wants to write to it, but if there is, I can fix it when the situation comes to light.)
The user does not need to do anything, I have reproduced and fixed their problem.
You reported a different error message when you tried to reproduce it yourself. That is the only thing i do not understand, and why I have, repeatedly, been trying to ask you to reproduce that error message again.