Please describe the problem.
Finally got to try https://git-annex.branchable.com/todo/lockdown_hooks/ solution to avoid adjusted branches on local HPC which uses ACLs on some (but not all) partitions/etc.
For the use-case at hands I figured how to "freeze" and "thaw" to be done via nfs4_{s,g}etfacl
. Here are my two scripts which also dump debug output to stderr
freeze-content -- adds "D::EVERYONE@:wadTN" at index 1
#!/bin/bash
set -eu
{
echo "D: freezing [$@] while under `pwd`"
ls -ld "$@"
# Cumbersome and buggy -- I found no other way to just say "remove write bits"
#nfs4_getfacl "$@" | sed -e 's,:r[^:]*$,:rxtncy,g' | nfs4_setfacl -S- "$@"
# This should be first and thus preventing any changes to that path
# 'thaw'ing would just remove that rule
nfs4_setfacl -a D::EVERYONE@:wadTN 1 "$@"
echo "D: finished freezing ok. Current ACL:"
nfs4_getfacl "$@"
} >&2
thaw-content -- removes that ACL
#!/bin/bash
set -eu
echo "unlocking [$@] while under `pwd`" >&2
#nfs4_getfacl "$@" | sed -e 's,:r[^:]*$,:rxtncy,g' | nfs4_setfacl -S- "$@"
nfs4_setfacl -x D::EVERYONE@:wadTN "$@"
echo "finished locking ok. ACL:" >&2
nfs4_getfacl "$@" >&1
and I verified that it does work as expected/desired:
on a directory
(git-annex) [d31548v@discovery7 tmp]$ mkdir testdir && ~/bin-annex/freeze-content testdir
D: freezing [testdir] while under /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp
drwxrwx--- 2 d31548v rc-CANlab-admin 0 Feb 23 15:48 testdir
D: finished freezing ok. Current ACL:
# file: testdir
D::EVERYONE@:wadTN
A::OWNER@:rwadxtTnNcy
A:fdi:OWNER@:rwadxtTnNcy
A:fd:GROUP@:rwaDdxtTnNcCoy
A:fdg:rc-CANlab@KIEWIT.DARTMOUTH.EDU:rxtncy
(git-annex) [d31548v@discovery7 tmp]$ touch testdir/123
touch: cannot touch 'testdir/123': Permission denied
(git-annex) [d31548v@discovery7 tmp]$ rm -rf testdir
rm: cannot remove 'testdir': Permission denied
(git-annex) [d31548v@discovery7 tmp]$ ~/bin-annex/thaw-content testdir && rm -rf testdir
unlocking [testdir] while under /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp
finished locking ok. ACL:
# file: testdir
A::OWNER@:rwadxtTnNcy
A:fdi:OWNER@:rwadxtTnNcy
A:fd:GROUP@:rwaDdxtTnNcCoy
A:fdg:rc-CANlab@KIEWIT.DARTMOUTH.EDU:rxtncy
(git-annex) [d31548v@discovery7 tmp]$ ls -ld testdir
ls: cannot access testdir: No such file or directory
and a file
(git-annex) [d31548v@discovery7 tmp]$ touch testfile && ~/bin-annex/freeze-content testfile
D: freezing [testfile] while under /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp
-rwxrwx--- 1 d31548v rc-CANlab-admin 0 Feb 23 15:49 testfile
D: finished freezing ok. Current ACL:
# file: testfile
D::EVERYONE@:wadTN
A::OWNER@:rwadxtTnNcy
A::GROUP@:rwadxtTnNcCoy
A:g:rc-CANlab@KIEWIT.DARTMOUTH.EDU:rxtncy
(git-annex) [d31548v@discovery7 tmp]$ echo 123 >> testfile
bash: testfile: Permission denied
(git-annex) [d31548v@discovery7 tmp]$ rm testfile
rm: remove write-protected regular empty file 'testfile'? y
rm: cannot remove 'testfile': Permission denied
(git-annex) [d31548v@discovery7 tmp]$ ~/bin-annex/thaw-content testfile && rm -rf testfile
unlocking [testfile] while under /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp
finished locking ok. ACL:
# file: testfile
A::OWNER@:rwadxtTnNcy
A::GROUP@:rwadxtTnNcCoy
A:g:rc-CANlab@KIEWIT.DARTMOUTH.EDU:rxtncy
(git-annex) [d31548v@discovery7 tmp]$ ls -ld testfile
ls: cannot access testfile: No such file or directory
But that still dissatisfies git-annex
(10.20220127-gf616630):
(git-annex) [d31548v@discovery7 tmp]$ d=repo; chmod +r -R "$d" ; rm -rf "$d"; mkdir -p "$d" && cd $d && git init && git -c annex.freezecontent-command="$HOME/bin-annex/freeze-content %path" -c annex.thawcontent-command="$HOME/bin-annex/thaw-content %path" annex init
chmod: cannot access 'repo': No such file or directory
Initialized empty Git repository in /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp/repo/.git/
init D: freezing [.git/annex/misctmp/gaprobe] while under /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp/repo
-rwxrwx--- 1 d31548v rc-CANlab-admin 0 Feb 23 15:51 .git/annex/misctmp/gaprobe
D: finished freezing ok. Current ACL:
# file: .git/annex/misctmp/gaprobe
D::EVERYONE@:wadTN
A::OWNER@:rwadxtTnNcy
A::GROUP@:rwadxtTnNcCoy
A:g:rc-CANlab@KIEWIT.DARTMOUTH.EDU:rxtncy
unlocking [.git/annex/misctmp/gaprobe] while under /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp/repo
finished locking ok. ACL:
# file: .git/annex/misctmp/gaprobe
A::OWNER@:rwadxtTnNcy
A::GROUP@:rwadxtTnNcCoy
A:g:rc-CANlab@KIEWIT.DARTMOUTH.EDU:rxtncy
Filesystem does not allow removing write bit from files.
Detected a crippled filesystem.
Disabling core.symlinks.
Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
Switched to branch 'adjusted/master(unlocked)'
ok
(recording state in git...)
may be it is because at POSIX level write bits are still there? e.g. here is with the file
(git-annex) [d31548v@discovery7 repo]$ touch testfile && ~/bin-annex/freeze-content testfile && ls -l testfile
D: freezing [testfile] while under /dartfs-hpc/rc/lab/C/CANlab/labdata/data/tmp/repo
-rwxrwx--- 1 d31548v rc-CANlab-admin 0 Feb 23 15:51 testfile
D: finished freezing ok. Current ACL:
# file: testfile
D::EVERYONE@:wadTN
A::OWNER@:rwadxtTnNcy
A::GROUP@:rwadxtTnNcCoy
A:g:rc-CANlab@KIEWIT.DARTMOUTH.EDU:rxtncy
-rwxrwx--- 1 d31548v rc-CANlab-admin 0 Feb 23 15:51 testfile
filing as a bug since I think I satisfied all desires of git-annex for ensuring that target path is frozen, but it is still not satisfied
edit 1: I thought that may be "Locking down a directory only needs to do the equivilant of removing its write bit, does not need to lockdown the files within it." from lockdown_hooks does not really "work", so I added -R
to above nfs4_facl calls, but seems the effect is the same -- annex doesn' like me
ok, if instead I do use more fragile version where I do adjust
A:
ACLs instead of just adding theD:
here are the two commands to freeze/thaw
then I do get
ls -l
reporting that write bit is not set, andgit annex
seems to be happy then... heh. So I guess some recipe like this could be used , but I still would prefer to avoid needing in-place changes of such "detailed" permissions listings in those ACLs.which results in permission denied on removal of the key (annex 10.20220127-gf616630) -- why should it freeze the key if it is about to be dropped?
as a workaround I added
-R
to my thawing script so thawing of keydir re-thaws the key as well, but that doesn't feel right.edit: interestingly,
drop
also thaws the file in index (e.g../sub-02/anat/sub-02_inplaneT2.nii.gz
in my case) -- why does it in "indirect" mode? hm...Looks like, while init does run the freeze hook in its test, as well as trying to remove the write bits (which it always does, freeze hook or no freeze hook), it still checks if the file's write permissions got removed.
I think that, in order to support filesystems where chmod -w has no effect, init should skip looking at write perms when there is a freeze hook. It can still check that freezing prevents writing to a file.
Hmm, init uses checkContentWritePerm to see if the write permission is removed. That is also used by fsck, and when ingesting a file. So even if it were fixed in init, those would still be problems. This really needs to be fixed in checkContentWritePerm.
I think that this issue with drop freezing the content needs to be treated as a separate bug. There also may be a good reason for it to do it, I have not looked into that yet.
Fixed it to ignore remaining write perms when annex.freezecontent-command is set.
I've added --debug output for both freezing and thawing.
The behavior will vary between v8 and v10 because of changes to locking; in v8 it has to thaw the content file before it can lock it, and then it will freeze it back after locking and before removing it.
In v8 drop looks like:
In v10 drop looks like:
So that's more or less as I expected (aside from the thawing of the worktree symlink foo which is kind of weird).
You said "drop freezes thawn key file before thawing key dir" and yes, that's the v8 behavior shown above.
The reason is safety: If it left the content file thawed after taking the posix lock of it, but then turned out to be unable to remove it (could not lock enough other copies of the object in other repos), it would then need to re-freeze it since it failed to drop it. But waiting until that point to re-freeze it would leave a perhaps long stretch of time where it was thawed and vulnerable to being written to. And also if it were interrupted before it could re-freeze it would leave it thawed until fsck finally noticed and fixed the problem. So I don't feel what it's doing is wrong with respect to thawing and then freezing in v8.
Notice that, in v10, it thaws the content directory, but never directly thaws the content file. So if the content file remaining frozen prevents removing it, that would also affect v10.
This does complicate freeze hooks that do things that prevent removal of the file, rather than just preventing write to the file. Of course those are two different permissions in classical unix permissions, but not necessarily with your ACLs or with immutability bits etc.
So, I think that what git-annex ought to do is, when there's a thaw hook, thaw the content file just before unlinking it.
Implemented that. Should solve your second problem.
This turns out to be due to removeAnnex's resetpointer. It was not intended to run on symlinks. I've fixed this too.