Please describe the problem.
Since cron build of 20210828
(git)smaug:/mnt/datasets/datalad/ci/git-annex/builds/2021/08[master]git
$> git grep -l 'Unable to remove all write permissions'
cron-20210828/build-macos.yaml-403-69466103-failed/2_test-annex (crippled-tmp).txt
cron-20210828/build-macos.yaml-403-69466103-failed/test-annex (crippled-tmp)/8_Run tests.txt
cron-20210829/build-macos.yaml-404-69466103-failed/2_test-annex (crippled-tmp).txt
cron-20210829/build-macos.yaml-404-69466103-failed/test-annex (crippled-tmp)/8_Run tests.txt
cron-20210830/build-macos.yaml-405-69466103-failed/2_test-annex (crippled-tmp).txt
cron-20210830/build-macos.yaml-405-69466103-failed/test-annex (crippled-tmp)/8_Run tests.txt
cron-20210831/build-macos.yaml-406-69466103-failed/2_test-annex (crippled-tmp).txt
cron-20210831/build-macos.yaml-406-69466103-failed/test-annex (crippled-tmp)/8_Run tests.txt
we got two test fails on a crippled FS on Mac (does not happen on linux afaik)
Both look like
2021-08-31T02:15:42.0758760Z magic: OK (2.41s)
2021-08-31T02:15:42.6972710Z import: FAIL (0.62s)
2021-08-31T02:15:42.6973680Z ./Test/Framework.hs:57:
2021-08-31T02:15:42.6974230Z import failed (transcript follows)
2021-08-31T02:15:42.6974760Z import import1/f
2021-08-31T02:15:42.6976570Z Unable to remove all write permissions from /Volumes/crippledfs/importtestvjfjz3/import1/f -- perhaps it has an xattr or ACL set.
2021-08-31T02:15:42.6977430Z failed
2021-08-31T02:15:42.6977830Z import: 1 failed
2021-08-31T02:15:44.1985050Z reinject: OK (1.50s)
here is the script to setup such a crippled (FAT32) FS on OSX.
Seems that mounting that way on OSX results in a FS where files are always mode 777 and the permissions cannot be changed.
When I tried using git-annex on such a FS, I saw:
And it skips the new permissions check when on a crippled filesystem.
But in that that test run, it seems it is failing to detect a crippled filesystem. Both because of the failure and also the test suite does not even run the "v8 unlocked" tests when it detects a crippled filesystem.
Is the test suite running as root? Looks like probably yes. Running as root prevents detecting the issue that made it use a crippled FS above. And it seems that, when a FAT fs is mounted on OSX that way, symlinks actually work (!!!) so the other crippled FS tests also don't notice a problem.
So, the fix should be for init to also test if it can remove the write bits from a file, and it should try that test even when root.
FWIW, it is a
runneruser ref (did in a temp PR) who is notrootbut is part of theadmingroup thus with super privileges indeed (that is why I guess we can also usehdiutildirectly to mount that crippled FS).My prior analysis seems right, as far as it running as root would go, but it is not running as root. So I missed something.
The test failures are both of
git-annex import. Otherwise locking down files does succeed. The difference with import must be that the file located in a directory outside the repository.Aha... The test suite is being run with TEMPDIR set to the crippled FS, but
.tis in another, non-crippled FS. A very smart idea to test that, although I think this import test is the only one that actually uses TEMPDIR. (Reading the workflow file, I think it was maybe expected that all the tests would run in TEMPDIR, but they don't;git-annex testwrites to./.t, other than this one test.When the import directory is on a crippled FS, and the repo is not, it will think the FS is not crippled. Then it fails to remove write perms from the file while it is in the import directory, and the perm check then fails.
So, I think it should skip the perm check when doing the initial lockdown of the file it's going to import.