Please describe the problem.
actually it fails whenever HOME is crippled (not /tmp)
2021-01-22T03:59:51.9212395Z Unit Tests v8 adjusted unlocked branch
2021-01-22T03:59:51.9218586Z add dup: Init Tests
2021-01-22T03:59:52.1497485Z init: OK (0.23s)
2021-01-22T03:59:53.3199842Z add: OK (1.17s)
2021-01-22T03:59:53.3200475Z
2021-01-22T03:59:53.3200888Z All 2 tests passed (1.40s)
2021-01-22T03:59:53.9335540Z OK (0.61s)
2021-01-22T03:59:54.2385075Z add extras: OK (0.30s)
2021-01-22T03:59:54.8735634Z ignore deleted files: OK (0.63s)
2021-01-22T03:59:55.7703551Z metadata: OK (0.90s)
2021-01-22T03:59:58.5056762Z export_import: FAIL (2.74s)
2021-01-22T03:59:58.5061241Z Test.hs:1884:
2021-01-22T03:59:58.5061997Z did not find expected content of dir/import
2021-01-22T04:00:00.8296862Z export_import_subdir: OK (2.32s)
2021-01-22T04:00:01.1359269Z shared clone: OK (0.31s)
2021-01-22T04:00:01.4846753Z log: OK (0.35s)
from https://github.com/datalad/git-annex/runs/1746481387?check_suite_focus=true on 8.20201129+git169-gaa07e68ed-1~ndall+1 (presumably... identified 3 regressions today, may be we have a problem with our CI setup and building some old annex?)
Reproduced on fat with the current version. (You didn't say what your filesystem was however.)
Seems quite possibly related to the recent change in 73df633a6215faa093e4a7524c6d328aa988aed1. Since that made the inode not be considered when importing, and since FAT has very low resolution timestamps (1s), repeated modifications of a file that don't change its size could not be noticed when repeatedly importing.
But hmm, the test suite already guards against such a thing by making sure, when writing a new value to a file, that the modification time is different than before.
Confirmed that a sleep at the right place in the test suite avoids the failure. Unsure yet why the test suite's existing mtime managing didn't manage to.
I do think this should be fixed in the test suite; it's acceptable that, if using a FS like FAT that doesn't support high-rest mtimes, git-annex not deal with high-frequency modifications. The same behavior could already have happened on FAT when it was including the inode, if the modifications were done w/o overwriting changing the inode.
.git/configfiles:gitseems to rewrite those files (so inode changes) which provided us a nice consistent indication and avoided crippled systems workarounds. Otherwise we were hitting all kinds of weird/hard to debug corner cases.Oh this is interesting:
So despite seeing the file is modified when it is asked to remove it, it somehow does not notice that when writing to it, so overwrites it with the content anyway. That's how it ends up with unexpected content, despite the overall
git-annex exportfailing as expected.And the root cause is, the test suite writes to the file on the remote, and then Remote.Directory writes to a temp file, and then gets its content identifier (mtime), and for some reason accepts that content identifier as one it can overwrite. Due to the timestamp resolution, that's the same content identifier as the newly written file on the remote. So it overwrites it.
I can't find any good reason for it to accept the temp file's content identifier as a known content identifier, which it's always done. Other remotes don't do that. So I think this is a real bug in Remote.Directory and will fix it there.