Please describe the problem.
Recently I found that two repo tests, namely 'concurrent get of dup key regression' and 'import', fail due to locking problems. I bisected this on my Windows laptop and found that 5a98f2d50913682c4ebe0e0c4ce695c450a96091 is the first bad commit.
What steps will reproduce the problem?
stack setup && stack build
. Then copy git-annex.exe to C:\annx
and in (Git) Bash say:
$ ./git-annex test -p 'Repo Tests' 2>&1 | tee git-annex.test--p-Repo_Tests.LOG~202
$ grep -E '(concurrent get of dup key regression|import):' git-annex.test--p-Repo_Tests.LOG~202
Observe two FAILs.
(It is unfortunate that you can't call for instance ./git-annex.exe test -p 'concurrent get of dup key regression'
directly because of an unrelated setup issue with Tasty. I guess this would warrant a separate bug report.)
Please provide any additional information below.
Under Windows with 10.20220505-g5a98f2d50:
jkniiv@AINESIS MINGW64 /c/annx
$ cat git-annex.test--p-Repo_Tests.LOG~202
All 0 tests passed (0.00s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (10.30s)
add: OK (14.20s)
borg remote: OK
uninit: OK (15.54s)
conflict resolution (mixed directory and file): OK (154.59s)
concurrent get of dup key regression: FAIL (18.41s)
.\\Test\\Framework.hs:70:
drop with dup failed (transcript follows)
drop foo ok
drop foo2
failed
(recording state in git...)
git-annex: content is locked
drop: 1 failed
Use -p '/Repo Tests/&&/concurrent get of dup key regression/' to rerun this test only.
migrate (via gitattributes): OK (35.88s)
fsck (basics): OK (33.18s)
copy: OK (29.05s)
drop (no remote): OK (18.30s)
shared clone: OK (8.98s)
1 out of 11 tests failed (338.52s)
All 0 tests passed (0.03s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (10.26s)
add: gpg testing not implemented on Windows
OK (13.95s)
crypto: OK
uninit (in git-annex branch): OK (12.13s)
conflict resolution symlink bit: OK
union merge regression: OK (148.89s)
unused: OK (59.95s)
fsck (bare): OK (5.63s)
lock: OK (29.89s)
drop (with remote): OK (22.18s)
log: OK (10.16s)
add dup: OK (15.61s)
All 12 tests passed (328.74s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (10.27s)
add: OK (14.07s)
rsync remote: OK (8.80s)
conflict resolution (mixed locked and unlocked file): OK (70.62s)
conflict resolution (adjusted branch): OK (92.89s)
version: OK (10.99s)
conversion annexed to git: OK (18.28s)
fix: OK (9.54s)
move (ssh remote): OK
unannex (no copy): OK (12.49s)
export_import: OK (88.90s)
All 11 tests passed (336.95s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (9.80s)
add: OK (13.22s)
add subdirs: OK (21.85s)
hook remote: OK (9.82s)
conflict resolution (nonannexed file): OK (133.39s)
transition propagation: OK (139.19s)
merge: OK (11.89s)
fsck --from remote: OK (10.01s)
edit (pre-commit): OK (15.37s)
get (ssh remote): OK
import: FAIL (20.87s)
.\\Test\\Framework.hs:70:
drop failed (transcript follows)
drop import1/f ok
drop import2/f
failed
drop import5/f
failed
(recording state in git...)
git-annex: content is locked
git-annex: content is locked
drop: 2 failed
Use -p '/Repo Tests/&&/import/' to rerun this test only.
ignore deleted files: OK (8.43s)
1 out of 12 tests failed (393.92s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (10.27s)
add: OK (14.14s)
bup remote: OK (8.85s)
map: OK (12.69s)
conflict resolution movein regression: OK (92.81s)
sync: OK (58.83s)
migrate: OK (35.32s)
trust: OK (30.37s)
move (numcopies): OK (40.73s)
unannex (with copy): OK (14.55s)
export_import_subdir: OK (56.87s)
All 11 tests passed (375.49s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (8.97s)
add: OK (13.79s)
preferred content: OK (63.02s)
upgrade: OK (10.77s)
conflict resolution (uncommitted local file): OK (101.39s)
adjusted branch merge regression: OK (19.12s)
describe: OK (12.57s)
fsck (local untrusted): OK (26.13s)
lock --force: OK (16.35s)
drop (untrusted remote): OK (24.97s)
view: OK (15.62s)
add extras: OK (13.99s)
All 12 tests passed (326.79s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (10.67s)
add: OK (13.44s)
addurl: OK (16.86s)
directory remote: OK (32.54s)
conflict resolution (nonannexed symlink): OK (40.41s)
conflict resolution: OK (86.32s)
info: OK (13.57s)
conversion git to annexed: OK (19.97s)
partial commit: OK (20.86s)
move: OK (41.33s)
reinject: OK (16.08s)
metadata: OK (23.76s)
All 12 tests passed (335.87s)
Tests
Repo Tests v8 adjusted unlocked branch
Init Tests
init: OK (9.19s)
add: OK (13.38s)
required_content: OK (31.54s)
whereis: OK (25.92s)
conflict resolution (removed file): OK (203.83s)
adjusted branch subtree regression: OK (30.38s)
find: OK (26.15s)
fsck (remote untrusted): OK (18.86s)
edit (no pre-commit): OK (17.01s)
get: OK (13.68s)
magic: OK (5.86s)
readonly remote: OK
All 12 tests passed (395.88s)
(Failures above could be due to a bug in git-annex, or an incompatibility
with utilities, such as git, installed on this system.)
# End of transcript.
What version of git-annex are you using? On what operating system?
git-annex version: 10.20220505-g5a98f2d50
build flags: Assistant Webapp Pairing TorrentParser Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: mingw32 x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10
Windows 10 version 21H2 (build 19044.1706), 64 bit.
Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Git Annex is great. I use it several times a week with my multigigabyte backups, where it gives structure to my image-based backup routines, so you could say I'm a believer.
Thanks for another fine bug report.
I took a look at that commit, and it seems that reverting it for Windows makes the most sense. There were two places where it changed behavior on Windows, that were both separated from the code paths for other OS's.
I've made that change, which should fix it. I'm going to close this bug report, but please follow up if you still see the issue.
(Also, I've opened tasty test limiting broken by concurrency