projects/repronim/bugs-done/regression: fails to detect need for pidlock on an NSF mount
yoh
http://git-annex.branchable.com/projects/repronim/bugs-done/regression__58___fails_to_detect_need_for_pidlock_on_an_NSF_mount/
git-annex
ikiwiki
2023-01-05T17:30:31Z
comment 1
http://git-annex.branchable.com/projects/repronim/bugs-done/regression__58___fails_to_detect_need_for_pidlock_on_an_NSF_mount/comment_1_82a71d4e9656c83e2d50143cb8a31602/
joey
2023-01-05T17:30:31Z
2019-08-29T16:11:14Z
<p>No changes in the code of the lock test. Somehow the exception
was not caught, and it's only catching IO exceptions, so it must
be another type of exception.</p>
<p>Oddly, I can't find any indication that waitToSetLock can throw anything
other than an IO exception.</p>
<p>I've changed it to catch all exceptions, which will presumably fix the
problem, but not closing this bug as I don't fully understand it and so am
not certian of the fix.</p>
comment 2
http://git-annex.branchable.com/projects/repronim/bugs-done/regression__58___fails_to_detect_need_for_pidlock_on_an_NSF_mount/comment_2_9bd5f0acb9761dcf59ecd38092e3e14d/
yarikoptic
2023-01-05T17:30:31Z
2019-08-31T13:00:57Z
<p>I have ran git bisect, didn't double check but it points to</p>
<div class="highlight-sh"><pre class="hl">...
running <span class="hl opt">/</span>home<span class="hl opt">/</span>bids<span class="hl opt">/</span>git-annexes<span class="hl opt">/</span>datalad<span class="hl opt">/</span>tools<span class="hl opt">/</span>bisect-git-annex <span class="hl opt">/</span>home<span class="hl opt">/</span>bids<span class="hl opt">/</span>datalad-datalad-master-git-annex-dev.simg <span class="hl opt">/</span>home<span class="hl opt">/</span>bids<span class="hl opt">/</span>git-annexes<span class="hl opt">/</span>datalad<span class="hl opt">/</span>tools<span class="hl opt">/</span>bisect-git-annex.<span class="hl kwc">script</span>s<span class="hl opt">/</span>bisect-nfs-lock.sh
I<span class="hl opt">:</span> cleaning <span class="hl opt">/</span>home<span class="hl opt">/</span>bids<span class="hl opt">/</span>git-annexes<span class="hl opt">/</span>git-annex
I<span class="hl opt">:</span> building <span class="hl num">7.20181211</span><span class="hl kwb">-248-gd5f246370</span>
I<span class="hl opt">:</span> running the <span class="hl kwc">script</span>
Initialized empty Git repository <span class="hl kwa">in</span> <span class="hl opt">/</span>inbox<span class="hl opt">/</span>BIDS<span class="hl opt">/</span>.git<span class="hl opt">/</span>tmp<span class="hl opt">/</span>repo<span class="hl opt">/</span>.git<span class="hl opt">/</span>
init
git-annex<span class="hl opt">:</span> waitToSetLock<span class="hl opt">:</span> resource exhausted <span class="hl opt">(</span>No locks available<span class="hl opt">)</span>
failed
git-annex<span class="hl opt">:</span> init<span class="hl opt">:</span> <span class="hl num">1</span> failed
d5f2463702a7cd8e56ea000e21bfe5d4ba9cd386 is the first bad commit
commit d5f2463702a7cd8e56ea000e21bfe5d4ba9cd386
Author<span class="hl opt">:</span> Joey Hess <span class="hl opt"><</span>joeyh@joeyh.name<span class="hl opt">></span>
Date<span class="hl opt">:</span> Thu Jan <span class="hl num">17 15</span><span class="hl opt">:</span><span class="hl num">40</span><span class="hl opt">:</span><span class="hl num">44 2019</span> <span class="hl kwb">-0400</span>
misctmp cleanup
<span class="hl opt">*</span> Switch to using .git<span class="hl opt">/</span>annex<span class="hl opt">/</span>othertmp <span class="hl kwa">for</span> tmp files other than partial
downloads<span class="hl opt">,</span> and <span class="hl kwc">make</span> stale files left <span class="hl kwa">in</span> that directory when git-annex
is interrupted be cleaned up promptly by subsequent git-annex processes.
<span class="hl opt">*</span> The .git<span class="hl opt">/</span>annex<span class="hl opt">/</span>misctmp directory is no longer used and git-annex will
delete anything lingering <span class="hl kwa">in</span> there after it<span class="hl str">'s 1 week old.</span>
<span class="hl str"> </span>
<span class="hl str"> Also, in Annex.Ingest, made the filename it uses in the tmp dir be</span>
<span class="hl str"> prefixed with "ingest-" to avoid potentially using a filename used by</span>
<span class="hl str"> some other code.</span>
<span class="hl str"></span>
<span class="hl str">:040000 040000 3a3f3af1c5222bb96684c79b9bccc20b3ff266a2 24efba74a2b6e4a361e300f32325fafba76b8243 M Annex</span>
<span class="hl str">:040000 040000 193fa3b2bc663031a0e387e921c720e022daabbc 6a0081f3e1df14a182416e734eedb0a2d8bdd46d M Assistant</span>
<span class="hl str">:100644 100644 d313f9ee315a095e992b729f0d7001e14f413aee f5c5f511d85ffb52c303d1b718fc62e5179b66f9 M CHANGELOG</span>
<span class="hl str">:040000 040000 432613892dd51d2ebb804f11c96d1d2c7956dab7 9bcbedc855d35faeeaf9b6d74cc2f48c48d8d764 M Command</span>
<span class="hl str">:040000 040000 0d4841d00ee7ec5b59037ac2c37ab10482359671 0b12fe02a9028e3a7a083fd3dfdc01bf38a8626e M Remote</span>
<span class="hl str">:100644 100644 555166e29905dca5984403500528d4594b4b7bb6 8f9c460306db4c9f3abed23d9181a10f20b42b77 M Test.hs</span>
<span class="hl str">:040000 040000 2eb286c09f87da53c2377cc60b75c2c49fcd049a 33adb5b5e5b8a33b2012345b73f00eb094afd42c M Types</span>
<span class="hl str">:040000 040000 0406fc6fff2205a48d6a215cf18dba2c682de99e a688ce72ede0bbb71b7e2f1f6a811602ee333a16 M doc</span>
<span class="hl str">:100644 100644 839b469b5cea11a65f20652963fea316be367c93 ff2db6e39cee114d8bd386873b0dba6f89e4d4d1 M git-annex.cabal</span>
<span class="hl str">bisect run success</span>
</pre></div>
<p>may be that gives ideas? (build env was the same, freshly regenerated buster singularity image available via <code>singularity pull shub://datalad/datalad:git-annex-dev</code></p>
the issue persists in master
http://git-annex.branchable.com/projects/repronim/bugs-done/regression__58___fails_to_detect_need_for_pidlock_on_an_NSF_mount/comment_3_6d2fceb790e675cb73b193bf67f7a6f4/
yarikoptic
2023-01-05T17:30:31Z
2019-09-02T14:17:14Z
<p>FWIW, the issue persists with current version in master (which comes after comment from 3 days ago where wider exception catching was told to be implemented):</p>
<div class="highlight-sh"><pre class="hl"><span class="hl opt">[</span>bids@rolando git-annex<span class="hl opt">] ></span> ~<span class="hl opt">/</span>git-annexes<span class="hl opt">/</span>datalad<span class="hl opt">/</span>tools<span class="hl opt">/</span>bisect-git-annex.<span class="hl kwc">script</span>s<span class="hl opt">/</span>bisect-nfs-lock.sh
Initialized empty Git repository <span class="hl kwa">in</span> <span class="hl opt">/</span>inbox<span class="hl opt">/</span>BIDS<span class="hl opt">/</span>.git<span class="hl opt">/</span>tmp<span class="hl opt">/</span>repo<span class="hl opt">/</span>.git<span class="hl opt">/</span>
init
git-annex<span class="hl opt">:</span> waitToSetLock<span class="hl opt">:</span> resource exhausted <span class="hl opt">(</span>No locks available<span class="hl opt">)</span>
failed
git-annex<span class="hl opt">:</span> init<span class="hl opt">:</span> <span class="hl num">1</span> failed
<span class="hl opt">[</span>bids@rolando git-annex<span class="hl opt">] ></span> <span class="hl kwb">echo</span> $?
<span class="hl num">1</span>
<span class="hl opt">[</span>bids@rolando git-annex<span class="hl opt">] ></span> git annex version
git-annex version<span class="hl opt">:</span> <span class="hl num">7.20190826</span><span class="hl kwb">-gf951a6221</span>
build flags<span class="hl opt">:</span> Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
dependency versions<span class="hl opt">:</span> aws-0.20 bloomfilter-2.0<span class="hl num">.1.0</span> cryptonite-0.25 DAV-1.3<span class="hl num">.3</span> feed-1.0<span class="hl num">.0.0</span> ghc-8.4<span class="hl num">.4</span> http-client-0.5<span class="hl num">.13.1</span> persistent-sqlite-2.8<span class="hl num">.2</span> torrent-10000.1<span class="hl num">.1</span> uuid-1.3<span class="hl num">.13</span> yesod-1.6<span class="hl num">.0</span>
key<span class="hl opt">/</span>value backends<span class="hl opt">:</span> SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types<span class="hl opt">:</span> git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
operating system<span class="hl opt">:</span> linux x86_64
supported repository versions<span class="hl opt">:</span> <span class="hl num">5 7</span>
upgrade supported from repository versions<span class="hl opt">:</span> <span class="hl num">0 1 2 3 4 5 6</span>
</pre></div>
comment 4
http://git-annex.branchable.com/projects/repronim/bugs-done/regression__58___fails_to_detect_need_for_pidlock_on_an_NSF_mount/comment_4_bf6487c1740f163745b7dcc41468ec4d/
joey
2023-01-05T17:30:31Z
2019-09-10T16:28:12Z
<p>Thanks for the bisection. It's not the init code that's
trying and failing to set a lock, but the misctmp cleanup code. Which is
ironically now used in setting up the temp directory that init uses to
probe for locking problems. Chicken and egg problem.</p>
<p>Committed a fix.</p>