Please describe the problem.
If git annex fsck --from web
encounters a URL that is no longer available,
it will remove all URLs from the appropriate file's location log.
Surprisingly, adding one of the URLs the file was associated to before brings back all of them.
What steps will reproduce the problem?
$ git annex whereis file.txt
whereis file.txt (3 copies)
...
The following untrusted locations may also have copies:
00000000-0000-0000-0000-000000000001 -- web
web: http://example.org/dead/file.txt
web: http://example.org/alive/file.txt
ok
$ git annex fsck --from web
fsck file.txt (checking http://example.org/dead/file.txt...) (fixing location log)
** Based on the location log, file.txt
** was expected to be present, but its content is missing.
failed
(recording state in git...)
git-annex: fsck: 1 failed
$ git annex whereis file.txt
whereis file.txt (3 copies)
...
ok
$ git annex addurl http://example.org/alive/file.txt
addurl file.txt ok
(recording state in git...)
$ git annex whereis file.txt
whereis file.txt (3 copies)
...
The following untrusted locations may also have copies:
00000000-0000-0000-0000-000000000001 -- web
web: http://example.org/dead/file.txt
web: http://example.org/alive/file.txt
ok
The only fix that seems to fix this is using rmurl
on the dead one.
What version of git-annex are you using? On what operating system?
$ git annex version
git-annex version: 6.20171026
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.7.0 persistent-sqlite-2.6.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
local repository version: 5
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
operating system: linux x86_64
I'm running NixOS btw.
Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
I'm lovin' it!
I think that the real bug here is that, as long as one url still has the content, it's still present in the web, so should not be marked as not present. The rest of the operations in the web special remote work that way, trying urls until one succeeds, and so should
checkKey
.If you want it that way, that's fine with me. But I think it would make more sense to let
fsck
callrmurl
on all URLs that don't hold the content anymore.There may be one url that works in one situation, and another that always works. Eg, a url on the work LAN, and a public, slower url. In such a situation, the user would not want fsck to remove an url that is temporarily not available.