Please describe the problem.
When running git-annex sync on a regular repository, where a remote bare git repository is only, I am encountering the following:
$ git-annex sync
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
git-annex: fd:17: Data.ByteString.hGetLine: end of file
Remote webdav: Repository /u01/webdav/www/databases.git-annex.uxgit/MultiArchPackages.git-annex.git is at unsupported version 9. Automatic upgrade failed!
commit
On branch master
nothing to commit, working tree clean
ok
pull webdav
ok
I found two fixed bugs that seem to mirror this:
- https://git-annex.branchable.com/projects/datalad/bugs-done/annex_view_barfs__fatal__58___Unable_to_add___40__null__41___to_database/
- https://git-annex.branchable.com/projects/datalad/bugs-done/add_--force-small_fails_on_modified_submodules/
Unlike those bugs, I am not (or at least not aware of) using submodules.
What steps will reproduce the problem?
Apologies, at this stage I haven't tried to set this up as a reproducible scenario, but I suspect it will not be an easily contrived scenario because of repository versions.
What version of git-annex are you using? On what operating system?
Using neurodebian on actual Debian 12, amd64:
$ git-annex version
git-annex version: 10.20230321-1~ndall+1
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
local repository version: 9
Please provide any additional information below.
Just to avoid confusion, yes, this is webdav shared, but I am not using a webdav special repository. I am actually allowing the repository to be mounted via webdav.
Also, I've started using neurodebian's git-annex, because I want to get nearer the bleeding edge across Debian, Ubuntu and Windows hosts.
If I change directory to that remote itself, and run the command (determined from --debug) to perform the upgrade, I see this:
$ cd /u01/webdav/www/databases.git-annex.uxgit/MultiArchPackages.git-annex.git/
$ git-annex version
git-annex version: 10.20230321-1~ndall+1
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
local repository version: 9
$ git-annex upgrade --quiet --autoonly
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
fatal: Unable to add (null) to database
git-annex: fd:17: Data.ByteString.hGetLine: end of file
Have you had any luck using git-annex before?
Yes, it's one of my favourite opensource tools.
The "unsupported version 9" is wrong, version 9 is still supported and so this upgrade failing won't prevent git-annex from using the repository. I've corrected the message.
As to why the upgrade fails, it seems to me that mounting webdav as a filesystem could easily have incompatabilities that cause strange behavior.
git can fail with this error message in at least 3 different ways. Although in all cases it's supposed to be displaying a path where it has "(null)", so it seems something is causing git to have a null pointer. That seems like it almost has to be a bug in git.
It might help to run the git-annex command with --debug to determine what git command is doing this.
Thanks joey. Based off of your comments, I believe I have found the root cause, which you are right, I don't think it's git-annex.
Also, I realise I should have added a bit more detail.
The /u01/webdav/www/ path is not a webdav mount, it's the backing real filesystem for actual webdav mounts. And this is important.
Further, this scenario worked with the Debian 11 version of git-annex, and I think those other bugs I linked to might account for the difference, because handing of dot files changed... but I'm not sure.
Also, I did run --debug, but the reason I reached out is, the git calls were of the form:
and I couldn't work out what paths were being passed via stdin. However, I've run strace and the reason I believe I see these messages is this:
So for me, the solution is to use an actual webdav mount, even on the local hosting server, and that will hide these webdav "internals"/metadata.
Is there cause at all for config changes around git-annex to avoid that though?
I should say that, sometimes I forget my internal monologue/designs.
My initial intention was to use the webdav mount exclusively, but somehow I setup my remote to the backing filesystem path
So, there's no need for a workaround really. Please free to close.
Reproduced as follows:
So it's these empty directories indeed. (Empty .DAV files don't cause this.)
In particular, it's any empty directory in .git/annex/journal. Which is supposed to only contain files that git-annex wrote there. Staging the journal is why git hash-object gets involved.
Still unclear why git ends up with "(null)" in the error message.
While it will slow git-annex down a tiny bit to check if it's a regular file, it seems better for git-annex to be robust against this kind of pollution.