copying annex between remotes manually?

I'm currently trying to migrate a git-annex repository to a new machine, one of whose remotes is the one from this issue. I determined that the root cause there seemed to be under git rather than git-annex; in particular, any whole-repository operation would take multiple days to execute, for unclear reasons. Pulling the commit data to a new repository seems to fix this.

I'm now trying to move all the annexed data from the original, broken remote to the new one. The default option here would be git annex move. However, when I run this, it apparently does some git operation for every key moved, taking hours to days; there are tens of thousands of keys, so this is obviously unworkable.

Is there a way to simply mass-move the annexed data into the new repo, via rsync or similar, and then update the new repo's metadata all at once? The state of the old repository does not matter, since I intend to discard it as soon as this migration is done.

RSS Atom

Using fsck is an option?

Hi, this worked for me:

~$ mkdir ~/test/gta
~$ cd ~/test/gta
~/test/gta$ git init

Initialized empty Git repository in /home/lell/test/gta/.git/

~/test/gta$ git annex init
~/test/gta$ git annex add file
~/test/gta$ git commit -m test

~/test/gta$ cd ..
~/test$ git clone a b

~/test$ cd gta/
~/test/gta$ cd ..
~/test$ git clone gta gta2
~/test$ cd gta2

~/test/gta2$ mkdir .git/annex/objects
~/test/gta2$ cp -r ../gta/.git/annex/objects/* .git/annex/objects/
~/test/gta2$ cp -r ../gta/.git/annex/objects .git/annex/objects
~/test/gta2$ git annex list   # it does not yet know that the file is now also here
here
|origin
||web
|||bittorrent
||||
_X__ file
~/test/gta2$ git annex fsck
fsck file (fixing location log) (checksum...) ok
(recording state in git...)
$ git annex list   # now it knows
here
|origin
||web
|||bittorrent
||||
XX__ file

Comment by lell — Thu Feb 22 07:47:36 2024

Remove comment

comment 2

In the example above, of course I forgot one line that creates the file "file" in the repository gta, e.g. like this echo test > file

Comment by lell — Thu Feb 22 07:49:58 2024

Remove comment

comment 3

I would just clone the repo to the new machine, do git annex init, and then rsync the contents of .git/annex/objects, and then do git annex fsck --all to have to recheck every key it knows about.

Alternatively, if you're concerned that there might be keys that weren't properly recorded somehow, in your new repo, after .git/annex/objects has been transferred, you can create an ingestion directory with a flat layout of the copied keys:

mkdir ingest && find .git/annex/objects -type f | xargs mv ingest && git annex reinject --known ingest/*

Finally, if you just want to rebuild it from scratch, do cp with the -cL option. If you are on macOS, it will make a reflink copy, and follow the symlinks. Delete the target .git dir and re-create it.

Comment by unqueued — Wed Apr 10 12:46:53 2024

Remove comment

Add a comment