forum/Make GA aware of externally-made copiesgit-annexhttp://git-annex.branchable.com/forum/Make_GA_aware_of_externally-made_copies/git-annexikiwiki2021-05-05T07:33:07Zcomment 1http://git-annex.branchable.com/forum/Make_GA_aware_of_externally-made_copies/comment_1_0e2edea494810a8614f628ec46b934e4/Lukey2021-05-02T18:36:56Z2021-05-02T18:36:56Z
<p>Hi,
I tested it with <code>encryption=shared</code> and it works, but not with chunking. When creating the 2nd remote, you have to look up the cipher of the first one with <code>git cat-file blob git-annex:remote.log</code> and pass it to initremote with <code>cipher=</code>. Finally, after externally copying everything from the 1st remote to the 2nd remote, you have to run <code>git annex fsck --fast --all --from=remote2</code> to make git-annex aware of the copies.</p>
comment 2http://git-annex.branchable.com/forum/Make_GA_aware_of_externally-made_copies/comment_2_b7fcfd87efb17d5159ec49eb218fbd8e/joey2021-05-04T14:15:07Z2021-05-04T13:54:45Z
<p>I think it should be possible to get this to work with chunking, if you
have git-annex version 8.20201103 or newer, and if you configure the second
special remote with the same chunk size.</p>
<p>git-annex records state about a special remote's chunks, and that state is
not available for the second special remote. Which used to prevent accessing
chunks when the information is not available, but that version made it fall
back to trying chunks of the configured chunk size.</p>
<p>See the bug report that resulted in that change for details:
<span class="createlink"><a href="http://git-annex.branchable.com/ikiwiki.cgi?do=create&from=forum%2FMake_GA_aware_of_externally-made_copies%2Fcomment_2_b7fcfd87efb17d5159ec49eb218fbd8e&page=bugs%2Ffsck__60__em__62__--key_without__60__%2Fem__62____60__strong__62__34__60__%2Fstrong__62__chunking__60__strong__62__34__60__%2Fstrong__62___information_in_git-annex_does_not_try_chunks" rel="nofollow">?</a>strong> information in git-annex does not try chunks</span></p>
<p>Oh also this only works with keys that have a recorded size. Which is most
of them, but git-annex addurl --fast adds keys without a recorded size.</p>
<hr />
<p>An alternative you might consider is to use the --sameas flag to initremote
when setting up the second remote. Then git-annex would consider the two
remotes as one repository, which means it only considers them to be one
copy, but also it can retrieve content from either.</p>
<p>If git-annex only had a way to treat a repository a more than 1 copy, that
would do just what you want. I do think there might be the possibility to
add such a feature, but it would need some thought.
<a href="http://git-annex.branchable.com/todo/repositories_that_count_as_more_than_one_copy/">repositories that count as more than one copy</a></p>
comment 3http://git-annex.branchable.com/forum/Make_GA_aware_of_externally-made_copies/comment_3_c1d69396c7bb208b6f82f2f65a282cfb/Atemu2021-05-04T17:19:57Z2021-05-04T17:19:57Z
<p>I used the exact same settings for the second special remote as the first one: <code>type=directory chunk=50MiB encryption=hybrid mac=HMACSHA256</code>.</p>
<p>GA was 8.20200810 though because my server machine is built from the stable Nixpkgs channel; I will test that again with the most recent version tomorrow.</p>
<p><code>--sameas</code> won't help here; the special remotes are accessible via the same FS (the second is just a btrfs snapshot of the first) and they'd still only count as one copy. That's the same situation I have right now.</p>
<p>Counting it as two copies would work but there is a large delay between having moved the files to the special remote and them actually being mirrored (residential internet upload) which means the numcopies of somewhat newly added files wouldn't be correct. It'd be a step up though.</p>
comment 4http://git-annex.branchable.com/forum/Make_GA_aware_of_externally-made_copies/comment_4_b0d7ef111f4862d6b759cde45696a676/Atemu2021-05-05T07:33:07Z2021-05-05T07:33:07Z
<p>It worked! Thank you so much you two!</p>
<p>The cipher was indeed different for some reason, what could cause that?</p>