forum/Storing copies on LTO tapes?git-annexhttp://git-annex.branchable.com/forum/Storing_copies_on_LTO_tapes__63__/git-annexikiwiki2015-07-03T03:09:38Zcomment 1http://git-annex.branchable.com/forum/Storing_copies_on_LTO_tapes__63__/comment_1_9ae9ca3983f7d4209fd1de65d982731d/joey2015-07-03T03:09:38Z2015-07-02T21:00:00Z
<p>All your solutions are reasonable, but you're right that none of them
avoid random-access. And git-annex is generally not built in a way that
makes it easy to avoid random-access.</p>
<p>But, I think it's tractable with a special remote.</p>
<p>Consider a special remote that has two retrieval modes.
In one mode, it always fails to retrieve keys, but keeps a list. In the
second mode, it starts up by going through its list, retreives everything
in order to a temporary directory, and then when asked to retrieve a key,
just moves it from the temp dir into place. This is somewhat similar to how
Amazon Glacier works, and like git-annex's glacier support, it would result
in the first <code>git-annex get</code> failing, and a second <code>git annex get</code> being
needed to finish the retrival.</p>
<p>That could be improved.. Make the special remote fail to retrive keys,
and keep a list. On shutdown, it then sorts the list, retrieves the keys
in order, and runs <code>git annex setkey</code> to move the content into the annex.
Still a little bit weird, because <code>git annex get</code> would seem to fail
and then pause at the end for a long time, after which the files would
actually end up being present.</p>
<p>(Also, I er, removed <code>git annex setkey</code> in
2011, because it didn't seem very useful, but this is in fact a use case
for it, so I've added it back now.)</p>
another approachhttp://git-annex.branchable.com/forum/Storing_copies_on_LTO_tapes__63__/comment_2_6d16d00c7ef8d846e370e1b298a7bc7a/joey2015-07-03T03:09:38Z2015-07-02T22:25:25Z
<p>You could make a special remote that streams the whole tar file from the
tape, and uses <code>git annex setkey</code> to add each file from the tarball to the
annex.</p>
<p>Done this way, the first file that <code>git annex get</code> processed would actually
cause <em>every</em> file to be gotten from the tape. As it continued on to
subsequent files, the <code>git annex get</code> would see their content was already
present and skip them.</p>
<p>Of course, the downside is it works on a whole tape at a time, so if you
don't want to load the whole tape into the filesystem, you wouldn't want to
use this approach.</p>