forum/relying on git for numcopiesgit-annexhttp://git-annex.branchable.com/forum/relying_on_git_for_numcopies/git-annexikiwiki2013-11-27T22:47:37Zcomment 1http://git-annex.branchable.com/forum/relying_on_git_for_numcopies/comment_1_8ad3cccd7f66f6423341d71241ba89fc/joey2013-11-27T22:47:37Z2011-02-22T18:44:28Z
<p>I see the following problems with this scheme:</p>
<ul>
<li><p>Disallows removal of files when disconnected. It's currently safe to force that, as long as
git-annex tells you enough other repos are belived to have the file. Just as long as you
only force on one machine (say your laptop). With your scheme, if you drop a file while
disconnected, any other host could see that the counter is still at N, because your
laptop had the file last time it was online, and can decide to drop the file, and lose the last
version.</p></li>
<li><p>pushing a changed counter commit to other repos is tricky, because they're not bare, and
the network topology to get the commit pulled into the other repo could vary.</p></li>
<li><p>Merging counter files issues. If the counter file doesn't automerge, two repos dropping the same file will conflict. But, if it does automerge, it breaks the counter conflict detection.</p></li>
<li><p>Needing to revert commits is going to be annoying. An actual git revert
could probably not reliably be done. It's need to construct a revert
and commit it as a new commit. And then try to push that to remotes, and
what if <em>that</em> push conflicts?</p></li>
<li><p>I do like the pre-removal dropping somewhat as an alternative to
trust checking. I think that can be done with current git-annex though,
just remove the files from the location log, but keep them in-annex.
Dropping a file only looks at repos that the location log says have a
file; so other repos can have retained a copy of a file secretly like
this, and can safely remove it at any time. I'd need to look into this a bit more to be 100% sure it's safe, but have started <a href="http://git-annex.branchable.com/todo/hidden_files/">hidden files</a>.</p></li>
<li><p>I don't see any reduced round trips. It still has to contact N other
repos on drop. Now, rather than checking that they have a file, it needs
to push a change to them.</p></li>
</ul>
comment 2http://git-annex.branchable.com/forum/relying_on_git_for_numcopies/comment_2_be6acbc26008a9cb54e7b8f498f2c2a2/chrysn2013-11-27T22:47:37Z2011-02-23T16:43:59Z
<p>i'll comment on each of the points separately, well aware that even a single little leftover issue can show that my plan is faulty:</p>
<ul>
<li>force removal: well, yes -- but the file that is currently force-removed on the laptop could just as well be the last of its kind itself. i see the problem, but am not sure if it's fatal (after all, if we rely on out-of-band knowledge when forcing something, we could just as well ask a little more)</li>
<li>non-bare repos: pushing is tricky with non-bare repos now just as well; a post-commit hook could auto-accept counter changes. (but pushing causes problems with counters anyway, doesn't it?)</li>
<li>merging: i'd have them auto-merge. git-annex will have to check the validity of the current state anyway, and a situation in which a counter-decrementing commit is not a fast-forward one would be reverted in the next step (or upon discovery, in case the next step never took place).</li>
<li>reverting: my wording was bad as "revert" is already taken in git-lingo. the correct term for what i was thinking of is "reset". (as the commit could not be pushed, it would be rolled back completely).
<ul>
<li>we might have to resort to reverting, though, if the commit has already been pused to a first server of many.</li>
</ul>
</li>
<li><a href="http://git-annex.branchable.com/todo/hidden_files/">hidden files</a>: yes, this solves pre-removal dropping <img src="http://git-annex.branchable.com/smileys/smile.png" alt=":-)" /></li>
<li>round trips: it's not the number of servers, it's the number of files (up to 30k in my case). it seems to me that an individual request was made for every single file i wanted to drop (that would be N*M roundtrips for N affected servers and M files, and N roundtrips with git managed numcopies)</li>
</ul>
<p>all together, it seems to be a bit more complicated than i imagined, although not completely impossible. a combination of <a href="http://git-annex.branchable.com/todo/hidden_files/">hidden files</a> and maybe a simpler reduction of the number of requests might though achieve the important goals as well.</p>
relation to [[todo/branching]]http://git-annex.branchable.com/forum/relying_on_git_for_numcopies/comment_3_43d8e1513eb9947f8a503f094c03f307/chrysn2013-11-27T22:47:37Z2011-02-23T21:48:14Z
the non-bare repository issue would go away if this was combined with the "alternate" approach to <span class="createlink"><a href="http://git-annex.branchable.com/ikiwiki.cgi?do=create&from=forum%2Frelying_on_git_for_numcopies%2Fcomment_3_43d8e1513eb9947f8a503f094c03f307&page=todo%2Fbranching" rel="nofollow">?</a>branching</span>. (with the "fleshed out proposal" of branching, this would not work at all for lack of shared commits.)