bugs/Continual space exhaustion from syncing metadatagit-annexhttp://git-annex.branchable.com/bugs/Continual_space_exhaustion_from_syncing_metadata/git-annexikiwiki2021-08-07T12:25:02Zcomment 1http://git-annex.branchable.com/bugs/Continual_space_exhaustion_from_syncing_metadata/comment_1_93e3deab54b34e9ad608fd549119e221/joey2016-05-23T18:15:00Z2016-05-23T18:02:11Z
<p>It would be nice if git checked disk space before writing object
files. However, it unfortunatly does not do so currently, and not checking
disk space is typical of unix tools so it might be hard to convince the git
devs to add that.</p>
<p>git-annex has annex.diskreserve because it's dealing with so much data and
such large files that it's best to not let it fill the disk and instead
abort before it downloads too much data.</p>
<p>I don't see any way that git-annex can avoid git objects taking up too much
disk space, without re-implementing all of git + space checking.</p>
<p>Sure, <code>git annex sync</code> could avoid pulling if annex.diskreserve was not
free, but this would not help with manual <code>git pull</code>, or <code>git commit</code>, or
<code>git receive-pack</code>, or any of the other ways objects can be added to a git
repository.</p>
<p>What you can do though, is set annex.diskreserve to a reasonably
large amount, so that git-annex tries to keep that much space free.
Eg, set it to at least half the current size of <code>du -hs .git/objects</code></p>
comment 2http://git-annex.branchable.com/bugs/Continual_space_exhaustion_from_syncing_metadata/comment_2_9401be5e6af886edadaf3ad7ba6a071d/xloem2016-10-09T12:13:38Z2016-10-09T12:13:38Z
<p>I'm still running into this issue. Although it is true that users can manually exhaust space, they are in control of what is happening, get more feedback, and there is no automatic process that will continue to fail further when the space is exhausted.</p>
<p>I do not use the assistant because of its frequent syncing causing complete space exhaustion, and then its inability to recover because it cannot write temporary or log files. Occasionally I have accidentally committed huge files directly to git, and this has also caused the issue.</p>
<p>I think it would be really great if git annex prevented this workflow-halting event at least by refusing to pull/commit/etc if reserved space is not met unless --force is passed. Additionally, it would really be great if actions that result in space exhaustion (fail, and then free space == 0) cleaned up their temporary files after the fact, certainly before trying to log anything.</p>
Automatic watch for the heuristichttp://git-annex.branchable.com/bugs/Continual_space_exhaustion_from_syncing_metadata/comment_3_3309c51dbd251cc46a25557a0d25e946/jgsuess2021-08-07T12:25:02Z2021-08-07T12:25:02Z
It seems there is a good heuristic based on the object size. It is given in this thread. Could this check not be used in annex automatic syncing modes to avoid this problem. I have run into it several times now and recovery, while possible, is time-consuming and messy.