Please describe the problem.
When the preferred content for a repo is greater than the free space on the drive for me, the space becomes exhausted due to upload that occurs during syncing -- the git history ends up exhausting the space. The result for me has been corrupted repos with broken refs and heads.
I think it would be better if the git syncing process respected the same space limitations that the content syncing does, and did not push if space was exhausted on the destination, nor pull if space exhausted locally.
What steps will reproduce the problem?
Make enough git commits that it exhausts the free space on some repo.
What version of git-annex are you using? On what operating system?
I've been using a few different versions; right now I am using 6.2016-0511 and 6.2016-0517-g766728c but I have not tested this issue with these most recent versions. I am still trying to get my smaller repos back together.
Please provide any additional information below.
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
# End of transcript or log.
Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes !!! I'm moving all my files into my annex. It is very robust; whenever something is wrong there is always some other copy somewhere that can be used.
It would be nice if git checked disk space before writing object files. However, it unfortunatly does not do so currently, and not checking disk space is typical of unix tools so it might be hard to convince the git devs to add that.
git-annex has annex.diskreserve because it's dealing with so much data and such large files that it's best to not let it fill the disk and instead abort before it downloads too much data.
I don't see any way that git-annex can avoid git objects taking up too much disk space, without re-implementing all of git + space checking.
Sure,
git annex synccould avoid pulling if annex.diskreserve was not free, but this would not help with manualgit pull, orgit commit, orgit receive-pack, or any of the other ways objects can be added to a git repository.What you can do though, is set annex.diskreserve to a reasonably large amount, so that git-annex tries to keep that much space free. Eg, set it to at least half the current size of
du -hs .git/objectsI'm still running into this issue. Although it is true that users can manually exhaust space, they are in control of what is happening, get more feedback, and there is no automatic process that will continue to fail further when the space is exhausted.
I do not use the assistant because of its frequent syncing causing complete space exhaustion, and then its inability to recover because it cannot write temporary or log files. Occasionally I have accidentally committed huge files directly to git, and this has also caused the issue.
I think it would be really great if git annex prevented this workflow-halting event at least by refusing to pull/commit/etc if reserved space is not met unless --force is passed. Additionally, it would really be great if actions that result in space exhaustion (fail, and then free space == 0) cleaned up their temporary files after the fact, certainly before trying to log anything.