bugs/Git repos corrupt themselvesgit-annexhttp://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/git-annexikiwiki2022-08-22T16:05:51Zcomment 1http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_1_c7fbc3da4404fb33a4b0babf068dc4d3/joey2021-05-10T15:05:07Z2021-05-10T14:25:20Z
<p>git-annex does not write directly to your git repository; all writes go via
git. So it is hard to see how any corruption like this could be due to a
bug in git-annex.</p>
<p>What it usually is caused by is a filesystem losing data, eg because of an
unclean shutdown or a crash, or sometimes because of a disk error.</p>
<p>Setting the git config core.fsyncObjectFiles may help. Configuring the
filesystem to write data to disk more promptly (eg enabling journalling of
file contents) may help.</p>
<p><code>git-annex repair --force</code> may be able to recover from this situation.</p>
<p>The "evacuate: strange closure" error does seem notable, something strange
is going on in the memory of git-annex. Presumably due to either a compiler
bug or some kind of memory problem.</p>
comment 2http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_2_f2368358a0ccdd36a551a2c833cb9c9a/Lukey2021-05-11T07:37:07Z2021-05-11T07:37:07Z
<p>This also looks suspicious:</p>
<pre><code>Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
</code></pre>
<p>And after that something (git-annex?) is complaining about missing git objects. So maybe you can try to set <code>gc.auto</code> to <code>0</code> and see if that works better.</p>
comment 3http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_3_f7d921256107773dcb294747cdc38ebd/Atemu2021-05-12T14:17:05Z2021-05-12T14:17:05Z
<p>The weird thing is, this only ever happens with my Documents git-annex repo, the only one where I use the assistant. It has never happened in 4 years of actively using regular git nor my manually committed and synced git-annex repos.</p>
<p>Another corruption just occured but this time on the MBP:</p>
<pre><code>[2021-05-11 08:04:43.038047] main: starting assistant version 8.20210330
[2021-05-11 08:04:43.169107] Cronner: Consistency check in progress
[2021-05-11 08:04:43.175482] TransferScanner: Syncing with SOTERIA
(scanning...) [2021-05-11 08:04:43.340744] Watcher: Performing startup scan
(recording state in git...)
(recording state in git...)
(recovering from race...)
To ssh://192.168.101.24/~/Annex/Documents.git/
7b587c48b..4c7b284c5 git-annex -> synced/git-annex
[2021-05-11 08:04:47.087899] Cronner: Attempting to repair MB-<REDACTED> [here]
Watcher crashed: PauseWatcher
[2021-05-11 08:04:47.138205] Watcher: warning Watcher crashed: PauseWatcher
Unpacking all pack files.
fatal: early EOF
[2021-05-11 08:05:27.149344] main: starting assistant version 8.20210330
[2021-05-11 08:05:27.325057] Cronner: Consistency check in progress
(recording state in git...)
error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log'
fatal: git-write-tree: error building trees
.git-annex-wrapped: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:28.719256] TransferScanner: Syncing with SOTERIA
(recording state in git...)
error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log'
fatal: git-write-tree: error building trees
Merger crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:28.905248] Merger: warning Merger crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
(recording state in git...)
error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log'
fatal: git-write-tree: error building trees
TransferScanner crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:29.430437] TransferScanner: warning TransferScanner crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:52.410361] main: warning git-annex has been shut down
</code></pre>
<p>This happened just after I started the assistant/webapp and you can see that I shut it down in the end.</p>
<p>My filesystems are ZFS on the Linux machine and APFS on the MBP. Being an Apple product, I'd expect APFS to be built to high standards (though I don't fully trust it) but ZFS corrupting anything or losing data is almost unheard of.<br />
I'll try setting <code>core.fsyncObjectFiles</code> on this repo and update this issue accordingly when I see another corruption with it on. Shouldn't take too long as this occurs worringly frequently.</p>
<p>Btw, repair always fails in my setup because git seems to be unable to find the <code>SOTERIA</code> remote; is it trying to use the remote name literally as an address perhaps?</p>
comment 4http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_4_cf152fccbdb281dcbeb02cb398686ac0/Atemu2021-05-16T10:27:47Z2021-05-16T10:27:47Z
<p>Something peculiar just happened. I had the MBP with me while I was out and was editing documents with no connection to any of my machines.</p>
<p>I usually stop the assistant at home so that it only runs on one machine at a time because of https://git-annex.branchable.com/bugs/OSX<strong>58</strong>_Pushed_changes_are_autocommited/ but because I was out, I started it.</p>
<p>On startup, it said that it was trying to repair my repo. I shut it down because it couldn't do any repairs without a connection to my machines (and it wouldn't succeed even if it did unfortunately).</p>
<p>I ran <code>git fsck</code> to see what happened but, to my surprise, it returned no errors besides missing reflog entries from previous corruptions. <code>git gc</code> didn't error either.</p>
<p>I commited my changes manually while I was out.</p>
<p>That was about two days ago. I synced my changes manually at home where an unrelated and trivial issue happened during which I ran another successful <code>git fsck</code> and, after fixing that issue, I started the webapp.<br />
It was attempting to repair again and was complaining about a thread having crashed (logs below).</p>
<p>Now however, the repo is actually broken with missing blobs, trees and links everywhere.</p>
<p>daemon.log.1:</p>
<pre><code>[2021-05-16 11:58:04.848709] main: starting assistant version 8.20210330
[2021-05-16 11:58:04.946261] Cronner: Consistency check in progress
[2021-05-16 11:58:04.974666] TransferScanner: Syncing with SOTERIA
(scanning...) [2021-05-16 11:58:05.191375] Watcher: Performing startup scan
(recording state in git...)
(recording state in git...)
(recovering from race...)
ControlSocket .git/annex/ssh/atemu@192.168.101.24 already exists, disabling multiplexing
To ssh://192.168.101.24/~/Annex/Documents.git/
343d6291c..184ff9756 git-annex -> synced/git-annex
[2021-05-16 11:58:09.404901] Cronner: Attempting to repair MB-<REDACTED> [here]
Watcher crashed: PauseWatcher
[2021-05-16 11:58:09.445038] Watcher: warning Watcher crashed: PauseWatcher
Unpacking all pack files.
fatal: early EOF
</code></pre>
<p>daemon.log:</p>
<pre><code>[2021-05-16 11:58:21.699212] main: starting assistant version 8.20210330
[2021-05-16 11:58:21.778787] Cronner: You should enable consistency checking to protect your data.
[2021-05-16 11:58:22.249053] TransferScanner: Syncing with SOTERIA
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
Everything up-to-date
[2021-05-16 11:58:25.957064] Transferrer: Uploaded <1 pdf file>
[2021-05-16 11:58:25.95937] Pusher: Syncing with SOTERIA
(recording state in git...)
error: invalid object 100644 3d5a69ecabdbf871de3b7c339b16bc9315ae17e4 for '5d3/ae2/SHA256E-s56534--2de587e678bd455f4346a5304e450eafac892f1a389ee4046cd25948a01789a9.pdf.log.met'
fatal: git-write-tree: error building trees
Pusher crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-16 11:58:26.037969] Pusher: warning Pusher crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-16 11:58:28.564268] Transferrer: Uploaded <a bunch of files because I changed numcopies>
[2021-05-16 11:58:51.983601] Pusher: Syncing with SOTERIA
(recording state in git...)
error: invalid object 100644 3b1a2b60bcb28a299c7e6b706ce3de5808909bcc for '031/bc2/SHA256E-s138590--273323cbd322923f9d19280b6b4fa73bfa3638d2982b685b9efcaee43c988a07.pdf.log.met'
fatal: git-write-tree: error building trees
Pusher crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-16 11:58:52.11129] Pusher: warning Pusher crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-16 11:59:08.905898] main: warning git-annex has been shut down
</code></pre>
<p>I haven't set the fsync option yet but this system hasn't been restarted in a long time and the corruption happened while the system was on; right in front of my eyes basically.</p>
<p>I'll now manually fix the corruption again by rsync'ing the <code>.git/objects/</code> dir to the state of a working repo's.</p>
comment 5http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_5_652928a8f8bd0250973f19fa28cbb029/Atemu2021-05-18T09:13:32Z2021-05-18T09:13:32Z
<p>Another crash on the Linux machine as I was starting the assistant today. Usually it doesn't happen nearly this frequently though.</p>
<p><code>core.fsyncObjectFiles</code> was set to <code>true</code> this time.</p>
<p>log.3 (2d ago 19:xx):</p>
<pre><code>git-annex: internal error: evacuate: strange closure type 4566880
(GHC version 8.10.4 for x86_64_unknown_linux)
Please report this as a GHC bug: https://www.haskell.org/ghc/reportabug
</code></pre>
<p>log.2 (today 10:16):</p>
<pre><code><regular git-annex fsck output>
ControlSocket .git/annex/ssh/e5a88504f7bbb083855b27a1c1ca4a26 already exists, disabling multiplexing
To ssh://192.168.101.24/~/Annex/Documents.git/
f153f6740..b52bda802 git-annex -> synced/git-annex
To ssh://git-annex-.MacBook.2DPro.2Efritz.2Ebox-TobiasDe_22_Documents/~/Documents/
bb6821230..b52bda802 git-annex -> synced/git-annex
(started...)
Unpacking all pack files.
fatal: early EOF
</code></pre>
<p>at the time of log.1 (10:55), corruption has already set in.</p>
<p>I'm sniffing an issue with the auto-repair feature wrongly detecting corruption, trying to fix it and thereby actually corrupting the repo. I'd like to turn it off for investigation, is that possible somehow? By disabling consistency checks perhaps?</p>
comment 6http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_6_315606cfa07d2c4b20a6c9ec477c83d1/joey2021-05-21T16:39:49Z2021-05-21T16:26:55Z
<p>Note that the strange closure crash is a separate bug,
and has been fixed in git master. I do not think that bug could corrupt git
repos since as I mentioned before git-annex doesn't write directly to the
git repo, so any such bug in it should not be possible to corrupt it.</p>
<p>Anyway, it would probably be a good idea to upgrade to a current daily
build to get past that bug.</p>
comment 7http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_7_54aed251ddb9df1c8164a25240e1533f/Atemu2021-05-22T17:02:07Z2021-05-22T17:02:07Z
What could I do to get to the bottom of this? Is there a verbose logging feature for the assistant that could give more insight?
comment 8http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_8_a1131917149dd07ec73367e7cbc3b3f2/Lukey2021-05-22T17:19:19Z2021-05-22T17:19:19Z
You could run the equivalent of <code>strace</code> for mac os and make it log every syscall of git-annex assistant and it's children. This would show which <code>git</code> action corrupts the repo and how.
comment 9http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_9_033def91d2a30f1cee498fabbfe5d37f/Atemu2021-05-22T17:31:01Z2021-05-22T17:31:01Z
This also happens on Linux, so I'd rather use <code>strace</code> itself but I think it'd be extremely hard to spot anything useful in that and probably is not worth the performance cost.
comment 10http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_10_74e6d17a1b14b2fdc7eddf4ee1895733/Atemu2021-05-25T11:00:39Z2021-05-25T11:00:39Z
<p>A new crash just happened. I haven't upgraded git-annex to master yet but the strange closure bug didn't occur this time. Annotated daemon.log:</p>
<pre><code>(scanning...) (started...)
<lots of this>
ssh: connect to host 192.168.101.68 port 22: Connection timed out
<because the MBP is sleeping>
add file.etherpad ok <note: textconv is set up to pipe .etherpad through `jq .` to pretty-print it>
(recording state in git...)
<git annex fsck all the tree's files. I did not trigger this and automatic consistency checks are off!>
(recording state in git...)
(recording state in git...)
git-annex: fsck: 1 failed
(recovering from race...)
ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
remote: error: cannot lock ref 'refs/heads/master': is at 6e737fee3f50ee203c9db151e3f9d0308c63df9b but expected fc38ba7550f8dd0d29262bc47ae3085403a55784 To ssh://192.168.101.24/~/Annex/Documents.git/ ! [remote rejected] master -> master (failed to update ref)error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/'ssh: connect to host 192.168.101.68 port 22: No route to host
fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: No route to host
fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists.ssh: connect to host 192.168.101.68 port 22: No route to host
fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists. Transfer failed
Unpacking all pack files.
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /tmp/tmprepoZrGQTy/.git/
Trying to recover missing objects from remote MB-<REDACTED>.
fatal: 'MB-<REDACTED>' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Trying to recover missing objects from remote SOTERIA.
fatal: 'SOTERIA' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Trying to recover missing objects from remote MB-<REDACTED>.
fatal: 'MB-<REDACTED>' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Trying to recover missing objects from remote SOTERIA.
fatal: 'SOTERIA' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
2 missing objects could not be recovered!
To force a recovery to a usable state, retry with the --force parameter.
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
add <.txt render of the etherpad file> fatal: Invalid revision range refs/heads/master..refs/synced/d7d728f7-891a-4035-a758-c7ee80a8017a/master
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
ok
(recording state in git...)
error: refs/heads/master does not point to a valid object!
error: refs/heads/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
CONFLICT (add/add): Merge conflict in <unrelated but frequently edited file>.org
Auto-merging <file>.org
<a handful of conflicts like this>
Automatic merge failed; fix conflicts and then commit the result.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: Merging is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
100% 3.33 KiB 198 MiB/s 0s
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: Merging is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: Merging is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
ssh: connect to host 192.168.101.68 port 22: No route to host
ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.Please make sure you have the correct access rightsand the repository exists. Transfer failed
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
To ssh://192.168.101.24/~/Annex/Documents.git/
105d6d0..4bd9661 git-annex -> synced/git-annex
! [rejected] master -> synced/master (fetch first)
error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
ssh: connect to host 192.168.101.68 port 22: No route to host
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
error: refs/remotes/SOTERIA/master does not point to a valid object!
error: refs/remotes/SOTERIA/synced/master does not point to a valid object!
From ssh://192.168.101.24/~/Annex/Documents
* [new branch] master -> SOTERIA/master
* [new branch] synced/master -> SOTERIA/synced/master
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: Merging is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
error: Merging is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
error: Merging is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
error: Merging is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm <file>'
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
To ssh://192.168.101.24/~/Annex/Documents.git/
! [rejected] master -> synced/master (non-fast-forward)
error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
ssh: connect to host 192.168.101.68 port 22: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
ssh: connect to host 192.168.101.68 port 22: No route to host
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
To ssh://192.168.101.24/~/Annex/Documents.git/
8cd959c..4bd9661 git-annex -> refs/synced/8dc76e63-3dac-4cc3-a752-e73e81d57c1a/git-annex
! [rejected] master -> refs/synced/8dc76e63-3dac-4cc3-a752-e73e81d57c1a/master (non-fast-forward)
error: failed to push some refs to 'ssh://192.168.101.24/~/Annex/Documents.git/'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
ssh: connect to host 192.168.101.68 port 22: Connection timed out
</code></pre>
<p>Git is now trying to merge refs/synced/d7d728f7-891a-4035-a758-c7ee80a8017a/master which points at a commit made on the MBP 4d ago. This commit has previously been <strong>merged</strong> into the original branch on this machine and contains a change to a txt render of a different .etherpad file. This also means that the commit before it contains a change to the according .etherpad file.</p>
<p>My branch is now a single commit made at 11:49:21 that adds the whole state of the working dir.
However, the diff for the .etherpad file says (without textconv):</p>
<pre><code>diff --git a/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.etherpad b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.etherpad
new file mode 100644
index 0000000..33e0a8f
--- /dev/null
+++ b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.etherpad
@@ -0,0 +1 @@
+/annex/objects/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2
diff --git a/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.txt b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.txt
new file mode 100644
index 0000000..f9ce6fe
--- /dev/null
+++ b/Uni/LAWR/V10/132c6ef4e421c88547d4a228fef3c39491ba0610.txt
@@ -0,0 +1 @@
+/annex/objects/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt
</code></pre>
<p>It added these as a unlocked largefiles despite my largefiles rule excluding them. This has happened before once I think but I manually committed at that time.</p>
<p>The other .etherpad files I had in the tree were not added as largefiles.</p>
<p>I remember having issues with textconv and annex before too and disabling a few rules that I didn't really need.</p>
<p>The git-annex branch seems to be in-tact. However, its commits are a bit weird. Here is <code>git log synced/git-annex..git-annex --patch</code>:</p>
<pre><code>commit 4bd9661256a12d81186420a16d1bc1fb5f5d261a
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:22 2021 +0200
update (recovery from race #1; expected commit parent Ref "5edabb3782060a334f966387e03252440ef50b10" but found [Ref "c518e6a6cac2b966e757a97dc8d26b64743be377"] )
commit efdaa77dbf258954289366e236f056e50a258e1a
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:22 2021 +0200
update
commit c518e6a6cac2b966e757a97dc8d26b64743be377
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:22 2021 +0200
update (recovery from race #1; expected commit parent Ref "5edabb3782060a334f966387e03252440ef50b10" but found [Ref "79a6d5f9b7b45ae891529a9bf2ead592f3968faa"] )
commit fd7d4f7a6c875ca43cfe5e2e34b27c4cd6daab0a
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:22 2021 +0200
update
commit 79a6d5f9b7b45ae891529a9bf2ead592f3968faa
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:22 2021 +0200
update (recovery from race #1; expected commit parent Ref "5edabb3782060a334f966387e03252440ef50b10" but found [Ref "79190c0161b13aae6570cb24f232ea6ef9288f66"] )
commit 1276807833ad40c19190049a85ba82b5b171edd2
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:22 2021 +0200
update
commit 79190c0161b13aae6570cb24f232ea6ef9288f66
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:22 2021 +0200
update
diff --git a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log
index 92b1312..0ff0604 100644
--- a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log
+++ b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log
@@ -1 +1,2 @@
1621936161.088002573s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a
+1621936161.329141017s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a
commit 5edabb3782060a334f966387e03252440ef50b10
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:49:21 2021 +0200
update
diff --git a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log
new file mode 100644
index 0000000..92b1312
--- /dev/null
+++ b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log
@@ -0,0 +1 @@
+1621936161.088002573s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a
diff --git a/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log.met b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log.met
new file mode 100644
index 0000000..9972f75
--- /dev/null
+++ b/672/674/SHA256E-s3406--d5168f8722e96719e2cf32d21760690930264aa8c88d76098477d3438fa247ab.txt.log.met
@@ -0,0 +1 @@
+1621936161.086396922s day +25 month +5 year +2021
commit 6dbb8cc706949f8ef1dbbbd1003dbe728b642f8a
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:45:51 2021 +0200
update (recovery from race #1; expected commit parent Ref "1ccaa3ce85972818d9afee1ff047183d8836c927" but found [Ref "260477393b0e18fbf0d4fc77afa10ab0069f2d40"] )
commit 1f035559a42966383b424ebb0dd62509f8259543
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:45:51 2021 +0200
update
commit 260477393b0e18fbf0d4fc77afa10ab0069f2d40
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:45:51 2021 +0200
update
diff --git a/activity.log b/activity.log
index e388b9e..890cae5 100644
--- a/activity.log
+++ b/activity.log
@@ -1,37 +1,8 @@
00b14f31-5c38-4eb3-b16f-17140b671470 Fsck timestamp=1610259369.931977s
1968d496-fca5-4b48-ac56-b4de73d6af82 Fsck timestamp=1609853522.83856s
-283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614085193.126619514s
-283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614155865.00518998s
283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614159798.683064383s
4ea64ba8-3d84-439d-88c9-021380c3b097 Fsck timestamp=1610235815.801036s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161439.951758036s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161875.368919604s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614503753.902534237s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1618474275.133432744s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1619560801.449019468s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620081126.285146124s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620655938.401915298s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621159014.044988654s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161156.147989006s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161290.856706788s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161317.85070715s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161413.579088689s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621328885.881572492s
-8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621422002.80957147s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621935951.378910574s
a45a1abf-086e-40ea-8db5-f286a4fb69ef Fsck timestamp=1609849766.99771s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614161148.72413s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162182.895703s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162687.460662s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1618351280.306775s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619444550.542679s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619733602.694413s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619833893.739576s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620079818.367555s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620513005.967486s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620775822.267693s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621161025.720264s
-d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621239795.218301s
d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621444868.775013s
-f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1612954464.455509s
-f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614085579.354101s
f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614160037.595461s
commit 1ccaa3ce85972818d9afee1ff047183d8836c927
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:45:51 2021 +0200
update
diff --git a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log
index 0252166..3150f67 100644
--- a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log
+++ b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log
@@ -1 +1,2 @@
1621935950.772957836s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a
+1621935951.025150823s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a
commit 105d6d0ddea616d0f62f5bb78080954545966947
Author: Atemu <atemu.main@gmail.com>
Date: Tue May 25 11:45:50 2021 +0200
update
diff --git a/activity.log b/activity.log
index f925d0a..e388b9e 100644
--- a/activity.log
+++ b/activity.log
@@ -1,8 +1,37 @@
00b14f31-5c38-4eb3-b16f-17140b671470 Fsck timestamp=1610259369.931977s
1968d496-fca5-4b48-ac56-b4de73d6af82 Fsck timestamp=1609853522.83856s
+283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614085193.126619514s
+283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614155865.00518998s
283a9cfc-cc28-481f-8ab3-28564f61e278 Fsck timestamp=1614159798.683064383s
4ea64ba8-3d84-439d-88c9-021380c3b097 Fsck timestamp=1610235815.801036s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161439.951758036s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614161875.368919604s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1614503753.902534237s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1618474275.133432744s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1619560801.449019468s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620081126.285146124s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1620655938.401915298s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621159014.044988654s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161156.147989006s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161290.856706788s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161317.85070715s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621161413.579088689s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621328885.881572492s
8dc76e63-3dac-4cc3-a752-e73e81d57c1a Fsck timestamp=1621422002.80957147s
a45a1abf-086e-40ea-8db5-f286a4fb69ef Fsck timestamp=1609849766.99771s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614161148.72413s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162182.895703s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1614162687.460662s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1618351280.306775s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619444550.542679s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619733602.694413s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1619833893.739576s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620079818.367555s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620513005.967486s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1620775822.267693s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621161025.720264s
+d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621239795.218301s
d7d728f7-891a-4035-a758-c7ee80a8017a Fsck timestamp=1621444868.775013s
+f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1612954464.455509s
+f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614085579.354101s
f5f5f557-ecf2-4c2e-b704-add87173d58a Fsck timestamp=1614160037.595461s
diff --git a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log
new file mode 100644
index 0000000..0252166
--- /dev/null
+++ b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log
@@ -0,0 +1 @@
+1621935950.772957836s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a
diff --git a/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log.met b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log.met
new file mode 100644
index 0000000..2aa827d
--- /dev/null
+++ b/af5/b21/SHA256E-s675354--3b69c1b0549173d731e6259d6c817eebabcd5b43a4e5cf6b83c7498603dae6d2.log.met
@@ -0,0 +1 @@
+1621935950.772381243s day +25 month +5 year +2021
diff --git a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log
index 7fd0604..d907850 100644
--- a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log
+++ b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log
@@ -1,3 +1,4 @@
1621350596.596630026s 1 8dc76e63-3dac-4cc3-a752-e73e81d57c1a
+1621350602.99782945s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a
1621350603.027097277s 1 b0512e28-91b3-4792-a4ef-71357fc4b63a
1621445067.293418s 1 d7d728f7-891a-4035-a758-c7ee80a8017a
diff --git a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met
index b7f2ee1..2e12d4b 100644
--- a/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met
+++ b/efe/f7f/SHA256E-s2496630--0f778a1c32dfddf8098406ff28297d826b28759b4efeb1aa27a1841dd4897248.pdf.log.met
@@ -1 +1,2 @@
+1621350596.859331937s day +18 month +5 year +2021
1621445071.205197s day -18 +19 month +5 year +2021
diff --git a/schedule.log b/schedule.log
index 88f8251..f29a67c 100644
--- a/schedule.log
+++ b/schedule.log
@@ -1,7 +1,16 @@
00b14f31-5c38-4eb3-b16f-17140b671470 fsck self 1h every day at any time timestamp=1610259355.086542s
1968d496-fca5-4b48-ac56-b4de73d6af82 fsck self 1h every day at any time timestamp=1609853466.30146s
+283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 1h every day at any time timestamp=1609829311.089081351s
+283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 1h every day at any time timestamp=1609829331.993124414s
283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 1h every day at any time timestamp=1609831717.238587482s
+283a9cfc-cc28-481f-8ab3-28564f61e278 fsck self 8d13h every day at any time timestamp=1609829516.974129064s
4ea64ba8-3d84-439d-88c9-021380c3b097 timestamp=1610030955.40944s
+4ea64ba8-3d84-439d-88c9-021380c3b097 fsck self 1h every day at any time timestamp=1609857462.383747s
8dc76e63-3dac-4cc3-a752-e73e81d57c1a timestamp=1621328896.270884603s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time timestamp=1614161325.360455592s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time timestamp=1616330220.055006692s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time timestamp=1620642662.423064041s
+8dc76e63-3dac-4cc3-a752-e73e81d57c1a fsck self 1h every day at any time; fsck self 10h1m every day at any time timestamp=1616329120.414652786s
d7d728f7-891a-4035-a758-c7ee80a8017a timestamp=1621444873.058565s
+d7d728f7-891a-4035-a758-c7ee80a8017a fsck self 1h every day at any time timestamp=1616364223.127229s
f5f5f557-ecf2-4c2e-b704-add87173d58a fsck self 1h every day at any time timestamp=1610259937.483933s
</code></pre>
<p>synced/master is in the same state as it was 4d ago, the date of the merge commit git is now trying to merge.
There have only been smallfile changes made to master since.</p>
<p>I'm especially confused by the changes to schedule.log.</p>
<hr />
<p>Looks like there are some major incompatibilities with textconv and git-annex that need to be documented and/or fixed.</p>
<p>Smallfiles don't seem to be handled that well either, perhaps my other issues are related to using those almost exclusively in this repo?<br />
Automatic merges have never worked for me for example. (Though I probably wouldn't want them anyways).</p>
<p>Is anyone else having issues with this combination of features? I can't be the only one, right?</p>
comment 11http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_11_d1f454482feef0350a6fa61bdf7673cc/Atemu2021-05-31T19:07:31Z2021-05-31T19:07:31Z
The consistency check somehow enabled itself the other day and sure enough, as soon as I started the webapp, the repo corrupted itself on the MBP. Haven't seen a corruption on the Linux machine since I disabled consistency checks.
comment 12http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_12_5a13945d6bec6fa5fac84555f7ef8e21/joey2021-06-29T17:17:23Z2021-06-28T18:04:26Z
<p>This seems conclusive that the repair is somehow triggering unncessarily
and also corrupting the repo in this situation.</p>
<p>The comment #3 log shows that the repair is started, and then 1 minute
later a git object is missing.</p>
<p>(It's odd that log shows a second fsck run after the repair was already
triggered. I do not see a way that this would happen unless fscks are
scheduled very close together.)</p>
<p>The automatic repair is supposed to be a non-destructive repair; the
destructive repair only happens after prompting in the UI.</p>
<p>This also reminds me of a persistent issue with a git-annex repo, using the
assistant, on my sister's laptop corrupting itself.</p>
comment 13http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_13_6715bc3370f115f90627cfc4b06cf0e8/joey2021-06-29T17:17:23Z2021-06-28T18:13:40Z
<p>The repair process moves all pack files to a temp dir and then unpacks the
loose objects from them. So, there is a time window, when the repair is
running, where git objects that were present before will be missing. And if
the assistant stops before that is complete, it would leave it in that
state. Unpacking pack files can take a long time, so this might
be a sufficient explanation.</p>
<p>But then, something must be causing it to incorrectly think it needs
a repair in the first place. Assuming it is incorrect, of course. Either git
fsck is exiting nonzero for some reason, or git-annex is thinking
it sees git fsck complain about a missing object, that is not really
missing. While there are fsck outputs that it can misinterpret, it
double-checks by trying to cat the object, which should avoid the latter
problem.</p>
comment 14http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_14_6234b04455acc0b2b2f22a1d8c2ab475/joey2021-06-29T17:17:23Z2021-06-28T18:28:06Z
<p>To avoid moving the pack files, repair could set <code>GIT_OBJECT_DIRECTORY</code>
to a temp directory, and copy each pack file into it in turn, and unpack.
And after each unpack, move the unpacked objects from the temp directory to
the real object directory, followed by deleting the pack file (in case it's
corrupt).</p>
comment 16http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_16_7a1cdfba17f129f6bf87a71549bbaab6/joey2021-06-29T17:17:23Z2021-06-28T18:48:32Z
<p>Removing repair from the assistant (and git-annex repair) should be on the
table as a solution to this. It's a whole lot of complexity that might fix
a few user's repos sometimes, but is outside of git-annex's scope and is
mostly only used by assistant users.</p>
comment 16http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_16_586e591a1e3cc53fd0108da91492c5c2/joey2022-08-22T16:05:51Z2021-06-29T16:11:03Z
<p>I have verified that an interrupted repair results in data loss, when
.git/objects contains pack files.</p>
<p>Fixed that. Which leaves 2 open questions:</p>
<ul>
<li>Was the assistant interrupted somehow while it was running repair, in
Atemu's case? That could have been anything from the laptop being
powered off, to it being stopped explicitly, to it crashing.</li>
<li>What caused the assistant to think it needed repair in the first place?</li>
</ul>
<p>Although the answer to the second question doesn't matter much if the
data loss bug is fixed -- if there's a problem of some kind causing
unnecessary repairs, it would only be some excess CPU load.</p>
comment 17http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_17_faaa433ba8df649771469fdbcd9bb69e/Atemu2021-06-29T22:57:39Z2021-06-29T22:57:39Z
<p>Hi Joey, thank you for looking into this again.</p>
<blockquote><p>It's odd that log shows a second fsck run after the repair was already triggered. I do not see a way that this would happen unless fscks are scheduled very close together.)</p></blockquote>
<p>Note that I started the assistant again in that log. The consistency check didn't finish the first time because somehow corruption was detected and the assistant crashed itself in trying to repair (I didn't accept any repair prompts in the GUI that time IIRC).</p>
<p>AFAICT the consistency check is what triggers the issue because, since turning it off (and creating new repos on my clients), I haven't experienced a single repo corruption.</p>
<p>The schedule was set to whatever the assistant nudges you to set by default.</p>
<blockquote><p>comment 14</p></blockquote>
<p>That sounds a lot more sane.</p>
<p>Perhaps the repair functionality could use a complete overhaul. I've also found it to be extremely slow compared to simply copying missing or corrupt objects/packs from another remote.</p>
<p>Ideally it should be able to restore a git repo to a working state without .git/objects present at all as it should only needs remotes and refs.<br />
Extracting objects from the corrupt objects dir which aren't in any remote should be done if possible though because important local-only objects are a thing (stashes, additional branches, branches that are ahead of the remote's)</p>
<blockquote><p>Removing repair from the assistant (and git-annex repair) should be on the table as a solution to this.</p></blockquote>
<p>Manual repair and detection can stay IMO but any <em>automatic</em> repair needs to go entirely or be optional.</p>
<blockquote><p>Removing repair from the assistant (and git-annex repair) should be on the table as a solution to this. It's a whole lot of complexity that might fix a few user's repos sometimes, but is outside of git-annex's scope and is mostly only used by assistant users.</p></blockquote>
<p>I've actually (tried) using it from the CLI too, having it as an option there would be useful.</p>
<p>I agree that it's out of scope though. It would probably be better off as a separate project as it's useful in any git repo, not just git-annex ones. Some integration with git-annex would be appreciated though.<br />
Perhaps this functionality could be fully spun out into git-repair which could then become an optional add-on to git-annex.</p>
<blockquote><p>Was the assistant interrupted somehow while it was running repair, in Atemu's case?</p></blockquote>
<p>The laptop being powered off etc. was not the case. Though I do shut down the daemon quite often because of https://git-annex.branchable.com/bugs/OSX<strong>58</strong>_Pushed_changes_are_autocommited/ (via the GUI or --stop). Do the shutdown and restart procedures check for a running repair?</p>
<p>I also sometimes had to kill all git processes for one reason or another but I don't think that ever happened after I pressed the repair button (I wait that out for good reason).</p>
<hr />
<p>So current issues distilled from this thread so far:</p>
<ul>
<li>The scheduled consistency check can trigger automatic(!) repairs and even do so erroneously</li>
<li>This repair procedure does something destructive without a prompt</li>
<li>Applying the destructive repair to a working repo corrupts it</li>
<li>Repair can't actually repair the mishap despite all the data being available</li>
<li>Repair would probably be better off as a dedicated project outside of git-annex</li>
</ul>
comment 18http://git-annex.branchable.com/bugs/Git_repos_corrupt_themselves/comment_18_238bbbcabd626c33acc14f930a7d536c/joey2022-08-22T16:05:51Z2021-06-30T15:36:39Z
<p>If the assistant crashed in the middle of a repair that confirms my
analysis, and my fix will avoid both the data loss problem <em>and</em> the crash.</p>
<p>Only remaining question to me is why would it trigger an unnecessary repair.</p>
<p>But that could be anything that causes git fsck to exit nonzero. Or it
might be that git fsck found an actual problem, but not one that was
preventing the repo from working. Eg, a missing/corrupt object used
somewhere deep in the history.</p>
<p>(Note that git-repair already exists. git-annex integrates it.)</p>