bugs/Packfile does not match digest: gcrypt with assistantgit-annexhttp://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/git-annexikiwiki2024-02-05T14:58:33Zcomment 1http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_1_bf61defadd3be9bc867be961b3149be6/joey2016-04-13T18:16:53Z2016-04-13T18:10:03Z
<p>That error message is certainly not coming from git-annex. It sounds like a
git error message, but I don't find it in the current git source code.</p>
<p>What it sounds like is some form of corruption of the git repository,
probably to a pack file. Since git-annex doesn't have anything to do with
writing such files, it's hard to see how this could be a bug in git-annex.</p>
<p>This kind of corruption problem tends to happen when a disk loses data,
perhaps having to do with an unclean shutdown. Or perhaps it received
bad git repository data from the VPS.</p>
<p>Suggest you run <code>git fsck</code> and if it reports problems, you may be ableo
to fix them by running <code>git annex repair</code>.</p>
comment 2http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_2_8a8ead857917e7b843f1bdfe07b5d52d/Don2016-04-14T19:31:52Z2016-04-14T19:31:52Z
I've been watching this thread because I have the same issue. Joey, it seems like this error message is coming from the <code>get_verify_decrypt_pack()</code> function in git-remote-gcrypt.
comment 3http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_3_2432df64f920f3962b6ba02adaa9b61d/joey2016-04-14T20:08:45Z2016-04-14T19:57:58Z
<p>Ok, so definitely not a bug in git-annex then.</p>
<p><code>get_verify_decrypt_pack</code> downloads an encrypted pack file, and then
uses gpg to hash the pack file and compares this to the hash
encoded in the name of the pack file.</p>
<p>So, this could happen if the pack files in the gcrypt remote have gotten
the wrong data into them. Or it could be a bug of some kind in
git-remote-gcrypt.</p>
A caution on thishttp://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_4_536fbc38a7f2e7ab42188b5a8700e2cf/jgoerzen2016-06-23T21:24:49Z2016-06-23T21:24:49Z
Although I believe Joey's analysis is correct, I want to interject a caution: while investigating this issue on my own system, I discovered that a gcrypt remote has an unencrypted pack file and unencrypted index, which somehow contained actual data of mine. I would consider it a git-annex bug that data (file contents) wound up in a pack file (this is a git-annex assistant repo) but a gcrypt bug that it made it there unencrypted.
Same issuehttp://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_5_f45d09939527be362c586ae1d470afe4/pot2016-07-25T18:23:13Z2016-07-25T18:23:13Z
Anyone got any ideas on how to fix this, or move forward without starting again and trashing the gcrypt repos? Bit of a serious bug this, if it can't be repaired. I was using git-annex for backups, but finding your backups are corrupt would be a bit of a nightmare!
comment 6http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_6_88178fa312f351a7667b3b2c48335b02/jgoerzen2016-07-25T19:09:09Z2016-07-25T19:09:09Z
This was concerning enough to me that I wound up switching to Syncthing for this particular use case.
Transcrypt? http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_7_cd6a850119dd1aa0d9bb34c6d8fe560c/branchable2016-08-05T07:28:27Z2016-08-05T07:28:27Z
<p>Any experience with using transcrypt instead of git crypt?</p>
<p>https://github.com/elasticdog/transcrypt</p>
Hmmm, seems odd such a serious bug is allowed to linger without serious investigationhttp://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_8_39139e25175ae265a4dc15f0b6b7b618/pot2016-08-09T19:22:48Z2016-08-09T19:22:48Z
<p>Should a developer not be all over this, would seem like a very major issue with the gcrypt backend (and annex's interaction with it) that this outcome could even be possible. You can't be using an encrypted store, which becomes irrepairably corrupted and with some of your data sat outside, unencrypted!</p>
<p>Any thoughts from more knowledgeable developers on how to fix, investigate further, or do something useful on this front?</p>
comment 9http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_9_16c004fe7c77cbbba113787ce6410c9e/spwhitton2016-08-09T20:17:47Z2016-08-09T20:17:47Z
<p>I'm the git-remote-gcrypt maintainer, but as it stands it's not really possible to do anything about this bug without steps to reliably reproduce the corrupted/unencrypted remote repository.</p>
<p>I have been using git-remote-gcrypt with git-annex and with plain git repos for more than two years and I have never encountered anything like this issue. So there's no starting point to work on a fix for the issue.</p>
comment 10http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_10_9d4fdddd7ab05de9dfa4cc90f1051ef5/joey2016-09-21T22:03:16Z2016-09-21T21:36:18Z
<p>@Don and @pot, did you actually see un-encrypted data end up in the gcrypt
repository? Or by "same issue" do you mean you saw the same error message,
perhaps due to a very different cause?</p>
<p>@jgoerzen, when you say that the gcrypt reposotory had an "unencrypted
index", are you referring to a regular git index file, or to a pack's .idx
file?</p>
<p>It would be very strange if a gcrypt repository had an index file.
I've never been able to use git-remote-gcrypt with a non-bare repository,
and bare repositories don't normally have index files.</p>
inodes....http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_11_bbd8b537e277d24df254ed058ad40e24/pot2016-09-30T22:00:31Z2016-09-30T22:00:31Z
<p>I think I may have tracked this down partly, I ran out of inodes on a removeable drive and started getting the same error. Will try and rectify underlying inode issue and then see.</p>
<p>(secondary point, how on earth has my docs directory used up all 59 million inodes available to it when put into an annex???)</p>
comment 12http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_12_5090e41cf96dfe542d1d2326aebc556c/Schnouki2017-05-29T09:56:59Z2017-05-29T09:56:59Z
<p>Not sure how I did it, but I have 2 repos that give me the same error. (Perhaps it happened when killing the assistant in the middle of a sync? No idea…)</p>
<p>However I "solved" the issue for one repo:</p>
<ol>
<li>cloned the gcrypt remote to a temporary directory</li>
<li>removed all the (encrypted) files with <code>git rm</code>, committed that, and pushed it (the repo is hosted on Gitlab and I can't just remove the branch or do something like that)</li>
<li>removed the <code>gcrypt-id| from my</code>.git/config`</li>
<li>ran <code>git annex sync</code> again</li>
</ol>
<p>And voilà! It works <img src="http://git-annex.branchable.com/smileys/smile.png" alt=":)" /></p>
<p>I don't have any content in this remote, only metadata, so it's easy to do. But I still don't know what caused this.</p>
A hint?http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_13_2bfa76c4083ff9c1d2a4adfe38ced2f0/jgoerzen2017-10-02T02:05:32Z2017-10-02T02:05:32Z
<p>Hi folks,</p>
<p>I no longer use git-annex so I can't comment directly on this. However, in some testing with git-remote-gcrypt, I came across an issue where changes seem to be lost. In short, every git push behaves as if --force were given. Details in <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=877464">Debian but #877464</a>.</p>
comment 14http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_14_2274479a1eef9ffc6a53ffb65e2c3511/xwvvvvwx2020-06-17T01:18:32Z2019-11-21T17:32:31Z
<p>I just reproduced this when pushing to a gcrypt remote on rsync.net using the assistant. There is only one client pushing to the gcrypt remote.</p>
<p>It was during the initial sync of a moderately large amount of data (~22G), perhaps this has something to do with it?</p>
<p>I could reproduce the issue by cloning with gcrypt directly (<code>git clone gcrypt::ssh://....</code>).</p>
<p>I was able to recover by following the steps outlined in Schnouki's comment (#12), but this is obviously quite an unsatisfactory fix.</p>
<p>I am using annex to replicate important personal data, and I find this issue highly concerning.</p>
<p>Foolishly, I did not keep a copy of the bad repo before I forced pushed over it on the remote, so I do not have a copy available to experiment with <img src="http://git-annex.branchable.com/smileys/sad.png" alt=":(" /></p>
<hr />
<h2>logs</h2>
<p><code>daemon.log</code> excerpt: <a href="https://ipfs.io/ipfs/QmcoPuTLY2v5FWPABQLVwgyqW5WdsvkBbVS33cJh6zjzi4">https://ipfs.io/ipfs/QmcoPuTLY2v5FWPABQLVwgyqW5WdsvkBbVS33cJh6zjzi4</a></p>
<p><code>git clone</code> output:</p>
<pre><code>[annex@xwvvvvwx:~]$ git clone gcrypt::ssh://<URL> remote
Cloning into 'remote'...
gcrypt: Decrypting manifest
gpg: Signature made Thu 21 Nov 2019 04:02:40 PM CET
gpg: using RSA key 92E9F58E9F8C6845423C251AACD9A98951774194
gpg: Good signature from "git-annex <annex@xwvvvvwx.com>" [ultimate]
gcrypt: Remote ID is :id:tWrcOFKu2yX7y+jLDLxm
gcrypt: Packfile e7b619864585f3c921b491fd041127cf0ae33c4480810610dcb2e37ec46a82be does not match digest!
fatal: early EOF
</code></pre>
<p><code>git annex version</code>:</p>
<pre><code>git-annex version: 7.20191114
build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.2.0.1 ghc-8.6.5 http-client-0.6.4 persistent-sqlite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
operating system: linux x86_64
supported repository versions: 7
upgrade supported from repository versions: 0 1 2 3 4 5 6
</code></pre>
Duplicate 'gcrypt-id' may be the issue?http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_15_e8deee63316c8af39d7cf0996d26ecba/sirio2020-06-17T01:18:32Z2019-12-29T22:10:26Z
<p>Had a repo exhibit this behavior just now:</p>
<ul>
<li>commit graph <code>XX -> YY</code></li>
<li>host <code>A</code> @ commit <code>YY</code></li>
<li>host <code>B</code> @ commit <code>XX</code> (1 behind)</li>
<li>remotes <code>hub</code> and <code>lab</code> both @ commit <code>XX</code></li>
<li><code>B</code> pushes and pulls from both <code>hub</code> and <code>lab</code>: OK</li>
<li><code>A</code> pushes to <code>hub</code> (updates to commit <code>YY</code>): OK</li>
<li><code>B</code> pulls from <code>hub</code>: FAIL with <em>Packfile does not match digest</em></li>
<li><code>B</code> pulls from <code>lab</code>: OK</li>
<li><code>B</code> pushes to <code>hub</code>: FAIL with <em>Packfile does not match digest</em></li>
<li><code>A</code> pulls from <code>hub</code>: OK</li>
<li><code>A</code> pulls from <code>lab</code>: OK</li>
</ul>
<p>When looking in <code>.git/config</code> I noticed that <code>remote.hub.gcrypt-id</code> and <code>remote.lab.gcrypt-id</code> were identical.</p>
<p>To fix, I:</p>
<ul>
<li>removed <code>remote.hub.gcrypt-id</code> from <code>.git/config</code> on both <code>A</code> and <code>B</code></li>
<li>deleted and re-created a blank repo on <code>hub</code></li>
<li><code>git push hub</code> on <code>B</code></li>
<li><code>git pull hub master</code> on <code>A</code></li>
</ul>
<p>This resulted in a new and unique value for <code>remote.hub.gcrypt-id</code>, which is the same on both <code>A</code> and <code>B</code>.</p>
<p>Have not had time to dig into why but this is the only thread I can find about this problem so I figured I would log this somewhere for posterity.</p>
comment 16http://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_16_2139bcb34591be137ff3da959f08bc76/branch2023-10-23T09:56:59Z2023-10-23T09:56:59Z
<p>This issue is still present, when using the assistant with git-remote-gcrypt.</p>
<p>Although one can recover from this issue by resetting the repository, this is handicaping, as we have to continuously monitor whether pushes have been successful, which contradicts the idea of the assistant.</p>
<p>This may be anecdotal, but the push before the 'Packfile *** does not match digest!' error, have had some network issues, as follows:</p>
<pre><code>gcrypt: Due to a longstanding bug, this push implicitly has --force.
gcrypt: Consider explicitly passing --force, and setting
gcrypt: gcrypt's require-explicit-force-push git config key.
gcrypt: Encrypting to: -r ***
gcrypt: Requesting manifest signature
error: src refspec refs/gcrypt/gitception+ does not match any
fatal: the remote end hung up unexpectedly
error: failed to push some refs to ***
error: failed to push some refs to ***
</code></pre>
Same issue after kernel oopshttp://git-annex.branchable.com/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_17_7fcd45354f94756c34b2f3f912972c19/felix2024-02-05T14:58:33Z2024-02-05T14:58:33Z
<p>Just to add one more data point – I ran into the situation after a kernel oops had caused my rootfs to become read-only. So maybe related to the "out of inodes" issue reported above.</p>
<pre><code>$ git push
gcrypt: Decrypting manifest
gpg: error getting version from 'scdaemon': No SmartCard daemon
gpg: Signature made Mo 05 Feb 2024 15:22:01 CET
gpg: using RSA key XXX
gpg: Good signature from "XXX" [ultimate]
Primary key fingerprint: XXX
gcrypt: Due to a longstanding bug, this push implicitly has --force.
gcrypt: Consider explicitly passing --force, and setting
gcrypt: gcrypt's require-explicit-force-push git config key.
gcrypt: Repacking remote origin, ...
gcrypt: Packfile XXX does not match digest!
fatal: early EOF
error: failed to push some refs to 'XXX'
</code></pre>
<p>XXX is obviously my redactions. I was not running the assistant, just plain git-annex.</p>
<p>There was no plaintext on the remote and the only way for me to recover was to nuke the remote repo.</p>
<p>I kept copies of both repos, though I unfortunately cannot share the unencrypted one.</p>