Recent comments posted to this site:

I just saw your post and agree with you. Maybe we shall create a bug report. I would also need to keep my files in original timestamps. (may not mind about permissions though).
Comment by Emre Fri Sep 19 21:31:23 2014
If box does not have the encryption keys in this "shared encryption" scenario and if you had only your computer and this remote repo, does that mean losing your computer (ie your git repository) would mean also losing access to those encrypted content? So, actually an encrypted remote, even if marked as full backup, is not actually a backup unless you have a 3rd computer that has the same git repo, in the case of losing your original computer or accidentally wiping it...
Comment by Emre Fri Sep 19 21:24:19 2014

Need to understand something. I just installed git-annex to my home PC, to my vps and to my office notebook. When I add the VPS as a shared encryption remote on both home PC and office notebook, they do not sync files. I tried assigning Full Backup and Transfer category to the VPS, but it did not help. The home pc and office notebook just sit there and do nothing to sync. For test purpose, I started with empty annex directory and started populating files on the home. It does sync to the VPS, but the office notebook is unaware of it and even if I try a manual sync, nothing happens.

On the other hand, if I start from scratch, add local repo's on both home and office ones, then add XMPP Account on both, then I'm asked to add cloud account, and once I do it on one end, it appears on the other and when I enable sync, they start to work. However, this is not a good scenario as the work network uses a Proxy and as far as I understand, git-annex does not yet support it (did not find any options for it in the webapp).

So did I get right that an XMPP connection is required between indirectly connected repositories on different connections to be able to sync? Ie a having a middleman computer with a repo is not enough.

Comment by Emre Fri Sep 19 19:37:36 2014
Your version supports both new and old style chunking. Which is used depends on how the S3 remote was configured when it was set up. It can't really be changed w/o re-setting up the remote. You can check which is used by git show git-annex:remote.log, find the line for the UUID of the remote, and see if it has chunk= (new chunking) or chunksize= (old chunking).
Comment by joeyh.name Fri Sep 19 18:33:17 2014

delete the synced/master branch: $ git branch -d synced/master Branch synced/master entfernt (war 20ec8b3).

then call annex merge: $ git annex merge merge git-annex ok

check branches: $ git branch -a git-annex * master synced/master remotes/origin/git-annex remotes/origin/master and there is the synced/master branch again.

But that's not my problem. My problem was: how to use annex with a central repository. I done that by deleting all remote synced/* branches. And now I'm updating the git-annex branch by git fetching and git annex mergeing again.

PS: the MD for code blocks is broken

Comment by Torkaly Fri Sep 19 18:22:36 2014

Brilliant! Thanks for taking time to analyze the issue and taking the bug to gcrypt.

[I'm surprised that a different key than my git-annex key is used and that it's a symmetric key, but I will explore the technology on my own].

Comment by rasmus Fri Sep 19 06:29:58 2014
Was the version I listed above not using the new chunking from the s3-aws branch? How do I determine if my version of git-annex was built with the new or old chunking?
Comment by annexuser Fri Sep 19 04:43:42 2014

These objects are the ones written by git-remote-gcrypt when pushing to a remote. That's why the weird dates, root pseudo-commit, crazy filenames, and big gpg encrypted blobs. All countermeasures that git-remote-gcrypt uses to keep your encrypted git remote safe and not leak information about what's in it.

So, this is a bug in git-remote-gcrypt. It needs to clean these objects up after pushing them! (Also after failed pushes.)

Comment by joeyh.name Fri Sep 19 02:43:22 2014

Hi Joey,

Thanks for giving the thread a more appropriate title and thanks for the helpful messages.

Let me start with the easy points:

  • Looking at my log file of installed packages I have never used etckeeper on my system. So unless it could have entered through annex then I think we can rule that one out.
  • According to git log the repos are from January 2014 where I restarted my repos.

      commit 029a8e76ab5f66aa4390987130985550a1ccd69c
      Author: Rasmus <w530@domain.eu>
      Date:   Thu Jan 23 21:06:13 2014 +0100
    
      created repository
    
  • When I start git repos I typically just use "init" so I don't think I did the 2012 commits.

  • I checked out one of the 74mb files. When I do file test.blob it shows test.blob: GPG symmetrically encrypted data (CAST5 cipher). But none of my normal passwords worked. Could such a gpg'ed file be from local network connections where the assistant asks for a passphrase? I'm pretty sure that my transfer repo has only been using gcrypt and I believe I "restarted" my repos because I switched to gcrypt repos. Also, my transfer repo is 10Gb as well which sounds big for transfer repo.

I performed a similar "analysis" on the conf.annex repo which should contain mostly no binary files (some 16x16 pngs etc).

conf.annex has 727 unreachable objects and 3477 commits in total. Of these 338 are commits. Here's an example of a larger commit message of an unreachable commit.

commit 601c10f9512e8d3502d9dd52ef409560ebb5b7e0
Author: root <root@localhost>
Date:   Mon Dec 31 19:00:01 2012 -0400

     Initial commit

 diff --git a/6fbbea493cdec9d912d256374199cc4c012022d35524c8789a7aceeb953442a5 b/6fbbea493cdec9d912d256374199cc4c012022d35524c8789a7aceeb953442a5
 new file mode 100644
 index 0000000..ea5fcc3
 Binary files /dev/null and b/6fbbea493cdec9d912d256374199cc4c012022d35524c8789a7aceeb953442a5 differ
 diff --git a/91bd0c092128cf2e60e1a608c31e92caf1f9c1595f83f2890ef17c0e4881aa0a b/91bd0c092128cf2e60e1a608c31e92caf1f9c1595f83f2890ef17c0e4881aa0a
 new file mode 100644
 index 0000000..a86c1a9
 Binary files /dev/null and b/91bd0c092128cf2e60e1a608c31e92caf1f9c1595f83f2890ef17c0e4881aa0a differ
 diff --git a/9da3fcfc1635c674012c35d90c21adce3c35440e629d64fe117fe349a6b3e194 b/9da3fcfc1635c674012c35d90c21adce3c35440e629d64fe117fe349a6b3e194
 new file mode 100644
 index 0000000..ef1d71c
 Binary files /dev/null and b/9da3fcfc1635c674012c35d90c21adce3c35440e629d64fe117fe349a6b3e194 differ
 diff --git a/ad4ae79c29b3756f7e41257db7454f3c319112d06385a8bc12d28209a82f2594 b/ad4ae79c29b3756f7e41257db7454f3c319112d06385a8bc12d28209a82f2594
 new file mode 100644
 index 0000000..61d3e5b
 Binary files /dev/null and b/ad4ae79c29b3756f7e41257db7454f3c319112d06385a8bc12d28209a82f2594 differ
 diff --git a/bd0e9cb492077e0c090bc62892c8de438c51a956c8215b2c68de7caa7e2431cc b/bd0e9cb492077e0c090bc62892c8de438c51a956c8215b2c68de7caa7e2431cc
 new file mode 100644
 index 0000000..92e9bd7
 Binary files /dev/null and b/bd0e9cb492077e0c090bc62892c8de438c51a956c8215b2c68de7caa7e2431cc differ

Across all commits 6006 objects are mentioned, but only 371 are unique.

I checked out one blob and again file reports GPG symmetrically encrypted data (CAST5 cipher). Interesting for conf.annex I get this line when trying to decrypt

gpg: DBG: cleared passphrase cached with ID: SBF83A0F822D0F664

For doc.annex I get

gpg: DBG: cleared passphrase cached with ID: S32DEAD1E8DD06A4D

And on my other computer I see a third ID. I'm not sure if this means anything when files are symmetrically encrypted, though.

Comment by rasmus Fri Sep 19 00:43:56 2014