Recent comments posted to this site:

I've removed Git for windows 2.5, and installed the latest msysgit 1.9, and it seems that this works much better.

However, I have to invoke git-annex from the command-line through its full path with : "C:\Program Files\Git\cmd\git-annex.exe" get .

I guess this gives hope :-)

Comment by OlivierBerger Thu Aug 27 15:02:14 2015

Here's a transcript of the --debug output (edited to mask file name and servername)

# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

$ git annex get big\ file --debug
[2015-08-27 16:21:36.8295694] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","ls-files","--cached","-z","--","big file"]
get big file [2015-08-27 16:21:36.8495982] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","show-ref","git-annex"]
[2015-08-27 16:21:36.8796414] process done ExitSuccess
[2015-08-27 16:21:36.8796414] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","show-ref","--hash","refs/heads/git-annex"]
[2015-08-27 16:21:36.8996702] process done ExitSuccess
[2015-08-27 16:21:36.8996702] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","log","refs/heads/git-annex..f98a471b32f963a0bf816800d766690d53ef150d","-n1","--pretty=%H"]
[2015-08-27 16:21:36.919699] process done ExitSuccess
[2015-08-27 16:21:36.9297134] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","log","refs/heads/git-annex..15e7ab1f3b70629db852f8565bf988b25af02ec6","-n1","--pretty=%H"]
[2015-08-27 16:21:36.9497422] process done ExitSuccess
[2015-08-27 16:21:36.9497422] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","log","refs/heads/git-annex..1e04e80b99361e5b516d3a266bbc7b1d1478dff5","-n1","--pretty=%H"]
[2015-08-27 16:21:36.9897998] process done ExitSuccess
[2015-08-27 16:21:36.9897998] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","cat-file","--batch"]
(from origin...) [2015-08-27 16:21:37.0098286] read: rsync ["--progress","--inplace","-e","'ssh' '-T' 'gitolite3@server' 'git-annex-shell ''sendkey'' ''/~/testing'' ''--debug'' ''SHA256E-s264620717--23e70aae588a049d52909cd68067cf2885429ae557f52e3f2d6033260c3ad9ea.mp4'' --uuid f8ef7445-2ccc-4871-a95a-7e4325fc763d ''--'' ''remoteuuid=8fb9f88a-1105-4278-aa85-7b5c78a0e0e5'' ''direct=1'' ''associatedfile=big file'' ''--'''","--","dummy:",".git/annex/tmp/SHA256E-s264620717--23e70aae588a049d52909cd68067cf2885429ae557f52e3f2d6033260c3ad9ea.mp4"]
rsync: Failed to exec ssh: No such file or directory (2)
rsync error: error in IPC code (code 14) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/pipe.c(84) [Receiver=3.0.9]
rsync: writefd_unbuffered failed to write 4 bytes to socket [Receiver]: Broken pipe (32)
rsync error: error in IPC code (code 14) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/io.c(1532) [Receiver=3.0.9]
[2015-08-27 16:21:37.0799294] process done ExitFailure 14

# End of transcript or log.
Comment by OlivierBerger Thu Aug 27 14:26:54 2015

I think I'm having the same problem. See my comments on git annex sync deleted a bunch of files (not expected)

I've run git annex sync or git annex webapp on the laptop annex, then git annex sync on the external drive. I'm pretty sure some of the syncs have been interrupted. Does it help to see the .git/config from the external drive?

    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    url = /home/edward/annex
    fetch = +refs/heads/*:refs/remotes/origin/*
    annex-uuid = 38d109c9-7f5f-47cd-b15a-7b2beac22c64
[branch "master"]
    remote = origin
    merge = refs/heads/master
    uuid = 822dec0f-a0d3-42f6-b0dc-a47b6bf91944
    version = 5
[remote "x230"]
    url = /home/edward/annex
    fetch = +refs/heads/*:refs/remotes/x230/*
    annex-uuid = 38d109c9-7f5f-47cd-b15a-7b2beac22c64

Observations about my config, I have bare = false, which is correct. Do you think it is a problem that I have two remotes, "origin" and "x230" pointing at the same location?

Comment by edward Thu Aug 27 09:24:38 2015

I have the same problem with another annex. I ran the webapp using git annex webapp in the annex on my laptop hard drive. It seemed to update and sync with the annex on my external USB drive, but now when I run git status in the annex directory on the drive it has staged lots of deletes. I don't understand what is going on here. Both annexes are in indirect mode.

On branch master
Your branch is behind 'origin/master' by 61 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        deleted:    4angle_tech_ltd/032-570247_20150312-074413_30313.pdf
        deleted:    4angle_tech_ltd/032-570247_20150312_09486613.pdf
        deleted:    android/RUU_PRIMO_U_ICS_40A_HTC_Europe_2.22.401.1_Radio_20.76.30.0835U_3831.19.00.120_release_273801_signed.exe
        deleted:    article/Fukuyama-End-of-history-article.pdf
        deleted:    article/The Selling of the Avocado - Health - The Atlantic.html

The symlinks and the data are still on the disk, as is the data that the symlinks point to.

$ ls 4angle_tech_ltd -l
total 8
lrwxrwxrwx 1 edward edward 197 Mar 15 08:39 032-570247_20150312-074413_30313.pdf -> ../.git/annex/objects/qM/mj/SHA256E-s21598--efb39974c5253d8059f0fe991c1b76aba8455d8439eefd6cd8943503f85109c0.pdf/SHA256E-s21598--efb39974c5253d8059f0fe991c1b76aba8455d8439eefd6cd8943503f85109c0.pdf
lrwxrwxrwx 1 edward edward 197 Mar 15 08:39 032-570247_20150312_09486613.pdf -> ../.git/annex/objects/JX/XP/SHA256E-s93692--8e88ca5071bc2155acfe16a41c9c6b756fecc6515cfb7907105dd1a83e73f57a.pdf/SHA256E-s93692--8e88ca5071bc2155acfe16a41c9c6b756fecc6515cfb7907105dd1a83e73f57a.pdf
Comment by edward Thu Aug 27 08:29:55 2015

This part:

  File "/usr/lib/python2.7/", line 833, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 0: ordinal not in range(128)

Tells me that you need to read glacier-cli problem report #61.

There is a one-line code change in a library named boto (glacier-cli depends on it) which will fix this. (And probably that change will get merged in sometime, so you won't have to do this anymore.)

Comment by dave Wed Aug 26 21:54:08 2015

just to be complete: during setup some useful assistant may have created rsa keys so that the assistant can access remote hosts without password and following the update of .git/config you probably will need to update .ssh/config as well.

Like many things with git annex and the assistant, its just "like dropbox with your own cloud", but you need first to become expert in editing .git/config and .ssh/config.

Comment by hasard Wed Aug 26 17:49:16 2015

I wonder how should I utilize this new API (WHEREIS) in my case: it seems just to lead to duplication of whereis information in my case of a special remote to support extracting of content from archives. If I make it to reply with the same url (which is not "public" per se, i.e. can't be used by annex directly) I just get it duplicated:

$> git annex whereis simple.txt
whereis simple.txt (1 copy) 
    82025765-5cac-4571-91ed-637620ec6fc7 -- [annexed-archives]

  annexed-archives: dl+archive:SHA256E-s173--5df2eeab61ea7d6479533d4e6b07c6bcfae46e040cad8cb1fc579f9f18c90790.tar.gz/a/d/%20%22%27%3Ba%26b%26cd%20%60%7C%20
  annexed-archives: dl+archive:SHA256E-s173--5df2eeab61ea7d6479533d4e6b07c6bcfae46e040cad8cb1fc579f9f18c90790.tar.gz/a/d/%20%22%27%3Ba%26b%26cd%20%60%7C%20

if I "explain" it a bit, also somewhat duplicate:

annexed-archives: file a/d/%20%22%27%3Ba%26b%26cd%20%60%7C%20 within archive SHA256E-s173--5df2eeab61ea7d6479533d4e6b07c6bcfae46e040cad8cb1fc579f9f18c90790.tar.gz
annexed-archives: dl+archive:SHA256E-s173--5df2eeab61ea7d6479533d4e6b07c6bcfae46e040cad8cb1fc579f9f18c90790.tar.gz/a/d/%20%22%27%3Ba%26b%26cd%20%60%7C%20

But if I just reply with "WHEREIS-FAILURE" it becomes more sensible (no duplicates), but I feel that then better documentation for this feature get adjusted to describe that it is only to complement information already known to annex, and not really to "provide any information about ways to access the content of a key stored in it". Or have I missed the point? ;)

Comment by EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s- [] Wed Aug 26 14:22:49 2015

I see there is an Ubuntu Unity search lens for recoll: It should be possible to integrate git-annex metadata with that ..

Comment by jean.jordaan Wed Aug 26 13:04:12 2015

Hi Joey,

Thanks for the hand, it started uploading once I had manually created the vault but then borked with:

fozz@cobol:~/Store/family_pictures $ git annex --verbose copy 201/2011/05/07/P1010004.RW2 --to familypictures-glacier
copy 201/2011/05/07/P1010004.RW2 (gpg) (checking familypictures-glacier...) (to familypictures-glacier...) 
81%           4.0MB/s 0sTraceback (most recent call last):
  File "/home/fozz/.vendor/bin/glacier", line 730, in <module>
  File "/home/fozz/.vendor/bin/glacier", line 716, in main
  File "/home/fozz/.vendor/bin/glacier", line 498, in archive_upload
    file_obj=self.args.file, description=name)
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 177, in create_archive_from_file
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 219, in write
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 61, in write
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 75, in _send_part
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 222, in _upload_part
    self.uploader.upload_part(self.next_part_index, part_data)
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 129, in upload_part
    content_range, part_data)
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 1279, in upload_part
  File "/usr/local/lib/python2.7/dist-packages/boto/glacier/", line 114, in make_request
  File "/usr/local/lib/python2.7/dist-packages/boto/", line 1071, in make_request
  File "/usr/local/lib/python2.7/dist-packages/boto/", line 943, in _mexe
    request.body, request.headers)
  File "/usr/lib/python2.7/", line 979, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/", line 1013, in _send_request
  File "/usr/lib/python2.7/", line 975, in endheaders
  File "/usr/lib/python2.7/", line 833, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 0: ordinal not in range(128)
gpg: [stdout]: write error: Broken pipe
gpg: DBG: deflate: iobuf_write failed
gpg: build_packet failed: file write error
gpg: [stdout]: write error: Broken pipe
gpg: iobuf_flush failed on close: file write error
gpg: symmetric encryption of `[stdin]' failed: file write error
git-annex: fd:17: hPutBuf: resource vanished (Broken pipe)
git-annex: copy: 1 failed
Comment by Wed Aug 26 08:30:53 2015

There are going to be many configurations where auto-enable of a special remote fails. Ie, when creds are needed. While some of these could be detected and skipped, I sort of like the simplicity of having it try to enable everything.

Maybe a command is also needed to autoenable remotes after the first git-annex init? Since it's actually ok to re-run git-annex init in an initalized repo, I suppose it could just be run again.

Comment by Tue Aug 25 18:25:44 2015