Recent changes to this wiki:

diff --git a/doc/forum/v6_thinning_seems_to_be_not_working_for_me.mdwn b/doc/forum/v6_thinning_seems_to_be_not_working_for_me.mdwn
new file mode 100644
index 0000000..a87421d
--- /dev/null
+++ b/doc/forum/v6_thinning_seems_to_be_not_working_for_me.mdwn
@@ -0,0 +1,42 @@
+git-annex version: 6.20160524+gitg2b7b2c4-1~ndall+1
+
+repo config:
+
+[core]
+	repositoryformatversion = 0
+	filemode = true
+	bare = false
+[annex]
+	uuid = c5c203ec-9353-4213-8af5-6fcb8de36ca2
+	version = 6
+	thin = true
+[filter "annex"]
+	smudge = git-annex smudge %f
+	clean = git-annex smudge --clean %f
+
+
+
+copied a big file to the working directory
+
+git commit
+
+
+Locking the big file creates the symlink, with the file in the annex.
+
+Unlocking the big file creates a real file in the working dir, and the annex file stays there.
+
+The total dir is therefore twice the size of the big file.
+
+ls -li shows a unique inode for each file.
+
+
+It seems thinning is not doing what I expected:
+
+Which is: unlocking a file creates a hard link in the working directory, linked to the annex file, with
+the resulting dir size being the size of the one big file.
+
+Are my expectations correct?
+
+Is there something I am missing?


diff --git a/doc/forum/v5_to_v6_upgrade_strategy/comment_2_ef2648a58037b867e2d745eb4b3a15a2._comment b/doc/forum/v5_to_v6_upgrade_strategy/comment_2_ef2648a58037b867e2d745eb4b3a15a2._comment
new file mode 100644
index 0000000..e39d8f2
--- /dev/null
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ subject="comment 2"
+ date="2016-06-28T23:10:07Z"
+ content="""
+Many thanks for the reply. I did not add too much detail to my post as it was my first, and I was a bit shy.
+
+I did do all of the steps indicated re v6. The binary has been fine for me also.
+
+My question is a bit more complex as I am looking to apply v6 at some point to a set of distributed repos. Essentially it has the typical duplicated system topology: 2 remotes, several clients, each of which is linked to both remotes. Thus it will survive multiple failure scenarios.
+
+I would be looking at the strategy of the order of repo upgrade. Say, a client first, and run a set of tests. And so on. Remotes last perhaps.
+
+In any case I built a test bed as above and have been experimenting with a single client at a v6 repo.
+
+So far it has been smooth, but right now I seem to be stuck where thinning is not having any effect: I still have a particular big file (1.5GB) in both the annex and the working dir. Lock remove the big file from the working dir, and leaves a link, and unlock restores the big file in the working dir. Yet, the annex still contains the big file no matter what. I was under the expectation that one of the big file copies would go away.
+"""]]


removed
diff --git a/doc/forum/v5_to_v6_upgrade_strategy/comment_2_c84054b4174bb89009147ab874891f7a._comment b/doc/forum/v5_to_v6_upgrade_strategy/comment_2_c84054b4174bb89009147ab874891f7a._comment
deleted file mode 100644
index 6933cb7..0000000
+++ /dev/null
@@ -1,17 +0,0 @@
-[[!comment format=mdwn
- subject="comment 2"
- date="2016-06-28T23:04:57Z"
- content="""
-Many thanks for the reply. I did not add too much detail to my post as it was my first, and I was a bit shy.
-
-I did do all of the steps indicated re v6. The binary has been fine for me also.
-
-My question is a bit more complex as I am looking to apply v6 at some point to a set of distributed repos. Essentially it has the typical duplicated system topology: 2 remotes, several clients, each of which is linked to both remotes. Thus it will survive multiple failure scenarios.
-
-I would be looking at the strategy of the order of repo upgrade. Say, a client first, and run a set of tests. And so on. Remotes last perhaps.
-
-In any case I build a test bed as above and have been experimenting with a single client at a v6 repo.
-
-So far it has been smooth, but right now I seem to be stuck where thinning is not having any effect: I still have a particular big file (1.5GB) in both the annex and the working dir. Lock removes the big file from the working dir, and leaves a link, and unlock restores the big file in the working dir. Yet, the annex still contains the big file no matter what. I was under the expectation that one of the big file copies would go away.
-"""]]


diff --git a/doc/forum/v5_to_v6_upgrade_strategy/comment_2_c84054b4174bb89009147ab874891f7a._comment b/doc/forum/v5_to_v6_upgrade_strategy/comment_2_c84054b4174bb89009147ab874891f7a._comment
new file mode 100644
index 0000000..6933cb7
--- /dev/null
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ subject="comment 2"
+ date="2016-06-28T23:04:57Z"
+ content="""
+Many thanks for the reply. I did not add too much detail to my post as it was my first, and I was a bit shy.
+
+I did do all of the steps indicated re v6. The binary has been fine for me also.
+
+My question is a bit more complex as I am looking to apply v6 at some point to a set of distributed repos. Essentially it has the typical duplicated system topology: 2 remotes, several clients, each of which is linked to both remotes. Thus it will survive multiple failure scenarios.
+
+I would be looking at the strategy of the order of repo upgrade. Say, a client first, and run a set of tests. And so on. Remotes last perhaps.
+
+In any case I build a test bed as above and have been experimenting with a single client at a v6 repo.
+
+So far it has been smooth, but right now I seem to be stuck where thinning is not having any effect: I still have a particular big file (1.5GB) in both the annex and the working dir. Lock removes the big file from the working dir, and leaves a link, and unlock restores the big file in the working dir. Yet, the annex still contains the big file no matter what. I was under the expectation that one of the big file copies would go away.
+"""]]


fixed meta tag
diff --git a/doc/todo/get_--batch.mdwn b/doc/todo/get_--batch.mdwn
index c0e12bd..619cfb5 100644
--- a/doc/todo/get_--batch.mdwn
+++ b/doc/todo/get_--batch.mdwn
@@ -1,3 +1,3 @@

-[[!meta name=yoh]]
+[[!meta author =yoh]]


diff --git a/doc/install/openSUSE.mdwn b/doc/install/openSUSE.mdwn
index 760def9..b24f7df 100644
--- a/doc/install/openSUSE.mdwn
+++ b/doc/install/openSUSE.mdwn
@@ -1,3 +1,3 @@
+From openSUSE:Leap:42.2 will be git-annex  part of distribution

-At the time of writing, there are [unofficial packages of git-annex](http://software.opensuse.org/package/git-annex) available for openSUSE.  It should also be possible to build it via cabal or from source as described on the [[install]] page.
+At the time of writing, there are [packages of git-annex](https://build.opensuse.org/project/show/devel:languages:haskell) available for openSUSE.  It should also be possible to build it via cabal or from source as described on the [[install]] page.


diff --git a/doc/openSUSE__58__Tumbleweed.mdwn b/doc/openSUSE__58__Tumbleweed.mdwn
new file mode 100644
index 0000000..844dfa7
--- /dev/null
+++ b/doc/openSUSE__58__Tumbleweed.mdwn
@@ -0,0 +1 @@
+*git annex* is included in openSUSE:Tumbleweed distribution


diff --git a/doc/install.mdwn b/doc/install.mdwn
index f462d04..dbb975e 100644
--- a/doc/install.mdwn
+++ b/doc/install.mdwn
@@ -15,7 +15,8 @@ detailed instructions             | quick install
&nbsp;&nbsp;[[Gentoo]]            | emerge git-annex
&nbsp;&nbsp;[[Void]]            | xbps-install git-annex
&nbsp;&nbsp;[[ScientificLinux5]]  |
-&nbsp;&nbsp;[[openSUSE]]          |
+&nbsp;&nbsp;[[openSUSE:Tumbleweed]]          | zypper in git-annex
+&nbsp;&nbsp;[[openSUSE]]          |
&nbsp;&nbsp;[[Docker]]            |
"""]]


diff --git a/doc/forum/Repository_backup/comment_5_287510b4e8b9043fe361b7c5dc1f4e9f._comment b/doc/forum/Repository_backup/comment_5_287510b4e8b9043fe361b7c5dc1f4e9f._comment
new file mode 100644
index 0000000..2fac3bc
--- /dev/null
+++ b/doc/forum/Repository_backup/comment_5_287510b4e8b9043fe361b7c5dc1f4e9f._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ subject="comment 5"
+ date="2016-06-27T19:36:58Z"
+ content="""
+I always backup my git-annex repos using 'git bundle create ... --all' which is similar to a git clone, but much more convenient for backups, because it is just a single file :).
+"""]]


rename forum/Git_annex__44___buf__44___windows_and_remote_linux_issue..mdwn to forum/Git_annex__44___bup__44___windows_and_remote_linux_issue..mdwn
diff --git a/doc/forum/Git_annex__44___buf__44___windows_and_remote_linux_issue..mdwn b/doc/forum/Git_annex__44___buf__44___windows_and_remote_linux_issue..mdwn
deleted file mode 100644
index ee024ec..0000000
--- a/doc/forum/Git_annex__44___buf__44___windows_and_remote_linux_issue..mdwn
+++ /dev/null
@@ -1 +0,0 @@
-I would like to make some thing. On windows one repo (R1) with files for work. On linux server the other (R2) is for syncing and the third (R3) on linux is for backup with bup. Is it any way to run remote backup command on R2 from R1 but not always when i sync them - only when i send special command? I am looking simple native solution avoiding cygwin or third scripts. Thanks.
diff --git a/doc/forum/Git_annex__44___bup__44___windows_and_remote_linux_issue..mdwn b/doc/forum/Git_annex__44___bup__44___windows_and_remote_linux_issue..mdwn
new file mode 100644
index 0000000..ee024ec
--- /dev/null
+++ b/doc/forum/Git_annex__44___bup__44___windows_and_remote_linux_issue..mdwn
@@ -0,0 +1 @@
+I would like to make some thing. On windows one repo (R1) with files for work. On linux server the other (R2) is for syncing and the third (R3) on linux is for backup with bup. Is it any way to run remote backup command on R2 from R1 but not always when i sync them - only when i send special command? I am looking simple native solution avoiding cygwin or third scripts. Thanks.


diff --git a/doc/forum/Git_annex__44___buf__44___windows_and_remote_linux_issue..mdwn b/doc/forum/Git_annex__44___buf__44___windows_and_remote_linux_issue..mdwn
new file mode 100644
index 0000000..ee024ec
--- /dev/null
+++ b/doc/forum/Git_annex__44___buf__44___windows_and_remote_linux_issue..mdwn
@@ -0,0 +1 @@
+I would like to make some thing. On windows one repo (R1) with files for work. On linux server the other (R2) is for syncing and the third (R3) on linux is for backup with bup. Is it any way to run remote backup command on R2 from R1 but not always when i sync them - only when i send special command? I am looking simple native solution avoiding cygwin or third scripts. Thanks.


diff --git a/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_4_bf0e16f9181438c2050521c5ff5bb7d0._comment b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_4_bf0e16f9181438c2050521c5ff5bb7d0._comment
new file mode 100644
index 0000000..e34ea04
--- /dev/null
+++ b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_4_bf0e16f9181438c2050521c5ff5bb7d0._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ subject="comment 4"
+ date="2016-06-24T07:16:20Z"
+ content="""
+Good questions.  The setup is essentially a star topology where the central node is the full-backup, bare git repository on the external drive on the server.  It's not encrypted, if that matters.  Since it's a bare repo, nothing actually changes in that node except when changes are pushed from elsewhere.
+
+Every other node is configured as a client and is connected to that repo on the server via SSH.  When a change is made to one of the client repos, syncing is happening automatically and nearly instantaneously--not just to the server repo, but the changes are very quickly propagated to the other SSH-connected clients.  (I think this is supposed to happen for SSH repos now without using XMPP.  In the release notes for version 5.20140421: \"This release begins to deprecate XMPP support. In particular, if you use the assistant with a ssh remote that has this version of git-annex installed, you don't need XMPP any longer to get immediate syncing of changes.\")
+
+But then there's this one other node in the star topology---the client repo (non-bare) that is actually on the same system as the bare, backup repo.  When changes are made to the client, they are synced immediately to the backup repo, and those changes are quickly synced to the other client repos on other computers connected by SSH.  The problem is changes in the other direction; when a client repo on a remote system is changed, the updates are synced to the backup repo on the server, but no farther.  The webapp running on the server never (or only very slowly) syncs those changes to that client repo that is on the same system.  The webapp on the server is running with that local client repo as the \"main\" repo (I mean, the upper right hand corner says \"Repository: ~/doc\", which is the client repo).  All of the status messages are green and say that the repos are synced.  You can, of course, force a sync from there, and and then the changes are noticed and propagated to the client.
+
+I did some testing and it seems like repositories with non-bare remotes on the same filesystem are synced immediately using inotify or something.  Maybe this just doesn't happen for bare repos?  That's the variable that seems to make syncing not happen automatically, in my limited testing.  I can test more to isolate exactly when it happens, but I was hoping Joey would know off the top of his head if this should be working or not.
+
+"""]]


diff --git a/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_3_ecb57e72c0bf583d1c3461313a83c4f2._comment b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_3_ecb57e72c0bf583d1c3461313a83c4f2._comment
new file mode 100644
index 0000000..15662ca
--- /dev/null
+++ b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_3_ecb57e72c0bf583d1c3461313a83c4f2._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ subject="Thanks"
+ date="2016-06-24T06:35:10Z"
+ content="""
+And the other repositories (which are syncing correctly), they are also marked as client? Clients should sync with each other whenever they can connect to each other, but perhaps your client repositories cannot connect to each other?
+
+[Here is a comment from Joey](https://git-annex.branchable.com/forum/Two_computer_setup__58_____34__transfer__34___or___34__full_backup__34___repository_groups__63__/) about a similar setup. He says you might need to set up an XMPP account for the various clients to message each other when they have new changes? But that was a long time ago. And anyways it seems like the webapp would be doing this automatically (yet in your post you say the computers do not use XMPP?).
+
+I think everyone would benefit from more info still. For example, what does the webapp look like when you are syncing this repo? Are all the clients and the backup on the same page? Is each client only combined to the backup itself and no other repos? What kind of remote repos are the other computers?
+
+Hope you don't mind the questions, I'm also curious about this issue myself. Thanks for bringing it up.
+"""]]


diff --git a/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_2_a7617cc63584dafb16bdc9373fb1e304._comment b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_2_a7617cc63584dafb16bdc9373fb1e304._comment
new file mode 100644
index 0000000..01b0e4c
--- /dev/null
+++ b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_2_a7617cc63584dafb16bdc9373fb1e304._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ subject="comment 2"
+ date="2016-06-24T05:40:23Z"
+ content="""
+The bare repo on the external drive is configured as full backup and the repo at ~/doc is client.
+"""]]


diff --git a/doc/bugs/easy_feature_request__58___enable_lan_sync_on_android/comment_2_471d2dfee80033072cf8732bb532eca5._comment b/doc/bugs/easy_feature_request__58___enable_lan_sync_on_android/comment_2_471d2dfee80033072cf8732bb532eca5._comment
new file mode 100644
index 0000000..94aa4a8
--- /dev/null
+++ b/doc/bugs/easy_feature_request__58___enable_lan_sync_on_android/comment_2_471d2dfee80033072cf8732bb532eca5._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ subject="Interested"
+ date="2016-06-24T03:22:38Z"
+ content="""
+I would like to help (though I have not enough time to fully make the feature for sure).
+
+With regards to:
+
+1) It seems that there is a library which implements an SSH server on Android. I see it [at this repo](https://github.com/barryk/android_external_dropbear) as well as at [this repo](https://github.com/berserker/android_dropbear), both of which are owned by the makers of popular sshd apps on the Google Play Store
+
+2) Do you mean that that library needs to be ported to Android/Java? If so, I could take a crack at it!
+"""]]


Added a comment: A caution on this
diff --git a/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_4_536fbc38a7f2e7ab42188b5a8700e2cf._comment b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_4_536fbc38a7f2e7ab42188b5a8700e2cf._comment
new file mode 100644
index 0000000..524f507
--- /dev/null
+++ b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_4_536fbc38a7f2e7ab42188b5a8700e2cf._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ subject="A caution on this"
+ date="2016-06-23T21:24:49Z"
+ content="""
+Although I believe Joey's analysis is correct, I want to interject a caution: while investigating this issue on my own system, I discovered that a gcrypt remote has an unencrypted pack file and unencrypted index, which somehow contained actual data of mine.  I would consider it a git-annex bug that data (file contents) wound up in a pack file (this is a git-annex assistant repo) but a gcrypt bug that it made it there unencrypted.
+"""]]


diff --git a/doc/forum/v5_to_v6_upgrade_strategy/comment_1_1505166c806ab3d1a3148a245faa713a._comment b/doc/forum/v5_to_v6_upgrade_strategy/comment_1_1505166c806ab3d1a3148a245faa713a._comment
new file mode 100644
index 0000000..68c9c52
--- /dev/null
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ date="2016-06-23T14:44:18Z"
+ content="""
+since you haven't explicitly mentionned it, i'll just mention a pointer to the [[upgrades]] page which has specific instructions regarding upgrading repositories to the v6 layout.
+
+as far as I know, you can just upgrade the git-annex *program* itself harmlessly. it will *not* upgrade your repositories without you deliberately running [[git-annex-upgrade]].
+
+then [[git-annex-upgrade]] will operate changes, but only on direct mode repositories, as far as i can tell. those repositories are switched to [[design/adjusted branches]], which have their own set of issues right name, mostly limitations with the webapp but have also seen a few bugfixes on crippled filesystems recently.
+
+i am personnally doing the \"wait and see\" game especially considering my specific use case for v6 ([[todo/hide_missing_files/]]) has not been completely implemented yet. but I am of course using 6.x binaries without problems.
+
+i am not the git-annex author, so take my comments with a grain of salt of course. :) --[[anarcat]]
+"""]]


Added a comment: related issues and response to problems raised
diff --git a/doc/todo/dumb__44___unsafe__44___human-readable_backend/comment_5_941b2364433ae42bc67a75f93564331d._comment b/doc/todo/dumb__44___unsafe__44___human-readable_backend/comment_5_941b2364433ae42bc67a75f93564331d._comment
new file mode 100644
index 0000000..ac2b16a
--- /dev/null
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ subject="related issues and response to problems raised"
+ date="2016-06-23T14:44:06Z"
+ content="""
+there are two related issues that were closed as wontfix here, which i missed in my original search as well:
+
+* [[todo/clear file names in special remotes]] ([archive](http://web.archive.org/web/20160413162353/http://git-annex.branchable.com/todo/clear_file_names_in_special_remotes/))
+* [[todo/New special remote suggeston - clean directory]] ([archive](http://web.archive.org/web/20160413171909/http://git-annex.branchable.com/todo/New_special_remote_suggeston_-_clean_directory/))
+
+those issues were actually removed, but are still on the internet archive for future reference.
+
+to summarize those issues, the rationale there is that those remotes are potentially destructuve (lack of locking and checks) and have workarounds (rsync -a was suggested). it is also clearly stated that this is contrary to the git-annex design and is a \"pony\" feature.
+
+i would counter that this is an often requested feature that seems to be a major usability issue for a lot of people. there are other unsafe remotes out there, like the bittorrent special remote, which is explicitely documented as such, and where we recommend users untrust the remote when it is setup. yet, those remotes have their uses and the rich diversitry of such remotes makes git-annex one of the most interesting projects out there.
+
+furthermore, rsync -a is not the same as git-annex's excellent tracking features. in my use case ([[forum/syncing_music_to_my_android_device]]), there is no git annex repository that has the same set of files which I want present on the android device, so I cannot use rsync without incurring a large storage space cost.
+
+as for this being contrary to git-annex design, you obviously know more about this than me, but from the outside it doesn't seem *that* counter-intuitive. it seems that we have go through a lot of hoops to try to make stuff like [[todo/Facilitate_public_pretty_S3_URLs]] work at all. having a different backend for specially crafted remotes would be a huge gain in usability and impact on data security could be limited with the usual trust/untrust mechanisms.
+
+i would be curious to hear more about how such a backend would be contrary to git-annex's design. i am assuming here that my main repo could still have a SHA256E backend and *some* remotes could have *different* backends. Obviously, maybe that's a flawed assumption and I obviously see how such a dumb backend could break way more assumptions git-annex makes about its data, if it *has* to be used on all the remotes. Yet, there *are* backends right now like URL and WORM that could be considered \"unsafe\", yet do not provide as great usability gains as this dumb backend could provide.
+
+i understand you have been requested this feature often and would understand if this other request would just be closed again. but considering how often it comes up, from different users, i think it should at least be considered as something that should be more explicitely documented (in [[not]], [[backends]] and [[special_remotes]] maybe?) to keep further requests from coming in. keeping an issue like this opened would also help in avoiding duplicate requests.
+
+thank you for your time and efforts on git-annex, i cannot state enough how helpful it is to me. the fact that I could write a special remote to accomplish the above filthy todo is a testament to how flexible and powerful git-annex is. :)
+"""]]


diff --git a/doc/design/adjusted_branches.mdwn b/doc/design/adjusted_branches.mdwn
index 7946e64..608c04c 100644
@@ -12,7 +12,7 @@ version of the original branch, eg adjusted/master.
There would be a adjustment function. For #1 above it would simply convert all
annex symlinks to annex file pointers. For #2 above it would omit files
whose content is not currently in the annex. Sometimes, both #1 and #2 would
-be wanted.
+be wanted. The function is currently implemented as the [[git-annex-adjust]] command.

[Alternatively, it could stay on the master branch, and only adjust the
work tree and index. See WORKTREE notes below for how this choice would


diff --git a/doc/todo/hide_missing_files.mdwn b/doc/todo/hide_missing_files.mdwn
index b90fb3e..45b925e 100644
--- a/doc/todo/hide_missing_files.mdwn
+++ b/doc/todo/hide_missing_files.mdwn
@@ -19,3 +19,7 @@ Thanks for any advice! --[[anarcat]]
> the adjusted branch code when file contents are get/dropped, and the
> naive way to do it could make things quite slow, if the branch needs to
> be re-adjusted after each get/drop of each file. --[[Joey]]
+
+> > I ended up creating a new special remote for this, documented in
+> > [[forum/syncing_music_to_my_android_device]]. It would be much
+> > cleaner, of course, if adjusted branches could do this. --[[anarcat]]


diff --git a/doc/forum/syncing_music_to_my_android_device/comment_1_81653a23424ac04675651fb7632c57ae._comment b/doc/forum/syncing_music_to_my_android_device/comment_1_81653a23424ac04675651fb7632c57ae._comment
new file mode 100644
--- /dev/null
+++ b/doc/forum/syncing_music_to_my_android_device/comment_1_81653a23424ac04675651fb7632c57ae._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ subject="special remote"
+ date="2016-06-23T13:03:48Z"
+ content="""
+I ended up writing a special remote for this, named [git-annex-remote-dumb](http://src.anarc.at/scripts.git/blob_plain/HEAD:/git-annex-remote-dumb) for lack of a better name. it mixes things up badly, is a abhorrent violation of protocol boundaries and is probably due to be taken to the back of the barn and shot, but it works for my case.
+
+this would be better implemented as [[todo/dumb, unsafe, human-readable_backend]], but I suspect this cannot be directly implemented either without significant changes of the git-annex design, something which is unlikely to happen since [[todo/hide missing files]] is likely to be implemented with [[design/adjusted branches]].
+
+"""]]


diff --git a/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_1_66ffaa679ba0fe86bba114fd9cdc5a53._comment b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_1_66ffaa679ba0fe86bba114fd9cdc5a53._comment
new file mode 100644
index 0000000..698338b
--- /dev/null
+++ b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo/comment_1_66ffaa679ba0fe86bba114fd9cdc5a53._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ date="2016-06-23T12:15:28Z"
+ content="""
+Can you please provide the following info? For curiosity's sake.
+
+When you are viewing the repos in the web app, on the right hand side for each repo is a drop down menu labelled \"actions\". Please click there and go to the option labelled \"Edit\". Click this option, and on the resulting page there should be a field labelled \"Repository group\". Can you tell us the \"Repository group\" for the repos you mentioned? What is the \"Repository group\" for the backup on the secondary drive, the repo in your home folder and the remote clients? My hunch is that these group types are not set up to reflect the configuration you want.
+"""]]


diff --git a/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_3_6dd54179aad379bb22638db518bd377f._comment b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_3_6dd54179aad379bb22638db518bd377f._comment
new file mode 100644
index 0000000..73eea64
--- /dev/null
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ subject="comment 3"
+ date="2016-06-23T12:07:10Z"
+ content="""
+Is there a way to link the objects with the gold linker? I am running into the same issue. Currently I am doing a workaround with mounting the drive via NFS, but that has quite poor performance (all git-related things).
+"""]]


Added a comment: dumb external remote implemented
diff --git a/doc/todo/dumb__44___unsafe__44___human-readable_backend/comment_4_bbee2fa4b4fed1c5117cc4ddb0395278._comment b/doc/todo/dumb__44___unsafe__44___human-readable_backend/comment_4_bbee2fa4b4fed1c5117cc4ddb0395278._comment
new file mode 100644
index 0000000..845eb7f
--- /dev/null
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ subject="dumb external remote implemented"
+ date="2016-06-23T00:34:59Z"
+ content="""
+so I went ahead and scratched my itch of implementing this dumb special remote. i think it would be better implemented as a backend, but i am not sure how to implement that without going deep into haskell, and it seems like a mad idea anyways, because it would need to modify the hashing structure as well (to keep the directory structure).
+
+the remote is a simple shell script inspired by the example script, but that does some nasty (and slow) git log -S stuff to find a matching file path to a given key. ideally, this would be streamlined into a git-annex command that could then be optimized, because as it is now, each call is a separate history search.
+
+
+i haven't added it to the [[special_remotes]] list yet, because maybe it warrants a separate [[tips]] page. i also do not want to introduce what can be seen as a \"bad idea\" from git-annex's integrity principles to the list of special remotes before agreement here.
+"""]]


devblog
diff --git a/doc/devblog/day_403__update_and_away.mdwn b/doc/devblog/day_403__update_and_away.mdwn
new file mode 100644
index 0000000..02c1834
--- /dev/null
+++ b/doc/devblog/day_403__update_and_away.mdwn
@@ -0,0 +1,4 @@
+Continued working on the enhanced smudge/clean interface in git today.
+Sent in a third version of the patch set, which is now quite complete.
+
+I'll be away for the next week and a half, on vacation.


removed
diff --git a/doc/tips/largefiles/comment_5_cf7354ecc5a3e69a0cc0d47d77e52051._comment b/doc/tips/largefiles/comment_5_cf7354ecc5a3e69a0cc0d47d77e52051._comment
deleted file mode 100644
index e0f8143..0000000
--- a/doc/tips/largefiles/comment_5_cf7354ecc5a3e69a0cc0d47d77e52051._comment
+++ /dev/null
@@ -1,14 +0,0 @@
-[[!comment format=mdwn
- nickname="christophfischer"
- subject="how to recursively exclude files from git-annex (.gitattributes)"
- date="2016-06-21T11:42:45Z"
- content="""
-Hello
-
-it took me some time to figure out how to exclude directories matching a specific structure within the .gitattributes file:
-
-    **/some_dir/**/below_this_dir_is_everything_in_plane_text/**/* annex.largefiles=nothing
-
-Maybe it helps someone else. (In case this way is the intended way)
-"""]]


Added a comment: how to recursively exclude files from git-annex (.gitattributes)
diff --git a/doc/tips/largefiles/comment_5_cf7354ecc5a3e69a0cc0d47d77e52051._comment b/doc/tips/largefiles/comment_5_cf7354ecc5a3e69a0cc0d47d77e52051._comment
new file mode 100644
index 0000000..e0f8143
--- /dev/null
+++ b/doc/tips/largefiles/comment_5_cf7354ecc5a3e69a0cc0d47d77e52051._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ nickname="christophfischer"
+ subject="how to recursively exclude files from git-annex (.gitattributes)"
+ date="2016-06-21T11:42:45Z"
+ content="""
+Hello
+
+it took me some time to figure out how to exclude directories matching a specific structure within the .gitattributes file:
+
+    **/some_dir/**/below_this_dir_is_everything_in_plane_text/**/* annex.largefiles=nothing
+
+Maybe it helps someone else. (In case this way is the intended way)
+"""]]


Added a comment: how to recursively exclude files from git-annex (.gitattributes)
diff --git a/doc/tips/largefiles/comment_4_b6380286f60c1cd46a82a2e90c7aa05f._comment b/doc/tips/largefiles/comment_4_b6380286f60c1cd46a82a2e90c7aa05f._comment
new file mode 100644
index 0000000..359c01e
--- /dev/null
+++ b/doc/tips/largefiles/comment_4_b6380286f60c1cd46a82a2e90c7aa05f._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ nickname="christophfischer"
+ subject="how to recursively exclude files from git-annex (.gitattributes)"
+ date="2016-06-21T11:42:37Z"
+ content="""
+Hello
+
+it took me some time to figure out how to exclude directories matching a specific structure within the .gitattributes file:
+
+    **/some_dir/**/below_this_dir_is_everything_in_plane_text/**/* annex.largefiles=nothing
+
+Maybe it helps someone else. (In case this way is the intended way)
+"""]]


diff --git a/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo.mdwn b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo.mdwn
new file mode 100644
index 0000000..459b08b
--- /dev/null
+++ b/doc/bugs/bare_remote_is_not_automatically_synced_to_local_repo.mdwn
@@ -0,0 +1,15 @@
+
+I have a git annex repository at ~/doc which has a remote stored on a second drive at /media/backup/doc.git, which is a bare git annex repo.  The webapp is run from ~/doc. Changes are made to /media/backup/doc.git by syncing from other computers, but then those changes to the remote are not automatically propagated to ~/doc by the webapp.
+
+The use case that lead to this situation is syncing several computers to a large hard drive on a server, utilizing a bare repo as a full backup, and then wishing to have a client repo in the home folder on the server.  So other computers sync their repos to /media/backup/doc.git and then any changes should be propagated to ~/doc on the server.
+
+Changes made to ~/doc *are* synced to the backup repository on the backup hard drive, and those changes are almost immediately synced to the other machines by the webapp running on those respective machines. Those computers are configured to connect to /media/backup/doc.git on the server using SSH only (and git-annex-shell on the server), and do not use XMPP.
+
+### What version of git-annex are you using? On what operating system?
+
+The amd64 pre-built tarball, git-annex version: 6.20160613-g35dbe35, running on Ubuntu 16.04.
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Migrated all of my syncing to git annex about 10 months ago and haven't had any significant trouble :)


diff --git a/doc/news/version_6.20160613/comment_1_89cfca1174ea3e50a536053e73027296._comment b/doc/news/version_6.20160613/comment_1_89cfca1174ea3e50a536053e73027296._comment
new file mode 100644
index 0000000..67a0dc7
--- /dev/null
+++ b/doc/news/version_6.20160613/comment_1_89cfca1174ea3e50a536053e73027296._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ subject="comment 1"
+ date="2016-06-21T08:46:50Z"
+ content="""
+     Improve SHA*E extension extraction code.
+
+This sounds really likely to break things like [recovering files by readding](https://git-annex.branchable.com/tips/recover_data_from_lost+found/), because the key is going to be different, despite having the same backend..
+"""]]


diff --git a/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__/comment_2_ab6420f7bc18e8143201c603c09a0d7e._comment b/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__/comment_2_ab6420f7bc18e8143201c603c09a0d7e._comment
new file mode 100644
index 0000000..e63f26c
--- /dev/null
+++ b/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__/comment_2_ab6420f7bc18e8143201c603c09a0d7e._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ nickname="m8r-568niv"
+ subject="comment 2"
+ date="2016-06-19T22:56:17Z"
+ content="""
+Looks like I missed an obvious link/reference:
+
+[https://git-annex.branchable.com/tips/Bup_repositories_in_git-annex/](https://git-annex.branchable.com/tips/Bup_repositories_in_git-annex/)
+"""]]


diff --git a/doc/bugs/v6__58___git_status__47__add_after_unlock_is_linear_to_the_file_size/comment_1_ea798d60cbdfa7534bad0ba47119379d._comment b/doc/bugs/v6__58___git_status__47__add_after_unlock_is_linear_to_the_file_size/comment_1_ea798d60cbdfa7534bad0ba47119379d._comment
new file mode 100644
index 0000000..2ebcf37
--- /dev/null
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ nickname="mail"
+ subject="comment 1"
+ date="2016-06-19T16:27:15Z"
+ content="""
+I noticed after filing the bug that the first git commit takes as much (or even more!) time as the git add -A before it.
+So you pay overhead twice somehow.
+"""]]


diff --git a/doc/bugs/bind__58___resource_busy___40__Address_already_in_use__41__.mdwn b/doc/bugs/bind__58___resource_busy___40__Address_already_in_use__41__.mdwn
new file mode 100644
index 0000000..ea2bd88
--- /dev/null
@@ -0,0 +1,42 @@
+I'm a user of git-annex for many years now (since the first stable releases) and I've always used it before on debian-like systems. I have now a gentoo OS where I have compiled git-annex.
+
+    git-annex version: 6.20160419
+    build flags: Assistant Webapp Pairing S3(multipartupload) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime EKG Feeds Quvi
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+
+I don't know if you exactly can solve my problem, as it may be related to haskell stuff, and not directly coming from git-annex code, so I barely hope some hints about the possible source of the problem. When I perform any git annex action (here a git annex sync), I get the following message in the cli:
+
+    commit
+    Error on startup:
+    ok
+    pull <ANNEX>
+    <etc...>
+
+Of course, the webapp as also troubles with this. My understanding of the phenomena is that only one git annex action can be performed at the time, so the rest of them crash or fail. Is my understanding correct, do you think? I can consequently, perform git annex actions, but only one at the time, and the whole thing feels unreliable. I suspect that the problem comes from the haskell compiler or platform, but I have no knowledge of haskell and I am consequently not really able to aim at what could be the problem. If you have an idea or some hints, I would be glad to hear from you, as that bug really comes in my way. I use git-annex for every bits of my computer life and that's really a handicap for me :-(.
+
+### Extract of .git/annex/daemon.log
+[[!format sh """
+
+[2016-06-19 17:41:57.98439] main: starting assistant version 6.20160419
+[2016-06-19 17:42:04.663738] TransferScanner: Syncing with hunbaut
+
+  No known network monitor available through dbus; falling back to polling
+
+  No known volume monitor available through dbus; falling back to mtab polling
+(scanning...) [2016-06-19 17:42:04.833047] Watcher: Performing startup scan
+Error on startup:
+RemoteControl crashed: user error (nice ["ionice","-c3","/usr/bin/git-annex","remotedaemon"] exited 1)
+[2016-06-19 17:42:04.945745] RemoteControl: warning RemoteControl crashed: user error (nice ["ionice","-c3","/usr/bin/git-annex","remotedaemon"] exited 1)
+
+
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yeah, for years! Thank you so much for it :-)
+


diff --git a/doc/forum/git_annex_wants_to_repair_every_time_it__39__s_running/comment_1_c182b03bae6e691eb68465fedf5c77d5._comment b/doc/forum/git_annex_wants_to_repair_every_time_it__39__s_running/comment_1_c182b03bae6e691eb68465fedf5c77d5._comment
new file mode 100644
index 0000000..585beea
--- /dev/null
+++ b/doc/forum/git_annex_wants_to_repair_every_time_it__39__s_running/comment_1_c182b03bae6e691eb68465fedf5c77d5._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ subject="comment 1"
+ date="2016-06-18T21:48:42Z"
+ content="""
+So there are no ideas on how to proceed from anyone? Was the way I have written this message in any way too complicated or was it for some reason formulated in a manner, that made it look too demanding of others peoples time? I was not intentionally doing so, but I would appreciate any kind of commend on how to better phrase the question or on how to improve it in a way that this issue can be solved. Thanks in advance!
+
+P.S.: Since people probably don't stumble onto this comment, I'll give it another week or so before I would open up a new post containing the same issue.
+"""]]


diff --git a/doc/bugs/v6__58___git_status__47__add_after_unlock_is_linear_to_the_file_size.mdwn b/doc/bugs/v6__58___git_status__47__add_after_unlock_is_linear_to_the_file_size.mdwn
new file mode 100644
index 0000000..487216f
--- /dev/null
@@ -0,0 +1,23 @@
+
+After unlocking a repository, the next git status will take time linear to the file size. It seems to be highly inefficient (the I/O on my SSD is not anywhere near the limit).
+For a 500Mb flac album it takes ~5–10s, for my 100GB local music archive it took well over 30min.
+
+### What steps will reproduce the problem?
+
+
+git annex unlock
+git status / git add -A
+
+
+### What version of git-annex are you using? On what operating system?
+
+6.20160527
+NixOS (master branch)
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Not yet. :)
+But will use it to keep my music library and sync a smaller (ogg) version to my various devices (with beets handling metadata and conversion)
+


diff --git a/doc/todo/Log_function_to_enumerate_all_recent_git-annex_changes/comment_1_1b1439d08fd064314e28f94caaa850be._comment b/doc/todo/Log_function_to_enumerate_all_recent_git-annex_changes/comment_1_1b1439d08fd064314e28f94caaa850be._comment
new file mode 100644
index 0000000..c03673f
--- /dev/null
+++ b/doc/todo/Log_function_to_enumerate_all_recent_git-annex_changes/comment_1_1b1439d08fd064314e28f94caaa850be._comment
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ subject="Use-case expansion"
+ date="2016-06-17T23:55:52Z"
+ content="""
+My use-case is to store the absolute disk offsets for files, to aid recovery in case of accidental loss.
+
+I'd like to run a script for every key added to the repo in which the script resides.  The script calls filefrag to determine the physical location of the key on the disk and adds it as metadata.
+
+If interested, my pre-commit-annex snippet is:
+
+                # requires recent e2fsprogs
+                if [ -x /usr/sbin/filefrag ]
+                then
+                        field=\"$(git config annex.uuid)\"-extents + LC_ALL=C /usr/sbin/filefrag -ev \"$f\" | sed -n \
+                                -e 's/.*([0-9]* blocks* of $$[0-9]*$$ bytes).*/!bs \1/p'\
+                                -e 's/^ *[0-9]*: *$$[0-9]*$$\.\. *[0-9]*: *$$[0-9]*$$\.\. *[0-9]*: *$$[0-9]*$$: *.*/\1 \2 \3/p' |
+                        do
+                                # !bs 4096     means values are in multiples of 4096 bytes (blocksize)
+                                # 0 5287839 5  means 5 blocks starting at block 0 of the file start at block 5287839 of the disk
+                                # extents which are not listed are from sparse files and contain all zeros
+                                equal='+=' addmeta \"$f\" \"$field\" \"$value\" + done + fi + +"""]]  diff --git a/doc/todo/Log_function_to_enumerate_all_recent_git-annex_changes.mdwn b/doc/todo/Log_function_to_enumerate_all_recent_git-annex_changes.mdwn new file mode 100644 index 0000000..fc10c5e --- /dev/null +++ b/doc/todo/Log_function_to_enumerate_all_recent_git-annex_changes.mdwn @@ -0,0 +1,5 @@ +It would be great to have something to call in post-update-annex which would give a list of all new and lost files given in the update. + +This could be git annex log --all --max-count=1 or somesuch. + +This capability could alternatively be provided with a new post-transfer hook, called for every file.  devblog diff --git a/doc/devblog/day_402__enhanced_smudge_clean_interface.mdwn b/doc/devblog/day_402__enhanced_smudge_clean_interface.mdwn new file mode 100644 index 0000000..a773975 --- /dev/null +++ b/doc/devblog/day_402__enhanced_smudge_clean_interface.mdwn @@ -0,0 +1,14 @@ +Continued working on the enhancaed smudge/clean interface in git, +incorporating feedback from the git developers. + +In a spare half an hour, I made an improved-smudge-filters branch +that teaches git-annex smudge to use the new interface. + +Doing a quick benchmark, git checkout of a deleted 1 gb file took: + +* 19 seconds before +* 11 seconds with the new interface +* 0.1 seconds with the new interface and annex.thin set + (while also saving 1 gb of disk space!) + +So, this new interface is very much worthwhile.  diff --git a/doc/bugs/silently_failing_when_attempting_to_add_ignored_files.mdwn b/doc/bugs/silently_failing_when_attempting_to_add_ignored_files.mdwn index 551abcf..d9b5f67 100644 --- a/doc/bugs/silently_failing_when_attempting_to_add_ignored_files.mdwn +++ b/doc/bugs/silently_failing_when_attempting_to_add_ignored_files.mdwn @@ -1,9 +1,9 @@ ### Please describe the problem. -When attempting to add a file listed in .gitignore, git annex exits silently. I would expect to see an error message a la plain git: "The following paths are ignored by one of your .gitignore files: <that file>" +When attempting to add a file listed in .gitignore, git annex exits silently. I would expect to see an error message a la plain git: "The following paths are ignored by one of your .gitignore files: _that file_" ### What steps will reproduce the problem? 1. Add a file to .gitignore -2. git annex add <that file> +2. git annex add _that file_ ### What version of git-annex are you using? On what operating system? git-annex version: 6.20160613  diff --git a/doc/bugs/silently_failing_when_attempting_to_add_ignored_files.mdwn b/doc/bugs/silently_failing_when_attempting_to_add_ignored_files.mdwn new file mode 100644 index 0000000..551abcf --- /dev/null +++ b/doc/bugs/silently_failing_when_attempting_to_add_ignored_files.mdwn @@ -0,0 +1,31 @@ +### Please describe the problem. +When attempting to add a file listed in .gitignore, git annex exits silently. I would expect to see an error message a la plain git: "The following paths are ignored by one of your .gitignore files: <that file>" + +### What steps will reproduce the problem? +1. Add a file to .gitignore +2. git annex add <that file> + +### What version of git-annex are you using? On what operating system? +git-annex version: 6.20160613 +build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi +key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL +remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external +local repository version: 5 +supported repository versions: 5 6 +upgrade supported from repository versions: 0 1 2 3 4 5 +operating system: darwin x86_64 + + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + +  Added a comment diff --git a/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__/comment_1_0bb5e21b4cbaaaa61fbdb2e0066f0ebf._comment b/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__/comment_1_0bb5e21b4cbaaaa61fbdb2e0066f0ebf._comment new file mode 100644 index 0000000..ae1721e --- /dev/null +++ b/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__/comment_1_0bb5e21b4cbaaaa61fbdb2e0066f0ebf._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="m8r-568niv@9cefd6353b1102081f43c2f5bc53afdddc153274" + nickname="m8r-568niv" + subject="comment 1" + date="2016-06-17T07:07:10Z" + content=""" +I've posted to the bup list. + +Some positive comments: + +[https://groups.google.com/d/topic/bup-list/ReIZxGpTQCo/discussion](https://groups.google.com/d/topic/bup-list/ReIZxGpTQCo/discussion) +"""]]  diff --git a/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__.mdwn b/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__.mdwn new file mode 100644 index 0000000..ba005e3 --- /dev/null +++ b/doc/forum/git-annex_and_bup_-_the_other_way_around___63____63____63__.mdwn @@ -0,0 +1,28 @@ +I've been wondering about using git-annex and bup together - but _not_ with bup as the backend, but rather backing up bup repos using git-annex. + +Let me try to explain... + +* bup is a a great deduplicating backup tool, but it does not have encryption +* git-annex is a awesome in so many ways. Including 1) multiple copies, 2) encryption + +(I know the following reads like the motivation for git-annex, but let me add the word **backup**) + +* Recovering large backups over the internet can be costly and time consuming +* Local copies are fast, but are risky + +So I was wondering about having my bup repos in git-annex, with multiple copies, including, say, an encrypted S3 bucket and some local copies. + +Then, if I had a problem and needed to restore I could use my local copies for as much as I could and then only pull part of the backup from the complete remote backup. + +If that all works, I then have: + +1. A more complicated process than a simple backup tool :-( +2. Multiple complete backups available :-D +3. Encrypted, offsite backups :-D +4. Small transfers (bup uses the awesomness of git to dedup the hell out of your data) :-D + +--- + +I've not done this yet, but was thinking it through. + +Can anyone share some opinions, thoughts, concerns or high-5s for the awesomeness of my idea? ;-)  link to patch diff --git a/doc/todo/smudge.mdwn b/doc/todo/smudge.mdwn index 2dffc90..32080fc 100644 --- a/doc/todo/smudge.mdwn +++ b/doc/todo/smudge.mdwn @@ -34,6 +34,9 @@ git-annex should use smudge/clean filters. such an interface: <http://news.gmane.org/find-root.php?message_id=20160512182432.GA27427%40kitenet.net> + And developed a patch set: + <http://thread.gmane.org/gmane.comp.version-control.git/297475> + * Checking out a different branch causes git to smudge all changed files, and write their content. This does not honor annex.thin. A warning message is printed in this case.  devblog diff --git a/doc/devblog/day_400-401__git_development.mdwn b/doc/devblog/day_400-401__git_development.mdwn new file mode 100644 index 0000000..a3b7f5d --- /dev/null +++ b/doc/devblog/day_400-401__git_development.mdwn @@ -0,0 +1,11 @@ +Working on git, not git-annex the past two days, I have implemented the +smudge-to-file/clean-from-file extension to the smudge/clean filter +interface. Patches have been [sent to the git developers](http://thread.gmane.org/gmane.comp.version-control.git/297475), +and hopefully they'll like it and include it. This will make git-annex +v6 work a lot faster and better. + +Amazing how much harder it is to code on git than on git-annex! While I'm +certianly not as familiar with the git code base, this is mostly because C +requires so much more care about innumerable details and so much verbosity +to do anything. I probably could have implemented this interface in +git-annex in 2 hours, not 2 days.  diff --git a/doc/todo/get_--batch.mdwn b/doc/todo/get_--batch.mdwn new file mode 100644 index 0000000..c0e12bd --- /dev/null +++ b/doc/todo/get_--batch.mdwn @@ -0,0 +1,3 @@ +It seems that it would be tremendously useful, see e.g. our [datalad install](https://github.com/datalad/datalad/issues/553) + +[[!meta name=yoh]]  diff --git a/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index.mdwn b/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index.mdwn new file mode 100644 index 0000000..ffad58e --- /dev/null +++ b/doc/bugs/unannex__58___Cannot_proceed_with_uncommitted_changes_staged_in_the_index.mdwn @@ -0,0 +1,37 @@ +### Please describe the problem. + +I understand that git annex unannex is essentially there for undoing an accidential git annex add. Unfortunately it doesn't do that. + +If I have uncommited changes, which is the case after a git annex add, it tells me: + + git-annex: Cannot proceed with uncommitted changes staged in the index. Recommend you: git commit + CallStack (from HasCallStack): + error, called at ./Command/Unannex.hs:48:19 in main:Command.Unannex + +But I would expect it to git reset the file and then replace the symlink by the actual file content. + +### What steps will reproduce the problem? + + > git init + Initialized empty Git repository in /somewhere/.git + > git annex init + init ok + (recording state in git...) + > touch foo + > git annex add foo + add foo ok + (recording state in git...) + > git annex unannex foo + git-annex: Cannot proceed with uncommitted changes staged in the index. Recommend you: git commit + CallStack (from HasCallStack): + error, called at ./Command/Unannex.hs:48:19 in main:Command.Unannex + +### What version of git-annex are you using? On what operating system? + +git-annex version: 6.20160527-gf21a425 + +Installed from the Arch Linux repository. + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + +Sure, I'm using it for photos, music and videos  Added a comment: Version diff --git a/doc/bugs/Empty_folders_don__39__t_get_remove/comment_3_ccc703f8667805ce2886719c57e76ef8._comment b/doc/bugs/Empty_folders_don__39__t_get_remove/comment_3_ccc703f8667805ce2886719c57e76ef8._comment new file mode 100644 index 0000000..93bb20f --- /dev/null +++ b/doc/bugs/Empty_folders_don__39__t_get_remove/comment_3_ccc703f8667805ce2886719c57e76ef8._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="jgoerzen" + subject="Version" + date="2016-06-14T22:35:59Z" + content=""" +Forgot to add: 5.20151208 +"""]]  Added a comment: Experienced this diff --git a/doc/bugs/Empty_folders_don__39__t_get_remove/comment_2_35f59adc30fe743221f29bbfecff639b._comment b/doc/bugs/Empty_folders_don__39__t_get_remove/comment_2_35f59adc30fe743221f29bbfecff639b._comment new file mode 100644 index 0000000..e471d33 --- /dev/null +++ b/doc/bugs/Empty_folders_don__39__t_get_remove/comment_2_35f59adc30fe743221f29bbfecff639b._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="jgoerzen" + subject="Experienced this" + date="2016-06-14T22:08:43Z" + content=""" +Hi, + +I experienced this just now. Have a git-annex assistant repo set up (so using direct mode). + +On one machine, I mv'd entire directories out of the git-annex repo. + +The other machine saw the files deleted, but the directories remained. + +On the machine on which I mv'd directories out, git-annex actually re-created some of them with symlinks to now-nonexistant hashes! I have no idea why it did that. +"""]]  diff --git a/doc/todo/PATCH__58___Fixes_bug_where_webapp_allows_user_to_attempt_deleting_an_unsynced_repo.mdwn b/doc/todo/PATCH__58___Fixes_bug_where_webapp_allows_user_to_attempt_deleting_an_unsynced_repo.mdwn new file mode 100644 index 0000000..61fb707 --- /dev/null +++ b/doc/todo/PATCH__58___Fixes_bug_where_webapp_allows_user_to_attempt_deleting_an_unsynced_repo.mdwn @@ -0,0 +1,9 @@ +I made an attempt to tackle the following bug: + +[webapp should notice when remote deletion cannot complete due to not syncing](https://git-annex.branchable.com/bugs/webapp_should_notice_when_remote_deletion_cannot_complete_due_to_not_syncing/) + +My changes are in the following git repository + +[https://github.com/kathawala/git-annex.git](https://github.com/kathawala/git-annex.git) + +And the specific commit which fixes the bug is in a branch titled "unsync-nodelete". I have tested the change, it greys out and disables the "Delete" option in the "Actions" dropdown menu of the webapp for any repo in the "Repolist" view which has its syncing disabled. There is also some hover text which explains why the option is greyed out. Hope it is satisfactory! Thanks for your great work!!  comment diff --git a/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_4_49bdbdc261c8e5b72f84c52014f892df._comment b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_4_49bdbdc261c8e5b72f84c52014f892df._comment new file mode 100644 index 0000000..991ad48 --- /dev/null +++ b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_4_49bdbdc261c8e5b72f84c52014f892df._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 4""" + date="2016-06-14T01:55:03Z" + content=""" +Forgot to mention that the repeated merge commits happened because git +merge -no-ff was being used, which makes a merge commit even when it can +just fast-forward. That was removed as part of the fix, so the repeated +merges should also be resolved. +"""]]  merged patch closes https://github.com/joeyh/git-annex/pull/54 diff --git a/CHANGELOG b/CHANGELOG index dcdcbbf..8612412 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,11 @@ +git-annex (6.20160614) UNRELEASED; urgency=medium + + * Webapp: Don't allow deleting a remote that has syncing disabled, + as such a deletion will never finish. + Thanks, Farhan Kathawala. + + -- Joey Hess <id@joeyh.name> Mon, 13 Jun 2016 21:52:24 -0400 + git-annex (6.20160613) unstable; urgency=medium * Improve SHA*E extension extraction code. diff --git a/doc/bugs/webapp_should_notice_when_remote_deletion_cannot_complete_due_to_not_syncing.mdwn b/doc/bugs/webapp_should_notice_when_remote_deletion_cannot_complete_due_to_not_syncing.mdwn index cee8da9..9850249 100644 --- a/doc/bugs/webapp_should_notice_when_remote_deletion_cannot_complete_due_to_not_syncing.mdwn +++ b/doc/bugs/webapp_should_notice_when_remote_deletion_cannot_complete_due_to_not_syncing.mdwn @@ -2,3 +2,6 @@ When deleting a remote via the webapp, if syncing is disabled the content will never get removed from the remote. So, the starting to delete a remote should probably enable syncing to it, or warn if syncing is disabled. --[[Joey]] + +> [[done]] via Farhan Kathawala's patch to simply grey out the menu item. +> --[[Joey]]  add news item for git-annex 6.20160613 diff --git a/doc/news/version_6.20160613.mdwn b/doc/news/version_6.20160613.mdwn new file mode 100644 index 0000000..db338b2 --- /dev/null +++ b/doc/news/version_6.20160613.mdwn @@ -0,0 +1,34 @@ +git-annex 6.20160613 released with [[!toggle text="these changes"]] +[[!toggleable text=""" + * Improve SHA*E extension extraction code. + * Windows: Avoid terminating git-annex branch lines with \r\n when + union merging and performing transitions. + * Remove Makefile from cabal tarball; man page building is now handled by + a small haskell program. + * sync --content: Fix bug that caused transfers of files to be made + to a git remote that does not have a UUID. This particularly impacted + clones from gcrypt repositories. + * Pass -S to git commit-tree when commit.gpgsign is set and when + making a non-automatic commit, in order to preserve current behavior + when used with git 2.9, which has stopped doing this itself. + * remotedaemon: Fixed support for notifications of changes to gcrypt + remotes, which was never tested and didn't quite work before. + * list: Do not include dead repositories. + * move --to: Better behavior when system is completely out of disk space; + drop content from disk before writing location log. + * Avoid a crash if getpwuid does not work, when querying the user's full + name. + * Automatically enable v6 mode when initializing in a clone from a repo + that has an adjusted branch checked out. + * v6: Fix initialization of a bare clone of a repo that has an adjusted + branch checked out. + * v6: Fix bad automatic merge conflict resolution between an annexed file + and a directory with the same name when in an adjusted branch. + * v6: Fix bad merge in an adjusted branch that resulted in an empty tree. + * v6: Fix bug in initialization of clone from a repo with an adjusted branch + that had not been synced back to master. + (This bug caused broken tree objects to get built by a later git annex + sync.) + * v6: Make lock and unlock work on files whose content is not present. + * v6: Fix update of associated files db when unlocking a file. + * v6: Make git clean filter preserve the backend that was used for a file."""]] \ No newline at end of file  devblog diff --git a/doc/devblog/day_399__weird_git_merge_bug.mdwn b/doc/devblog/day_399__weird_git_merge_bug.mdwn new file mode 100644 index 0000000..76e5638 --- /dev/null +++ b/doc/devblog/day_399__weird_git_merge_bug.mdwn @@ -0,0 +1,16 @@ +There was one more test suite failure when run on FAT, which I've +investigated today. It turns out that +[a bug report](https://git-annex.branchable.com/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/) +was filed about the same problem, and at root +it seems to be a [bug in git merge](http://thread.gmane.org/gmane.comp.version-control.git/297237). +Luckily, it was not hard to work around the strange merge behavior. + +It's been very worthwhile running the test suite on FAT; it's pointed me at +several problems with adjusted branches over the past weeks. It would be +good to add another test suite pass to test adjusted branches explicitly, +but when I tried adding that, there were a lot of failures where the test +suite is confused by adjusted branch behavior and would need to be taught +about it. + +I've released git-annex 6.20160613. If you're using v6 repositories and +especially adjusted branches, you should upgrade since it has many fixes.  close diff --git a/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn index c9752f0..7437192 100644 --- a/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn +++ b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn @@ -22,3 +22,11 @@ etc. ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) I use git-annex for everything. I've got 10 repositories and around 2.5TB of data in those repos which in turn is synced all over the place. It's excellent. + +> This seems to be a git bug probably; see +> <http://thread.gmane.org/gmane.comp.version-control.git/297237>. +> +> Luckily, easy to work around the problem. +> +> [[fixed|done]] in [[!commit bfd00a0f8ca69cb0669df50f8d98354f11c5253c]]. +> --[[Joey]]  reproduced diff --git a/Test.hs b/Test.hs index e432f54..93f63dd 100644 --- a/Test.hs +++ b/Test.hs @@ -1390,6 +1390,7 @@ test_mixed_lock_conflict_resolution = let v = filter (variantprefix isPrefixOf) l length v == 0 @? (what ++ " not exactly 0 variant files in: " ++ show l) + void$ boolSystem "sh" [Param "-l"]
conflictor elem l @? ("conflictor not present after conflict resolution")
git_annex "get" [conflictor] @? "get failed"
git_annex_expectoutput "find" [conflictor] [conflictor]
diff --git a/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_3_518bc431b16143b9ec0caa78e54c2d6b._comment b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_3_518bc431b16143b9ec0caa78e54c2d6b._comment
new file mode 100644
index 0000000..305e111
--- /dev/null
+++ b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_3_518bc431b16143b9ec0caa78e54c2d6b._comment
@@ -0,0 +1,47 @@
+[[!comment format=mdwn
+ subject="""comment 3"""
+ date="2016-06-13T16:22:27Z"
+ content="""
+repositories?
+
+While investigating a test suite failure that occurs only on FAT,
+I think I have reproduced this bug. I used two v6 repos, both of
+them using adjusted branches, and added a file with the same name and
+content to both independently, then merged the two.
+
+In a merge of two commits that both had the same tree, a merge
+commit was constructed with an empty tree.
+
+Also, much as in the original bug report, there was a pattern of
+repeated merges.
+
+	commit 4bcff45c9670007b8faee5c5514bdd7b9096984a
+	Merge: 4935ace 63fe78f
+	# empty tree
+
+	Merge: 4e42f30 4935ace
+	# populated tree
+
+	4935ace has populated tree
+	4e42f30 has populated tree
+
+Notice that 4935ace is merged in twice, a bit oddly. Indeed, that
+is not possible to construct with manual git commands:
+
+	joey@darkstar:~/tmp/meep/2>git checkout 63fe78f
+	joey@darkstar:~/tmp/meep/2>git merge 4935ace
+	joey@darkstar:~/tmp/meep/2>git checkout 4935ace
+	HEAD is now at 4935ace... git-annex in joey@darkstar:~/tmp/meep/2
+	joey@darkstar:~/tmp/meep/2>git merge 63fe78f
+	Updating 4935ace..63fe78f
+	Fast-forward
+
+In either case, git updates to 63fe78f. So, adusted branches must be
+breaking handling of one or the other of these two cases.
+"""]]


Added a comment: how do I get the autobuild?
diff --git a/doc/forum/git-annex__58___getUserEntryForID__58___does_not_exist___40__no_such_user__41___/comment_2_c9474c4e925948d0cabdfe7f988e1ebe._comment b/doc/forum/git-annex__58___getUserEntryForID__58___does_not_exist___40__no_such_user__41___/comment_2_c9474c4e925948d0cabdfe7f988e1ebe._comment
new file mode 100644
index 0000000..4db7b99
--- /dev/null
+++ b/doc/forum/git-annex__58___getUserEntryForID__58___does_not_exist___40__no_such_user__41___/comment_2_c9474c4e925948d0cabdfe7f988e1ebe._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ nickname="ivan.iossifov"
+ subject="how do I get the autobuild?"
+ date="2016-06-13T13:56:12Z"
+ content="""
+Hi Joy,
+
+Thanks a lot for the quick reply! It's seems that's exactly what will solve my problem.
+Would mind pointing me to it?
+
+Thanks again,
+Ivan
+"""]]


diff --git a/doc/forum/v5_to_v6_upgrade_strategy.mdwn b/doc/forum/v5_to_v6_upgrade_strategy.mdwn
new file mode 100644
index 0000000..9a34b01
--- /dev/null
@@ -0,0 +1,21 @@
+I have recently jumped into git-annex, and have several TB stored in several remote annexes.
+I am able to sync to clients, and have some understanding of the operations needed to get files back etc.
+
+So far I have used v5 in Ubuntu and Debian 8. I have tried out assistant but will leave this until I better understand how git-annex works. CLI is best for this learning.
+
+I am however concerned re the upgrade path from v5 to v6. I have not been able to find a comprehensive strategy. I have tried git-annex v6 and have upgraded a client repo to v6.
+
+All of my remote repos are v5, and to be honest I am very nervous about touching them.
+
+I do have a test bed where I try out all things before doing so on my 'production' repos. So, I can play, break, redo, in safety.
+
+Any direction as to formulating a game plan for upgrading my whole network of repos, clients, etc would be very much appreciated.
+
+And, if I have missed this, a pointer would be good for sure.
+
+
+Regards,
+
+Stan
+
+While it is obvious to all: this is amazing software. Genius.


Question about git-annex fsck on bare repository
diff --git a/doc/forum/Fsck_by_refs_or_keys__63__.mdwn b/doc/forum/Fsck_by_refs_or_keys__63__.mdwn
new file mode 100644
index 0000000..ff2d02d
--- /dev/null
+++ b/doc/forum/Fsck_by_refs_or_keys__63__.mdwn
@@ -0,0 +1,5 @@
+Hello,
+
+I'm trying to wrote a cronjob to fsck my git-annex bare repository on a server. My problem is that there are a lot of keys with no known copies because they were file that weren't meant to stay. So I want to only check files that are live in HEAD of master. Is there anyway to tell git-annex fsck to only check file that are used in some refs?
+
+Thanks.


comment
diff --git a/doc/bugs/Assistant_only_watches_one_repo_on_startup/comment_1_4606e5c7159e8e9f2f12e010cbd22e6a._comment b/doc/bugs/Assistant_only_watches_one_repo_on_startup/comment_1_4606e5c7159e8e9f2f12e010cbd22e6a._comment
new file mode 100644
index 0000000..355b12a
--- /dev/null
+++ b/doc/bugs/Assistant_only_watches_one_repo_on_startup/comment_1_4606e5c7159e8e9f2f12e010cbd22e6a._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ subject="""comment 1"""
+ date="2016-06-10T19:40:54Z"
+ content="""
+You're left out the command line which you use to start the assistant.
+
+Without that, I see no indication that there is a bug. You have to pass
+--autostart to make the assistant start up in all configured repositories.
+If you are not using that parameter, the assistant will only start up in
+the repository corresponding to the current working directory.
+"""]]


diff --git a/doc/bugs/Assistant_only_watches_one_repo_on_startup.mdwn b/doc/bugs/Assistant_only_watches_one_repo_on_startup.mdwn
new file mode 100644
index 0000000..052ab4d
--- /dev/null
+++ b/doc/bugs/Assistant_only_watches_one_repo_on_startup.mdwn
@@ -0,0 +1,27 @@
+
+I have multiple git-annex assistant repos that I want watched.  On startup, it is only watching one of them, despite all of them being listed in ~/.config/git-annex/autostart.
+
+Launching the webapp, clicking on the "switch repository" button, and selecting the other repository will cause git-annex to immediately launch a startup scan and thereafter it seems to be monitored.
+
+### What steps will reproduce the problem?
+
+killall git-annex, and then restart
+
+### What version of git-annex are you using? On what operating system?
+
+5.20151208-1 from jessie-backports
+
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Really nice tool.  Thanks Joey!


Added a comment: Seeing this everywhere
diff --git a/doc/bugs/high_cpu_usage_in_cat-file_and_webapp/comment_3_89dbc16c9c1b74ccb3f46d3c558e0c8d._comment b/doc/bugs/high_cpu_usage_in_cat-file_and_webapp/comment_3_89dbc16c9c1b74ccb3f46d3c558e0c8d._comment
new file mode 100644
index 0000000..46f8f4c
--- /dev/null
+++ b/doc/bugs/high_cpu_usage_in_cat-file_and_webapp/comment_3_89dbc16c9c1b74ccb3f46d3c558e0c8d._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ subject="Seeing this everywhere"
+ date="2016-06-10T16:59:46Z"
+ content="""
+I am now seeing this pattern on every machine I use.
+"""]]


diff --git a/doc/bugs/git_uninit_on_OS_X_causes_SqlLite3_error.mdwn b/doc/bugs/git_uninit_on_OS_X_causes_SqlLite3_error.mdwn
new file mode 100644
index 0000000..1250061
--- /dev/null
+++ b/doc/bugs/git_uninit_on_OS_X_causes_SqlLite3_error.mdwn
@@ -0,0 +1,76 @@
+I am basically having issues with git annex uninit
+
+### What steps will reproduce the problem?
+    $git init localrepo + Initialized empty Git repository in /Users/w/work/temp/test/localrepo/.git/ +$ cd localrepo
+    $git annex init + init ok + (recording state in git...) +$ git annex uninit
+    fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
+    Use '--' to separate paths from revisions, like this:
+    'git <command> [<revision>...] -- [<file>...]'
+    Deleted branch git-annex (was 98f3795).
+*********************
+Now if I add a local file I get a different error
+
+    $git init localrepo + Initialized empty Git repository in /Users/w/work/temp/test/localrepo/.git/ +$ cd localrepo
+    $git annex init + init ok + (recording state in git...) +$ date > secret_file
+    $git annex add secret_file + add secret_file ok + (recording state in git...) +$ git commit -am "added"
+     1 file changed, 1 insertion(+)
+     create mode 120000 secret_file
+    $git annex sync + commit + On branch master + nothing to commit, working directory clean + ok +$ git annex uninit
+    unannex secret_file ok
+    Deleted branch git-annex (was 6a727da).
+    git-annex: failed to commit changes to sqlite database: Just SQLite3 returned ErrorCan'tOpen while attempting to perform open ".git/annex/keys/db".
+    CallStack (from HasCallStack):
+      error, called at ./Database/Handle.hs:111:26 in main:Database.Handle
+
+
+### What version of git-annex are you using? On what operating system?
+
+    $brew info git-annex + git-annex: stable 6.20160527 (bottled), HEAD + Manage files with git without checking in file contents + https://git-annex.branchable.com/ + /usr/local/Cellar/git-annex/6.20160527 (104 files, 87.0M) * + Poured from bottle on 2016-06-06 at 15:36:47 + From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/git-annex.rb + ==> Dependencies + Build: ghc ✘, cabal-install ✘, pkg-config ✔ + Required: gsasl ✔, libidn ✔, libmagic ✔, gnutls ✘, quvi ✔ + ==> Options + --with-git-union-merge + Build the git-union-merge tool + --HEAD + Install HEAD version + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + +I am super excited about what I can do with git-annex. I hope to setup and maintain encrypted repo(s) of some of my files, and access them by cloning a local copy of the encrypted repo and getting the files I want, using them, and then deleting the local copy.  diff --git a/doc/todo/could_standalone___39__fixed__39___git-annex_binaries_be_prelinked__63__.mdwn b/doc/todo/could_standalone___39__fixed__39___git-annex_binaries_be_prelinked__63__.mdwn new file mode 100644 index 0000000..4481102 --- /dev/null +++ b/doc/todo/could_standalone___39__fixed__39___git-annex_binaries_be_prelinked__63__.mdwn @@ -0,0 +1,5 @@ +Since in datalad we are invoking git and git-annex quite frequently, and on debian systems atm relying on git-annex-standalone pkg, I wondered, if there is a possibility to get all 'shimmed' binaries prelinked against shipped core libs to avoid a current bunch of unsucesfull searches for libraries.... I thought it might provide a notable benefit. + +just an idea + +[[!meta author=yoh]]  diff --git a/doc/todo/git_annex_info_to_include_information_about_repo_version__63__.mdwn b/doc/todo/git_annex_info_to_include_information_about_repo_version__63__.mdwn new file mode 100644 index 0000000..e752262 --- /dev/null +++ b/doc/todo/git_annex_info_to_include_information_about_repo_version__63__.mdwn @@ -0,0 +1,3 @@ +I thought that 'git annex info' should be the best location to report current version (6 or before ATM?) of the repository, which seems to be stored only within .git/config. so I see how it somewhat possibly doesn't fit 'git annex info' which reports primarily from the state of git-annex branch, but since even "available local disk space" is in there, version better be reported there as well imho + +[[!meta author=yoh]]  Added a comment diff --git a/doc/bugs/git_annex_adjust_--unlock_seems_to_cause_migration_of_a_file_to_another_backend/comment_2_e55f1748bb3973ab9acb3313f10e79f5._comment b/doc/bugs/git_annex_adjust_--unlock_seems_to_cause_migration_of_a_file_to_another_backend/comment_2_e55f1748bb3973ab9acb3313f10e79f5._comment new file mode 100644 index 0000000..6638603 --- /dev/null +++ b/doc/bugs/git_annex_adjust_--unlock_seems_to_cause_migration_of_a_file_to_another_backend/comment_2_e55f1748bb3973ab9acb3313f10e79f5._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" + subject="comment 2" + date="2016-06-09T20:49:53Z" + content=""" +Thanks for the details! Indeed, IMHO it would be nice to preserve backend for a file, since otherwise I am afraid many 'heterogeneous' annex'es where different files were added manually with different backends (intentionally) would start migrating -- might lead to unpleasant surprises. +"""]]  filter out NoUUID remotes from syncDataRemotes diff --git a/Assistant/DaemonStatus.hs b/Assistant/DaemonStatus.hs index 4c42ffd..92aad07 100644 --- a/Assistant/DaemonStatus.hs +++ b/Assistant/DaemonStatus.hs @@ -55,6 +55,7 @@ calcSyncRemotes = do let good r = Remote.uuid r elem alive let syncable = filter good rs let syncdata = filter (not . remoteAnnexIgnore . Remote.gitconfig)$
+		filter (\r -> Remote.uuid r /= NoUUID) $filter (not . Remote.isXMPPRemote) syncable return$ \dstatus -> dstatus
new file mode 100644
index 0000000..e24cb21
--- /dev/null
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ subject="""comment 3"""
+ date="2016-06-09T20:27:07Z"
+ content="""
+This was fixed in [[!commit fbf5045d4f17accde9e20fa528e52cb1dce61c47]]
+for git annex sync --content
+
+I don't remember the immediate cause of it being in a code that that the
+webapp would call, but I did add a belt-and-suspenders fix at a lower level
+which I'd hope would prevent the webapp from uploading anything in any
+case.
+
+Sounds like the webapp tries to queue transfers to a NoUUID remote, and
+then presumably gives up before the object gets uploaded.
+
+Looking at the code, calcSyncRemotes does not filter out NoUUID remotes
+when populating syncDataRemotes. So, I've fixed that too now.
+"""]]


devblog
diff --git a/doc/devblog/day_398__fresh_eyes.mdwn b/doc/devblog/day_398__fresh_eyes.mdwn
new file mode 100644
--- /dev/null
+++ b/doc/devblog/day_398__fresh_eyes.mdwn
@@ -0,0 +1,11 @@
+Today I was indeed able to get to the bottom of and fix the bug that
+had stumped me the other day.
+
+Rest of the day was taken up by catching up to some bug requests and
+suggestions for v6 mode. Like making unlock and lock work for files
+that are not locally present. And, improving the behavior of the clean
+filter so it remembers what backend was used for a file before and
+continues using that same backend.
+
+About ready to make a release, but IIRC there's one remaining test suite
+failure on FAT.


followup
diff --git a/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_4_76aa27cdeefb1413baaa1c891ccd4d0d._comment b/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_4_76aa27cdeefb1413baaa1c891ccd4d0d._comment
new file mode 100644
index 0000000..a801357
--- /dev/null
+++ b/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_4_76aa27cdeefb1413baaa1c891ccd4d0d._comment
@@ -0,0 +1,27 @@
+[[!comment format=mdwn
+ subject="""comment 4"""
+ date="2016-06-09T19:51:10Z"
+ content="""
+The symbolic-link-like file is in fact, a symlink, which is what git-annex
+uses to represent an annexed file in git. If your filesystem does
+not support symlinks, git writes the link location to a regular file
+
+git annex drop removes the content of a file from the local repository, but
+its symlink remains checked into git. So, the content of the file is
+
+file.
+
+Based on your earlier comment, it does seem that the symlink standin file
+that git uses is being treated as new content for the file, and getting
+annexed. That would be a bug.
+
+Is that happening when you run git annex sync on Linux, or is it on Windows?
+
+What else can you tell or show me to help me reproduce your problem?
+I've tried setting up an NTFS filesystem, putting a git-annex repository on it, and dropping a file;
+git-annex sync did not do the wrong thing when I tried it.
+"""]]


diff --git a/doc/bugs/git_annex_adjust_--unlock_seems_to_cause_migration_of_a_file_to_another_backend/comment_1_7f87080ab44d7ff538e764c4444368e7._comment b/doc/bugs/git_annex_adjust_--unlock_seems_to_cause_migration_of_a_file_to_another_backend/comment_1_7f87080ab44d7ff538e764c4444368e7._comment
new file mode 100644
index 0000000..bba19d2
--- /dev/null
@@ -0,0 +1,41 @@
+[[!comment format=mdwn
+ subject="""comment 1"""
+ date="2016-06-09T18:48:45Z"
+ content="""
+git annex adjust preserves the current backend.
+
+However, when a file in a v6 repository is unlocked, the clean filter will
+use whatever backend git-annex is configured to
+use by git config annex.backends or annex.backend in .gitattributes.
+That defaults to SHA256E. It does not look at the current backend used by
+the file.
+
+So, you can get the same behavior by adding a file using a nonstandard
+backend, and then unlocking it.
+
+	joey@darkstar:~/tmp/rr>date > file
+	joey@darkstar:~/tmp/rr>git annex add file --backend=WORM
+	(recording state in git...)
+	joey@darkstar:~/tmp/rr>git annex unlock file
+	unlock file ok
+	(recording state in git...)
+	joey@darkstar:~/tmp/rr>git diff file
+	diff --git a/file b/file
+	--- a/file
+	+++ b/file
+	@@ -1 +1 @@
+	-/annex/objects/WORM-s30-m1465498400--file
+
+To avoid this, the clean filter would need to look at what's checked
+into git for the file and reuse the same backend as was used before.
+
+.. I guess it's ok to do that. It slows the clean filter down slightly,
+but the clean filter has to hash the whole file content anyway. And,
+it means that when a file gets modified, the same backend that it
+was using gets used for the new key. Which is a behavior change, but
+one that makes sense.
+"""]]


followup
diff --git a/doc/todo/use_CoW_capability___40__e.g._cp_--reflink__41___while_adjust_--unlock.mdwn b/doc/todo/use_CoW_capability___40__e.g._cp_--reflink__41___while_adjust_--unlock.mdwn
index 0eb304f..d9e9f79 100644
@@ -1,3 +1,5 @@
I might be wrong but given the observation that adjust --unlock took  26.04s user 33.95s system 31% cpu 3:12.36 total   to finish operation on an annex (on smaug, btrfs filesystem) with a relatively small (~400) number of relatively large files, I guess no reflink copying was done.

[[!meta author="yoh"]]
+
+> [[dup|done]] --[[Joey]]
new file mode 100644
index 0000000..bdaa128
--- /dev/null
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ subject="""comment 1"""
+ date="2016-06-09T19:44:14Z"
+ content="""
+Git's smudge interface requires git-annex to output the content to stdout;
+git writes it to the file. So, git-annex does not have an opportunity to
+efficiently copy the file.
+
+(All file copies in git-annex use cp --reflink)
+
+My proposed extended smudge interface at
+lets the smudge filter write the file to disk, and then git-annex could
+use more efficient methods. Since this is already discussed in
+[[todo/smudge]] I think I'll close this bug report as a duplicate.
+"""]]


pin crytonite on android to 0.15, which is the version I've been using
diff --git a/doc/forum/Building_Android_app_fails/comment_1_094b90cbc1e2e069900c0037fec145ad._comment b/doc/forum/Building_Android_app_fails/comment_1_094b90cbc1e2e069900c0037fec145ad._comment
new file mode 100644
index 0000000..6d735d4
--- /dev/null
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ subject="""comment 1"""
+ date="2016-06-09T19:27:03Z"
+ content="""
+FWIW, my android autobuilder currently has cryptonite-0.15 installed.
+
+Seems that version is not frozen in the cabal.config for android,
+so tried a newer one in your case. I'm not sure if that will fix your
+problem, but it may be worth a try.
+"""]]
diff --git a/standalone/android/cabal.config b/standalone/android/cabal.config
index a55d85b..dd57db4 100644
--- a/standalone/android/cabal.config
+++ b/standalone/android/cabal.config
@@ -200,3 +200,4 @@ constraints: unix installed,
bytestring installed,
scientific ==0.3.3.1,
clock ==0.4.6.0
+             cryptonite ==cryptonite-0.16


note
diff --git a/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file/comment_2_b1dfc6168bb7127312d3e06cc0939b8d._comment b/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file/comment_2_b1dfc6168bb7127312d3e06cc0939b8d._comment
new file mode 100644
index 0000000..9fbac07
--- /dev/null
+++ b/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file/comment_2_b1dfc6168bb7127312d3e06cc0939b8d._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ subject="""comment 2"""
+ date="2016-06-09T19:19:07Z"
+ content="""
+	LANG=de_DE.UTF-8 errno -s 'Die Struktur muss bereinigt werden'
+	EUCLEAN 117 Die Struktur muss bereinigt werden
+
+This seems to be a failure mode of XFS filesystems, where when it needs to
+be fscked, it throws EUCLEAN.
+"""]]


Make git clean filter preserve the backend that was used for a file.
diff --git a/Annex/Ingest.hs b/Annex/Ingest.hs
index 95bbff4..7b1db8a 100644
--- a/Annex/Ingest.hs
+++ b/Annex/Ingest.hs
@@ -11,6 +11,7 @@ module Annex.Ingest (
lockDown,
ingest,
+	ingest',
finishIngestDirect,
finishIngestUnlocked,
cleanOldKeys,
@@ -140,9 +141,12 @@ ingestAdd ld@(Just (LockedDown cfg source)) = do
- tree or the index.
-}
ingest :: Maybe LockedDown -> Annex (Maybe Key, Maybe InodeCache)
-ingest Nothing = return (Nothing, Nothing)
-ingest (Just (LockedDown cfg source)) = withTSDelta $\delta -> do - backend <- chooseBackend$ keyFilename source
+ingest = ingest' Nothing
+
+ingest' :: Maybe Backend -> Maybe LockedDown -> Annex (Maybe Key, Maybe InodeCache)
+ingest' _ Nothing = return (Nothing, Nothing)
+ingest' preferredbackend (Just (LockedDown cfg source)) = withTSDelta $\delta -> do + backend <- maybe (chooseBackend$ keyFilename source) (return . Just) preferredbackend
k <- genKey source backend
let src = contentLocation source
ms <- liftIO $catchMaybeIO$ getFileStatus src
diff --git a/CHANGELOG b/CHANGELOG
index a5ccf1a..762ccbf 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -31,6 +31,7 @@ git-annex (6.20160528) UNRELEASED; urgency=medium
* Make lock and unlock work in v6 repos on files whose content is not
present.
* Fix update of associated files db when unlocking a file in a v6 repo.
+  * Make git clean filter preserve the backend that was used for a file.

-- Joey Hess <id@joeyh.name>  Fri, 27 May 2016 13:12:48 -0400

diff --git a/Command/Smudge.hs b/Command/Smudge.hs
index 933a0a0..5a4b879 100644
--- a/Command/Smudge.hs
+++ b/Command/Smudge.hs
@@ -13,9 +13,11 @@ import Annex.Content
import Annex.FileMatcher
import Annex.Ingest
+import Annex.CatFile
import Logs.Location
import qualified Database.Keys
import Git.FilePath
+import Backend

import qualified Data.ByteString.Lazy as B

@@ -78,8 +80,16 @@ clean file = do
-- and not stdin, we need to consume all
-- stdin, or git will get annoyed.
B.length b seq return ()
+				-- Look up the backend that was used
+				-- for this file before, so that when
+				-- git re-cleans a file its backend does
+				-- not change.
+				currbackend <- maybe Nothing (maybeLookupBackendName . keyBackendName)
+					<$> catKeyFile file liftIO . emitPointer - =<< go =<< ingest =<< lockDown cfg file + =<< go + =<< ingest' currbackend + =<< lockDown cfg file , liftIO$ B.hPut stdout b
)
stop
index 443a1ab..8cb9395 100644
@@ -56,3 +56,5 @@ git-annex version: 6.20160523+gitg33c00ab-1~ndall+1

[[!meta author=yoh]]
+
+> [[fixed|done]] --[[Joey]]


close
diff --git a/doc/bugs/bash_completion_file_is_missing_in_the_6.20160527_tarball_on_hackage.mdwn b/doc/bugs/bash_completion_file_is_missing_in_the_6.20160527_tarball_on_hackage.mdwn
index 5bea510..6b43347 100644
--- a/doc/bugs/bash_completion_file_is_missing_in_the_6.20160527_tarball_on_hackage.mdwn
+++ b/doc/bugs/bash_completion_file_is_missing_in_the_6.20160527_tarball_on_hackage.mdwn
@@ -1 +1,3 @@
The bash-completion.bash was gone in the 6.20160527 tarball on hackage. It used to be inside the tarball in <= 6.20160511 versions.
+
+> not a bug; [[done]] --[[Joey]]


Make lock and unlock work in v6 repos on files whose content is not present.
diff --git a/CHANGELOG b/CHANGELOG
index 0db6d1e..f67afb4 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -28,6 +28,8 @@ git-annex (6.20160528) UNRELEASED; urgency=medium
that had not been synced back to master.
(This bug caused broken tree objects to get built by a later git annex
sync.)
+  * Make lock and unlock work in v6 repos on files whose content is not
+    present.

-- Joey Hess <id@joeyh.name>  Fri, 27 May 2016 13:12:48 -0400

diff --git a/Command/Lock.hs b/Command/Lock.hs
index 1cd50de..f9d9036 100644
--- a/Command/Lock.hs
+++ b/Command/Lock.hs
@@ -45,7 +45,7 @@ startNew file key = ifM (isJust <$> isAnnexLink file) ) where go (Just key') - | key' == key = error "content not present; cannot lock" + | key' == key = cont True | otherwise = errorModified go Nothing = ifM (isUnmodified key file) diff --git a/Command/Unlock.hs b/Command/Unlock.hs index 2fe1175..4dc0264 100644 --- a/Command/Unlock.hs +++ b/Command/Unlock.hs @@ -37,14 +37,9 @@ start :: FilePath -> Key -> CommandStart start file key = ifM (isJust <$> isAnnexLink file)
( do
showStart "unlock" file
-		ifM (inAnnex key)
-			( ifM versionSupportsUnlockedPointers
-				( next $performNew file key - , startOld file key - ) - , do - warning "content not present; cannot unlock" - next$ next $return False + ifM versionSupportsUnlockedPointers + ( next$ performNew file key
+			, startOld file key
)
, stop
)
@@ -52,11 +47,16 @@ start file key = ifM (isJust <$> isAnnexLink file) performNew :: FilePath -> Key -> CommandPerform performNew dest key = do destmode <- liftIO$ catchMaybeIO $fileMode <$> getFileStatus dest
-	replaceFile dest $\tmp -> do - r <- linkFromAnnex key tmp destmode - case r of - LinkAnnexOk -> return () - _ -> error "unlock failed" + replaceFile dest$ \tmp ->
+		ifM (inAnnex key)
+			( do
+				r <- linkFromAnnex key tmp destmode
+				case r of
+					_ -> error "unlock failed"
+			, liftIO $writePointerFile tmp key destmode + ) next$ cleanupNew dest key destmode

cleanupNew ::  FilePath -> Key -> Maybe FileMode -> CommandCleanup
diff --git a/doc/bugs/content_not_present__58___cannot_lock.mdwn b/doc/bugs/content_not_present__58___cannot_lock.mdwn
index ddf86f4..89bf043 100644
--- a/doc/bugs/content_not_present__58___cannot_lock.mdwn
+++ b/doc/bugs/content_not_present__58___cannot_lock.mdwn
@@ -22,4 +22,5 @@ lock git/bup.git/bupindex.hlink git-annex: content not present; cannot lock

### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

-
+> Well, that was simply not implemented, but I've done so now. (unlocking
+> too). [[done]] --[[Joey]]


diff --git a/doc/forum/git_annex_wants_to_repair_every_time_it__39__s_running.mdwn b/doc/forum/git_annex_wants_to_repair_every_time_it__39__s_running.mdwn
new file mode 100644
index 0000000..db4b368
--- /dev/null
+++ b/doc/forum/git_annex_wants_to_repair_every_time_it__39__s_running.mdwn
@@ -0,0 +1,5 @@
+As I tried to explain [here](http://git-annex.branchable.com/forum/__34__git_annex_assistant_--stop__34___doesn__39__t_work_in_OSX_10.11.4/): Every time I try to start annex, I have the problem that it wants to repair my repo. I'm not sure what's causing this and how to proceed. I would appreciate any pointers on how to triage the problem. There is a lot more info in the other thread.
+
+OS X: 10.11.4
+git: 2.8.3 (installed via brew)
+git annex: 6.20160527  (installed via brew)


Added a comment: Still seeing this
diff --git a/doc/bugs/high_cpu_usage_in_cat-file_and_webapp/comment_2_f26b8846e1651c507d259b9891d26136._comment b/doc/bugs/high_cpu_usage_in_cat-file_and_webapp/comment_2_f26b8846e1651c507d259b9891d26136._comment
new file mode 100644
index 0000000..f650709
--- /dev/null
+++ b/doc/bugs/high_cpu_usage_in_cat-file_and_webapp/comment_2_f26b8846e1651c507d259b9891d26136._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ subject="Still seeing this"
+ date="2016-06-08T18:03:51Z"
+ content="""
+This is on jessie with 5.20151208-1~bpo8.
+
+git-annex webapp and git --git-dir=.git --work-tree=. --literal-pathspecs -c core.bare=false cat-file --batch are using considerable CPU time.  The strace appears to be this:
+
+[[!format text \"\"\"
+[pid 29222] open(\".git/annex/journal/1e9_9e8_SHA256E-s109439--4df4c11870535f09bea79e309ac1d3de0e3ed5054900257c212277c0dc84290e.odt.log\", O_RDONLY|O_NOCTTY|O_NONBLOCK) = -1 ENOENT (No such file or directory)
+[pid 29222] write(15, \"refs/heads/git-annex:1e9/9e8/SHA\"..., 119) = 119
+[pid 29222] write(7, \"\1\0\0\0\0\0\0\0\", 8) = 8
+[pid 29161] read(7, \"\1\0\0\0\0\0\0\0\", 8) = 8
+[pid 29222] read(18, \"fb6b750de51cf247dae90279745c2478\"..., 8096) = 341
+\"\"\"]]
+
+What's more, the webapp shows that it's only synced to one of the two remotes, despite both being available.  The log has nothing about kicking off a sync run.
+
+I also wonder if this is and [[bugs/Assistant_having_a_child_git_cat-file_--batch_do_the_same_thing_over_and_over_and_using_a_lot_of_memory]] are the same issue.
+"""]]


Avoid a crash if getpwuid does not work, when querying the user's full name.
diff --git a/CHANGELOG b/CHANGELOG
index 79f07ba..b9c7792 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -22,6 +22,8 @@ git-annex (6.20160528) UNRELEASED; urgency=medium
drop content from disk before writing location log.
* Fix bad automatic merge conflict resolution between an annexed file
and a directory with the same name when in an adjusted branch.
+  * Avoid a crash if getpwuid does not work, when querying the user's full
+    name.

-- Joey Hess <id@joeyh.name>  Fri, 27 May 2016 13:12:48 -0400

diff --git a/Utility/UserInfo.hs b/Utility/UserInfo.hs
index c601011..c2edde2 100644
--- a/Utility/UserInfo.hs
+++ b/Utility/UserInfo.hs
@@ -15,6 +15,7 @@ module Utility.UserInfo (
) where

import Utility.Env
+import Utility.Exception

import System.PosixCompat
import Control.Applicative
@@ -47,7 +48,7 @@ myUserGecos :: IO (Maybe String)
#if defined(__ANDROID__) || defined(mingw32_HOST_OS)
myUserGecos = return Nothing
#else
-myUserGecos = Just <$> myVal [] userGecos +myUserGecos = catchMaybeIO$ myVal [] userGecos
#endif

myVal :: [String] -> (UserEntry -> String) -> IO String
new file mode 100644
index 0000000..a72c67e
--- /dev/null
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ subject="""comment 1"""
+ date="2016-06-08T17:31:41Z"
+ content="""
+IIRC it's supposed to be possible to configure a system such as LDAP
+so that getpwuid queries it, instead of failing as it seems to on your
+system, using /etc/nsswitch.conf. But I'm not sure about that or how to
+do it and you'd need to be the admin of the system.
+
+git-annex has three calls to getpwuid (aka getUserEntryForID).
+In two of them it first looks at commonly set environment variables
+(HOME and USER/LOGNAME) and only uses getpwuid as a fallback.
+It seems reasonable to crash when git-annex can't determine the user's home
+directory or username in code that needs it, especially since all three
+environment variables are almost always going to be set.
+
+The third use is a GECOS lookup, and here there's no corresponding
+environment variable and git-annex can deal without knowing the full name
+of the user. So, I've made that not crash if getpwuid fails.
+
+(There's also a little-used code path where getpwnam is used
+to expand ~ in a git remote path. Seems best to leave this crashing
+too if used on a system that does not have a working getpwnam.)
+
+So, you can try upgrading to an autobuild that has the fix, and if it still
+crashes, check that HOME and USER are set.
+"""]]


remove spam
diff --git a/doc/download/comment_5_46d4f8bba77ec52a5bffcf5b2591dc36._comment b/doc/download/comment_5_46d4f8bba77ec52a5bffcf5b2591dc36._comment
deleted file mode 100644
index be12a7e..0000000
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- subject="Coment 5"
- date="2016-06-06T14:59:08Z"
- content="""
-Long Path Tool” is very helpful for this error !
-You can use to solve this problem
-thanks
-"""]]


devblog
diff --git a/doc/devblog/day_397__befuddled.mdwn b/doc/devblog/day_397__befuddled.mdwn
new file mode 100644
index 0000000..a7d35fc
--- /dev/null
+++ b/doc/devblog/day_397__befuddled.mdwn
@@ -0,0 +1,28 @@
+Been having a difficult time fixing the two remaining test suite failures
+when run on a FAT filesystem.
+
+On Friday, I got quite lost trying to understand the first failure. At
+first I thought it had something to do with queued git staging commands not
+being run in the right git environment when git-annex is using a different
+index file or work tree. I did find and fix a potential bug in that area.
+It might be that some reports long ago of git-annex branch files getting
+written to the master branch was caused by that. But, fixing it did not
+help with the test suite failure at hand.
+
+Today, I quickly found the actual cause of the first failure.
+Of course, it had nothing to do with queued git commands at
+all, and was a simple fix in the end.
+
+But, I've been staring at the second failure for hours and am not much
+wiser. All I know is, an invalid tree object gets generated by the adjusted
+branch code that contains some files more than once. (git gets very
+confused when a repository contains such tree objects; if you wanted to break a
+git repository, getting such trees into it might be a good way. *cough*) This
+invalid tree object seems to be caused by the basis ref for the adjusted branch
+diverging somehow from the adjusted branch itself. I have not been able to
+determine why or how the basis ref can diverge like that.
+
+Also, this failure is somewhat indeterminite, doesn't always occur and
+reordering the tests in the test suite can hide it. Weird.
+
+Well, hopefully looking at it again later with fresh eyes will help.


diff --git a/doc/forum/git-annex__58___getUserEntryForID__58___does_not_exist___40__no_such_user__41___.mdwn b/doc/forum/git-annex__58___getUserEntryForID__58___does_not_exist___40__no_such_user__41___.mdwn
new file mode 100644
index 0000000..247407d
--- /dev/null
+++ b/doc/forum/git-annex__58___getUserEntryForID__58___does_not_exist___40__no_such_user__41___.mdwn
@@ -0,0 +1,24 @@
+Hi,
+
+I ran into this problem:
+$git annex mirror --from origin +git-annex: getUserEntryForID: does not exist (no such user) + +I found in the forum that, if the the machine uses LDAP etc. I should be able to use the HOME and USER environment variables. + +I'm not quite sure what the user authentication management on the machine I'm on is and I'd rather not deal with the sysadmins. +But I know that the /etc/passwd file does not have my user listed. +I do have the the HOME and USER variables properly set and exported. + +$ echo $HOME$USER
+/nethome/iiossifov iiossifov
+
+In the git-annex source code I found that the error occurs in Utility/UserInfo.hs.
+I can't read Haskell but my guess is that the problem is with the myUserGecos for which there is no environment variable fall back.
+
+I'm using the pre-build git-annex version: 6.20160527-gb7d4774
+
+Would you be able to advice me how to deal with the problem?
+
+Thank you,
+Ivan


diff --git a/doc/bugs/__34__metadata_only__34___git-remote-gcrypt_syncing_files_anyway/comment_2_b91f9337b7ed536539bb3236b7552d82._comment b/doc/bugs/__34__metadata_only__34___git-remote-gcrypt_syncing_files_anyway/comment_2_b91f9337b7ed536539bb3236b7552d82._comment
new file mode 100644
index 0000000..db88d6b
--- /dev/null
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ nickname="chris"
+ subject="comment 2"
+ date="2016-06-07T13:40:24Z"
+ content="""
+The way I got round this was to give the metadata-only repository an annex-uuid of 00000000-0000-0000-0000-000000000000, and to run \"git annex wanted remotename nothing\".  Now everything works as I want, whether with the assistant or when I run git annex sync --content [--all] manually.
+"""]]


diff --git a/doc/forum/Building_Android_app_fails.mdwn b/doc/forum/Building_Android_app_fails.mdwn
new file mode 100644
index 0000000..bf0e69e
--- /dev/null
+++ b/doc/forum/Building_Android_app_fails.mdwn
@@ -0,0 +1,69 @@
+When trying to build the Android app, I'm able to successfully set up the chroot, and enter it. Once I am inside, as the user "builder" I get a failure on the second step of the installation process "In the chroot, run standalone/android/install-haskell-packages".
+
+When I run this command, I get the following error:
+
+    cbits/cryptonite_rdrand.c: In function ‘cryptonite_get_rand_bytes’:
+
+    cbits/cryptonite_rdrand.c:63:2:
+         error: inconsistent operand constraints in an ‘asm’
+          asm(".byte 0x48,0x0f,0xc7,0xf0; setc %1" \
+          ^
+
+    cbits/cryptonite_rdrand.c:78:3:
+         note: in expansion of macro ‘inline_rdrand_rax’
+           inline_rdrand_rax(tmp, ok);
+           ^
+
+    cbits/cryptonite_rdrand.c:63:2:
+         error: inconsistent operand constraints in an ‘asm’
+          asm(".byte 0x48,0x0f,0xc7,0xf0; setc %1" \
+          ^
+
+    cbits/cryptonite_rdrand.c:87:3:
+         note: in expansion of macro ‘inline_rdrand_rax’
+           inline_rdrand_rax(tmp, ok);
+           ^
+
+    cbits/cryptonite_rdrand.c:63:2:
+         error: inconsistent operand constraints in an ‘asm’
+          asm(".byte 0x48,0x0f,0xc7,0xf0; setc %1" \
+          ^
+
+    cbits/cryptonite_rdrand.c:87:3:
+         note: in expansion of macro ‘inline_rdrand_rax’
+           inline_rdrand_rax(tmp, ok);
+           ^
+
+    cbits/cryptonite_rdrand.c:63:2:
+         error: inconsistent operand constraints in an ‘asm’
+          asm(".byte 0x48,0x0f,0xc7,0xf0; setc %1" \
+          ^
+
+    cbits/cryptonite_rdrand.c:94:3:
+         note: in expansion of macro ‘inline_rdrand_rax’
+           inline_rdrand_rax(tmp, ok);
+           ^
+    cabal: Error: some packages failed to install:
+    cryptonite-0.16 failed during the building phase. The exception was:
+    ExitFailure 1
+
+So, there is a problem with building the haskell package "cryptonite-0.16".
+
+I also get this output a little further up:
+
+    Failed to install cryptonite-0.16
+    Build log ( /home/builder/.cabal/logs/cryptonite-0.16.log ):
+    Warning: cryptonite.cabal: Unknown fields: extra-doc-files (line 37)
+    Fields allowed in this section:
+    copyright, maintainer, build-depends, stability, homepage,
+    package-url, bug-reports, synopsis, description, category, author,
+    tested-with, data-files, data-dir, extra-source-files,
+    extra-tmp-files
+
+So perhaps that is helpful as well.
+
+
+If anyone has any guidance on how to move beyond this or what needs to be changed, I'd appreciate it!
+
+Thanks!


diff --git a/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file/comment_1_7bcc0111015985bde398a2a1a02b28ba._comment b/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file/comment_1_7bcc0111015985bde398a2a1a02b28ba._comment
new file mode 100644
index 0000000..02c18f8
--- /dev/null
+++ b/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file/comment_1_7bcc0111015985bde398a2a1a02b28ba._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ subject="SOLVED"
+ date="2016-06-06T15:45:12Z"
+ content="""
+I solved the problem with fsck the disk.
+"""]]


diff --git a/doc/download/comment_5_46d4f8bba77ec52a5bffcf5b2591dc36._comment b/doc/download/comment_5_46d4f8bba77ec52a5bffcf5b2591dc36._comment
new file mode 100644
index 0000000..be12a7e
--- /dev/null
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ subject="Coment 5"
+ date="2016-06-06T14:59:08Z"
+ content="""
+Long Path Tool” is very helpful for this error !
+You can use to solve this problem
+thanks
+"""]]


removed
diff --git a/doc/tips/downloading_podcasts/comment_25_ae0d10d697004e8f8e615f5d10c14d23._comment b/doc/tips/downloading_podcasts/comment_25_ae0d10d697004e8f8e615f5d10c14d23._comment
deleted file mode 100644
index 3df52d0..0000000
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- subject="Long Path Tool"
- date="2016-06-06T14:57:48Z"
- content="""
-Long Path Tool” is very helpful for this error !
-You can use to solve this problem
-thanks
-"""]]


Added a comment: Long Path Tool
diff --git a/doc/tips/downloading_podcasts/comment_25_ae0d10d697004e8f8e615f5d10c14d23._comment b/doc/tips/downloading_podcasts/comment_25_ae0d10d697004e8f8e615f5d10c14d23._comment
new file mode 100644
index 0000000..3df52d0
--- /dev/null
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ subject="Long Path Tool"
+ date="2016-06-06T14:57:48Z"
+ content="""
+Long Path Tool” is very helpful for this error !
+You can use to solve this problem
+thanks
+"""]]


Added a comment: I think I have figured out something
diff --git a/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_3_f2770f86138d9b8489fbf9a25462b4ce._comment b/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_3_f2770f86138d9b8489fbf9a25462b4ce._comment
new file mode 100644
index 0000000..e0c0d3a
--- /dev/null
+++ b/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_3_f2770f86138d9b8489fbf9a25462b4ce._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ subject="I think I have figured out something"
+ date="2016-06-05T22:42:17Z"
+ content="""
+When I git-annex drop --force a file from my direct repository, it gets replaced by a symbolic-link-like file, containing a path.
+Then, when I git-annex sync the repository to propagate the changes I have made, the file's content gets updated as if the file has been replaced.
+
+My question is then: why does the original file gets replaced by the link-like file when I drop it?
+"""]]


diff --git a/doc/bugs/content_not_present__58___cannot_lock/comment_1_9a9a53d85660531cc3843dc9f339f187._comment b/doc/bugs/content_not_present__58___cannot_lock/comment_1_9a9a53d85660531cc3843dc9f339f187._comment
new file mode 100644
index 0000000..d09bc9e
--- /dev/null
+++ b/doc/bugs/content_not_present__58___cannot_lock/comment_1_9a9a53d85660531cc3843dc9f339f187._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ subject="comment 1"
+ date="2016-06-05T02:57:42Z"
+ content="""
+comment to get replies emailed
+"""]]


diff --git a/doc/bugs/content_not_present__58___cannot_lock.mdwn b/doc/bugs/content_not_present__58___cannot_lock.mdwn
new file mode 100644
index 0000000..ddf86f4
--- /dev/null
+++ b/doc/bugs/content_not_present__58___cannot_lock.mdwn
@@ -0,0 +1,25 @@
+Cannot lock files which are not present.  This prevents converting v6 files from flat files to symlinks when they are missing.
+If a file is not present, locking it should either replace it with a broken symlink, or continue on to the next file with a warning.
+
+### What steps will reproduce the problem?
+I think this happened from unlocking a file in one v6 repo and syncing with a different v6 repo, perhaps via a v5 repo in the middle.  In repo #2, the file became unlocked but missing for me.
+
+### What version of git-annex are you using? On what operating system?
+6.20160528-g191d2ba on linux
+
+
+[[!format sh """
+
+# I am trying to lock all files in my repo to downgrade from v6 back to v5, for stability
+
+$git annex lock +lock git/bup.git/bupindex.hlink git-annex: content not present; cannot lock + +# but I cannot +"""]] + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + +  diff --git a/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file.mdwn b/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file.mdwn new file mode 100644 index 0000000..0621f30 --- /dev/null +++ b/doc/forum/Structure_needs_cleaning_git-annex__58___fd__58__15__58___hGetLine__58___end_of_file.mdwn @@ -0,0 +1,24 @@ +I get this error after a hard reboot of the device when running git annex sync + + + fatal: Cannot open '/media/usbplatte/Podcasts/.git/annex/journal/e2f_ae4_SHA256E-s8388114--55d0a8881d06e24584ad9a94a6c5a369ea301309e0626497ea8c67bc65c3d0b2.mp3.log.web': Die Struktur muss bereinigt werden + + git-annex: fd:15: hGetLine: end of file + fatal: Cannot open '/media/usbplatte/Podcasts/.git/annex/journal/e2f_ae4_SHA256E-s8388114--55d0a8881d06e24584ad9a94a6c5a369ea301309e0626497ea8c67bc65c3d0b2.mp3.log.web': Die Struktur muss bereinigt werden + + git-annex: fd:15: hGetLine: end of file + +I ran + + git annex fsck + +and + + git annex repair + +Both commands finished with Status Ok. + +How can I save the repo? + +TIA +juh  Added a comment: A more detailed walkthrough diff --git a/doc/install/Docker/comment_1_909e3b2da0050ce1102c289cc5aac522._comment b/doc/install/Docker/comment_1_909e3b2da0050ce1102c289cc5aac522._comment new file mode 100644 index 0000000..ea43a4d --- /dev/null +++ b/doc/install/Docker/comment_1_909e3b2da0050ce1102c289cc5aac522._comment @@ -0,0 +1,66 @@ +[[!comment format=mdwn + username="matei.david@d6acba23dba331f26fc4756d01c7ab65cf3aee4d" + nickname="matei.david" + subject="A more detailed walkthrough" + date="2016-06-03T18:48:11Z" + content=""" +I want to use a recent git-annex version, and I prefer a Docker solution to a binary. There wasn't much documentation on how to do that, so here are the steps I took, just in case anybody finds this information useful. This is not a complete guide, just more than I was able to find so far. + +- Install Docker so that you can access it (eg, docker info) without sudo. (Beyond the scope in here.) + +- Create a git-annex image as follows: + +Dockerfile: + + FROM debian:unstable + RUN apt-get update && apt-get install -y ssh git man git-annex=6.20160511-1 + # maybe install gpg2, other remotes, etc + #RUN apt-get install -y gnupg2 && ln -s /usr/bin/gpg2 /usr/local/bin/gpg + #RUN apt-get install -y golang-go && GOPATH=/usr/local go get github.com/encryptio/git-annex-remote-b2 + VOLUME /data + WORKDIR /data + +Build image and run basic test: + + docker build -t git-annex-6.20160511-1 . + docker run -it --rm git-annex-6.20160511-1 git-annex version + +- You still need to: + + - On host: Replace git-annex by a script that invokes the Dockerized version. + - In container: + - Use non-root uids, otherwise you end up creating files with uid 0. + - Access ssh and gpg credentials on host. + - Access ~/.gitconfig on host. + - Mount git repository root as a volume, not just the current dir. + - Access AWS/B2 credentials, if applicable. + +Here is a sample script that achieves these. Name it git-annex, and place it in your PATH before the host git-annex (or just uninstall the latter). + + #!/bin/bash + CONT_NAME=${CONT_NAME:-git-annex-6.20160511-1}
+    # if in git repo, mount root as /data, and cd into relative subdir
+    # if not, mount cwd as /data
+    abs_dir=$(readlink -e .) + root_dir=$(git rev-parse --show-toplevel 2>/dev/null || true)
+    root_dir=${root_dir:-$abs_dir}
+    rel_dir=${abs_dir#$root_dir}
+    # if run by git, assume command is git-annex
+    # otherwise, don't assume, to allow other uses
+    cmd=
+    ! [ \"$(basename \"$(readlink -e /proc/$PPID/exe)\")\" = \"git\" ] || cmd=git-annex + exec docker run -it --rm \ + -u$(id -u):$(id -g) \ + -v /etc/passwd:/etc/passwd:ro \ + -v$HOME/.ssh:$HOME/.ssh \ + -v$HOME/.gnupg:$HOME/.gnupg \ + -v$HOME/.gitconfig:$HOME/.gitconfig \ + -v \"$root_dir\":/data \
+        ${AWS_ACCESS_KEY_ID:+-e AWS_ACCESS_KEY_ID=\"$AWS_ACCESS_KEY_ID\"} \
+        ${AWS_SECRET_ACCESS_KEY:+-e AWS_SECRET_ACCESS_KEY=\"$AWS_SECRET_ACCESS_KEY\"} \
+        ${B2_ACCOUNT_ID:+-e B2_ACCOUNT_ID=\"$B2_ACCOUNT_ID\"} \
+        ${B2_APP_KEY:+-e B2_APP_KEY=\"$B2_APP_KEY\"} \
+        -w /data\"$rel_dir\" \ +$CONT_NAME $cmd \"$@\"
+
+"""]]

`