Recent changes to this wiki:

try to structure a guide
diff --git a/doc/todo/build_a_user_guide.mdwn b/doc/todo/build_a_user_guide.mdwn
index 44b350e..1fcf9bf 100644
--- a/doc/todo/build_a_user_guide.mdwn
+++ b/doc/todo/build_a_user_guide.mdwn
@@ -3,3 +3,43 @@ there's a lot of good documentation on this wiki, but it's hard to find sometime
 a good example of this problem is [[todo/document_standard_groups_more_extensively_in_the_UI]]. --[[anarcat]]
 
 update: a beginning of this may be the the [[workflow]] page but it lacks a lot of details... 
+
+So we have those entry points so far:
+
+  * [[git-annex]] - the manpage
+  * [[walkthrough]] - "A walkthrough of some of the basic features of git-annex, using the command line", described as "only one possible workflow for using git-annex"
+  * [[assistant]] - a whole subtree of pages describing the assistant, includes a [[assistant/quickstart]] - introduction to the assistant with a series of screenshots, described in [[walkthrough]] as "If you don't want to use the command line, see quickstart instead.", linked from the [[assistant]] page
+  * [[workflow]] - a summary of the different workflows that git-annex can use
+  * [[special remotes]] - a good list of "supported backends", which may be a better wording
+  * inversely, [[not]] is what is *not* supported, obviously
+  * [[install]] - how to install git-annex, of course
+  * [[tips]] - a mish-mash list of "how to do X in git-annex", 68 pages at the time of writing
+  * there's the "details" section on the frontpage which covers lots of the [[internals]], [[design]] and so on
+  * there are also what i consider to be "leaf" pages like [[how it works]] or [[sync]] there
+
+So it seems the fundamentals of such a user guide are there. It's just a matter of grouping this in a meaningful way.
+
+I am thinking the following structure may be a good basis:
+
+ * Introduction
+   * [[how it works]]?
+   * ...?
+ * [[Install|install]]
+ * Walkthrough
+   * [[comandline|walkthrough]]
+   * [[graphical|assistant/quickstart]]?
+ * How do I...
+   * [[Supported backends|special remotes]]
+   * [[Unsupported features|not]]
+   * [[split repositories|tips/splitting_a_repository]]
+   * [[merge repositories|tips/migrating_two_seperate_disconnected_directories_to_git_annex]]
+   * deal with lots of files: [[tips/Repositories_with_large_number_of_files]] or merge into [[scalability/]]?
+   * decide which files go where? something about [[preferred_content]] and [[preferred_content/standard_groups/]]?
+   * sort and regroup the best [[tips]] pages
+ * Troubleshooting / FAQ?
+   * [[Common mistakes|tips/antipatterns]]
+   * sort out the best of [[forums]] and [[tips]] that commonly occur
+ * [[Design]]
+   * [[internals]]
+   * [[encryption]]
+   * ... etc - all the developer stuff users shouldn't usually have to know unless they care about performance or need to reimplement something

Added a comment: merge with scalability?
diff --git a/doc/tips/Repositories_with_large_number_of_files/comment_6_9169a33c06cf8aea231cdd8f51ce17b6._comment b/doc/tips/Repositories_with_large_number_of_files/comment_6_9169a33c06cf8aea231cdd8f51ce17b6._comment
new file mode 100644
index 0000000..1fb1de9
--- /dev/null
+++ b/doc/tips/Repositories_with_large_number_of_files/comment_6_9169a33c06cf8aea231cdd8f51ce17b6._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="anarcat"
+ avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
+ subject="merge with scalability?"
+ date="2017-04-24T14:10:29Z"
+ content="""
+shouldn't this tip be merged into the [[scalability]] page directly?
+"""]]

link to the assistant page here
diff --git a/doc/workflow.mdwn b/doc/workflow.mdwn
index 1855649..e5d440c 100644
--- a/doc/workflow.mdwn
+++ b/doc/workflow.mdwn
@@ -27,6 +27,8 @@ produce file changes.  When you move files into or out of your repository
 folder, git-annex should record the changes and automatically propagate
 them to other connected machines.
 
+The webapp is part of the larger [[assistant]].
+
 # 2. [[git annex assistant|git-annex-assistant]] without the webapp
 
 You could call [[`git annex assistant`|git-annex-assistant]] the

Added a comment
diff --git a/doc/forum/Lots_of_4k_symlinks/comment_1_96384eaeef1d067a24678c7aa3613063._comment b/doc/forum/Lots_of_4k_symlinks/comment_1_96384eaeef1d067a24678c7aa3613063._comment
new file mode 100644
index 0000000..10a9bbd
--- /dev/null
+++ b/doc/forum/Lots_of_4k_symlinks/comment_1_96384eaeef1d067a24678c7aa3613063._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="CandyAngel"
+ avatar="http://cdn.libravatar.org/avatar/15c0aade8bec5bf004f939dd73cf9ed8"
+ subject="comment 1"
+ date="2017-04-24T13:55:02Z"
+ content="""
+> For the \"archivist use case\" of annex, this might lead to tens or hundreds of MBs of disk occupied by symlinks which actually don't add up to more than a few MBs.
+
+    $ pwd
+    /home/sorting_annex/mnt/keyfile
+    $ du -shc *-*
+    ...
+    33M     fd0dc9d3-ad62-429e-ba1b-acc26a453ca4
+    33M     fd2fc989-bea7-4ffb-bbc8-2e34cd0e5be5
+    33M     fd79bbd4-d41e-4ea8-acc8-86437c5eed7c
+    33M     ffbd042e-f6d9-4450-9a57-8ed1086f587c
+    2.7G    total
+    $
+
+Just a bit :P (yes, that is 2.7G of symlinks *so far*)
+"""]]

put reverse on top of page
diff --git a/doc/tips/splitting_a_repository.mdwn b/doc/tips/splitting_a_repository.mdwn
index b7c04b1..89080f6 100644
--- a/doc/tips/splitting_a_repository.mdwn
+++ b/doc/tips/splitting_a_repository.mdwn
@@ -1,5 +1,7 @@
 [[!meta title="Splitting a git-annex repository"]]
 
+Note: this is the reverse of [[migrating two seperate disconnected directories to git annex]].
+
 I have a [git annex](https://git-annex.branchable.com/) repo for all my media
 that has grown to 57866 files and git operations are getting slow, especially
 on external spinning hard drives, so I decided to split it into separate
@@ -53,6 +55,4 @@ Update: Antoine Beaupré pointed me to
 [this tip about Repositories with large number of files](http://git-annex.branchable.com/tips/Repositories_with_large_number_of_files/)
 which I will try next time one of my repositories grows enough to hit a performance issue.
 
-Note: this is the reverse of [[migrating two seperate disconnected directories to git annex]].
-
 > This document was originally written by [Enrico Zini](http://www.enricozini.org/blog/2017/debian/splitting-a-git-annex-repository/) and added to this wiki by [[anarcat]].

link with brother page
diff --git a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn
index 8f078c7..c8faf34 100644
--- a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn
+++ b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn
@@ -1,3 +1,5 @@
+Note: this is the reverse of [[splitting_a_repository]].
+
 Scenario
 --------
 

link to sister page
diff --git a/doc/tips/splitting_a_repository.mdwn b/doc/tips/splitting_a_repository.mdwn
index b904303..b7c04b1 100644
--- a/doc/tips/splitting_a_repository.mdwn
+++ b/doc/tips/splitting_a_repository.mdwn
@@ -53,4 +53,6 @@ Update: Antoine Beaupré pointed me to
 [this tip about Repositories with large number of files](http://git-annex.branchable.com/tips/Repositories_with_large_number_of_files/)
 which I will try next time one of my repositories grows enough to hit a performance issue.
 
+Note: this is the reverse of [[migrating two seperate disconnected directories to git annex]].
+
 > This document was originally written by [Enrico Zini](http://www.enricozini.org/blog/2017/debian/splitting-a-git-annex-repository/) and added to this wiki by [[anarcat]].

add signatures to the various sections
diff --git a/doc/tips/Repositories_with_large_number_of_files.mdwn b/doc/tips/Repositories_with_large_number_of_files.mdwn
index a8f498d..347f6f9 100644
--- a/doc/tips/Repositories_with_large_number_of_files.mdwn
+++ b/doc/tips/Repositories_with_large_number_of_files.mdwn
@@ -44,7 +44,7 @@ You can avoid this by keeping the number of files in a directory to between 5000
 
 [fpart](https://sourceforge.net/projects/fpart/) can be a very useful tool to achieve this.
 
-This sort of usage was discussed in [[forum/Handling_a_large_number_of_files]] and [[forum/__34__git_annex_sync__34___synced_after_8_hours]].
+This sort of usage was discussed in [[forum/Handling_a_large_number_of_files]] and [[forum/__34__git_annex_sync__34___synced_after_8_hours]]. -- [[CandyAngel]]
 
 # Forget tracking information
 
@@ -52,4 +52,4 @@ In addition to keeping track of where files are, git-annex keeps a *log* that ke
 
 You can use the [[git-annex-forget]] command to drop historical location tracking info for files.
 
-Note: this was discussed in [[forum/scalability_with_lots_of_files]].
+Note: this was discussed in [[forum/scalability_with_lots_of_files]]. -- [[anarcat]]

drop heading that was unnecessary and asymetric
diff --git a/doc/tips/Repositories_with_large_number_of_files.mdwn b/doc/tips/Repositories_with_large_number_of_files.mdwn
index d0bbfd1..a8f498d 100644
--- a/doc/tips/Repositories_with_large_number_of_files.mdwn
+++ b/doc/tips/Repositories_with_large_number_of_files.mdwn
@@ -44,10 +44,7 @@ You can avoid this by keeping the number of files in a directory to between 5000
 
 [fpart](https://sourceforge.net/projects/fpart/) can be a very useful tool to achieve this.
 
-## Topics discussing this sort of usage
-
-* [[forum/Handling_a_large_number_of_files]]
-* [[forum/__34__git_annex_sync__34___synced_after_8_hours]]
+This sort of usage was discussed in [[forum/Handling_a_large_number_of_files]] and [[forum/__34__git_annex_sync__34___synced_after_8_hours]].
 
 # Forget tracking information
 

add git-annex-forget
diff --git a/doc/tips/Repositories_with_large_number_of_files.mdwn b/doc/tips/Repositories_with_large_number_of_files.mdwn
index 799ee96..d0bbfd1 100644
--- a/doc/tips/Repositories_with_large_number_of_files.mdwn
+++ b/doc/tips/Repositories_with_large_number_of_files.mdwn
@@ -1,5 +1,7 @@
 Just as git does not scale well with large files, it can also become painful to work with when you have a large *number* of files. Below are things I have found to minimise the pain.
 
+[[!toc]]
+
 # Using version 4 index files
 
 During operations which affect the index, git writes an entirely new index out to index.lck and then replaces .git/index with it. With a large number of files, this index file can be quite large and take several seconds to write every time you manipulate the index!
@@ -46,3 +48,11 @@ You can avoid this by keeping the number of files in a directory to between 5000
 
 * [[forum/Handling_a_large_number_of_files]]
 * [[forum/__34__git_annex_sync__34___synced_after_8_hours]]
+
+# Forget tracking information
+
+In addition to keeping track of where files are, git-annex keeps a *log* that keeps track of where files *were*. This can take up space as well and slow down certain operations.
+
+You can use the [[git-annex-forget]] command to drop historical location tracking info for files.
+
+Note: this was discussed in [[forum/scalability_with_lots_of_files]].

fix fpart 403 error
diff --git a/doc/tips/Repositories_with_large_number_of_files.mdwn b/doc/tips/Repositories_with_large_number_of_files.mdwn
index c1f219e..799ee96 100644
--- a/doc/tips/Repositories_with_large_number_of_files.mdwn
+++ b/doc/tips/Repositories_with_large_number_of_files.mdwn
@@ -40,7 +40,7 @@ If it takes a long time to list the files in a directory, naturally, git(-annex)
 
 You can avoid this by keeping the number of files in a directory to between 5000 and 20000 (depends on the filesystem and its settings).
 
-[fpart](http://contribs.martymac.org/fpart/) can be a very useful tool to achieve this.
+[fpart](https://sourceforge.net/projects/fpart/) can be a very useful tool to achieve this.
 
 ## Topics discussing this sort of usage
 

add zini's split repo procedure
diff --git a/doc/tips/splitting_a_repository.mdwn b/doc/tips/splitting_a_repository.mdwn
new file mode 100644
index 0000000..b904303
--- /dev/null
+++ b/doc/tips/splitting_a_repository.mdwn
@@ -0,0 +1,56 @@
+[[!meta title="Splitting a git-annex repository"]]
+
+I have a [git annex](https://git-annex.branchable.com/) repo for all my media
+that has grown to 57866 files and git operations are getting slow, especially
+on external spinning hard drives, so I decided to split it into separate
+repositories.
+
+This is how I did it, with some help from `#git-annex`. Suppose the old big repo is at `~/oldrepo`:
+
+```
+# Create a new repo for photos only
+mkdir ~/photos
+cd photos
+git init
+git annex init laptop
+
+# Hardlink all the annexed data from the old repo
+cp -rl ~/oldrepo/.git/annex/objects .git/annex/
+
+# Regenerate the git annex metadata
+git annex fsck --fast
+
+# Also split the repo on the usb key
+cd /media/usbkey
+git clone ~/photos
+cd photos
+git annex init usbkey
+cp -rl ../oldrepo/.git/annex/objects .git/annex/
+git annex fsck --fast
+
+# Connect the annexes as remotes of each other
+git remote add laptop ~/photos
+cd ~/photos
+git remote add usbkey /media/usbkey
+```
+
+At this point, I went through all repos doing standard cleanup:
+
+```
+# Remove unneeded hard links
+git annex unused
+git annex dropunused --force 1-12345
+
+# Sync
+git annex sync
+```
+
+To make sure nothing is missing, I used `git annex find --not --in=here`
+to see if, for example, the usbkey that should have everything could be missing
+some thing.
+
+Update: Antoine Beaupré pointed me to
+[this tip about Repositories with large number of files](http://git-annex.branchable.com/tips/Repositories_with_large_number_of_files/)
+which I will try next time one of my repositories grows enough to hit a performance issue.
+
+> This document was originally written by [Enrico Zini](http://www.enricozini.org/blog/2017/debian/splitting-a-git-annex-repository/) and added to this wiki by [[anarcat]].

Added a comment: git annex standalone on Synology NAS
diff --git a/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock/comment_1_d165e4895e7043192888751218261282._comment b/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock/comment_1_d165e4895e7043192888751218261282._comment
new file mode 100644
index 0000000..7b92837
--- /dev/null
+++ b/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock/comment_1_d165e4895e7043192888751218261282._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="ewen"
+ avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e"
+ subject="git annex standalone on Synology NAS"
+ date="2017-04-23T07:52:19Z"
+ content="""
+After posting this bug, I found a [git annex tip on running a standalone build on the Synology NAS](http://git-annex.branchable.com/tips/Synology_NAS_and_git_annex/), so that may provide another workaround for me.  Particularly since the Synology DS216+ is actually a x86-64 based system:
+
+    ewen@nas01:/volume1$ uname -a
+    Linux nas01 3.10.102 #15047 SMP Thu Feb 23 02:23:28 CST 2017 x86_64 GNU/Linux synology_braswell_216+II
+    ewen@nas01:/volume1$ 
+
+but it is probably still worth figuring out the cause of the transfer lock issue on SMB for other SMB use cases.
+
+Ewen
+"""]]

Added a comment: git annex get ... transfer lock issues
diff --git a/doc/forum/git_annex_init_timeout/comment_3_1e733fad01e6b420c7fd9f7832e9b3f7._comment b/doc/forum/git_annex_init_timeout/comment_3_1e733fad01e6b420c7fd9f7832e9b3f7._comment
new file mode 100644
index 0000000..a047306
--- /dev/null
+++ b/doc/forum/git_annex_init_timeout/comment_3_1e733fad01e6b420c7fd9f7832e9b3f7._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="ewen"
+ avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e"
+ subject="git annex get ... transfer lock issues"
+ date="2017-04-23T07:36:08Z"
+ content="""
+The fix a couple of months ago appears to allow `git annex init` to complete on a SMB file share, which is great.  But FTR there are currently [issues with file transfers](https://git-annex.branchable.com/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock/) (link is to bug report about failing to get transfer lock), so use with a NAS is still \"work in progress\".
+
+Ewen
+"""]]

CC me
diff --git a/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock.mdwn b/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock.mdwn
index dc1fd28..3077bd0 100644
--- a/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock.mdwn
+++ b/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock.mdwn
@@ -243,3 +243,5 @@ and the relevant share:
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 I'm an extremely regular user of git-annex on OS X and Linux, for several years, using it as a podcatcher and to manage most of my "large file" media.  It's one of those "couldn't live without" tools.  Thanks for writing it.
+
+(Touched, to ask for comments to be emailed to me)

Described steps to reproduce git annex get failure on SMB share
diff --git a/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock.mdwn b/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock.mdwn
new file mode 100644
index 0000000..dc1fd28
--- /dev/null
+++ b/doc/bugs/SMB__58___git_annex_clone_works__44___get_fails_on_transfer_lock.mdwn
@@ -0,0 +1,245 @@
+### Please describe the problem.
+
+I have a Synology DS216+ NAS, which can export shares via SMB2, AFP or NFS.  Since I have a lot of "media" data in git-annex, I'd like to be able to put a useful copy of that data (ie, original file names) onto the NAS, and have git-annex track that content as another "local" copy of the content that is available (rather than having to track it by hand).
+
+Because the NFS shares have UID mapping issues (AFAICT NFS v4 id mapping is not supported without Kerberos authentication, due to changes since about Linux 3.4; the Synology is based on a Linux 3.10 kernel), I have been trying to use SMBv2 shares, with symlinks enabled ("Allow symbolic links within shared folders").  I'm aware that `git-annex` on a network file system is less supported, but this is still a use case that would be quite useful to manage larger pools of files.
+
+After updating to the latest release (to [fix git annex init timeouts](https://git-annex.branchable.com/forum/git_annex_init_timeout/)), I am able to complete the `git clone ...`, `git annex init`, and `git sync` steps.  (The SMB mount supports symlinks, but not hard links; hence pidlocks being needed.)
+
+But `git annex get` or `git annex copy` to transfer any of the content files fails with:
+
+    get README (transfer already in progress, or unable to take transfer lock) 
+      Unable to access these remotes: ashram-data
+
+whether initiated from the SMB mounted `git-annex` or a `git-annex` with the data on a local file system.  No content files get transferred.
+
+Of note, on this "SMB share with symlinks enabled" I can manually create symlinks, but `git annex` still detects symlinks as not possible and switches to direct mode.  However it leaves the existing symlinks from `git clone ...` in place.  It is unclear to me whether this is part of the cause of the "transfer lock" issue, or whether the transfer locks are, eg, not using pidlocks and thus failing because flock() does not work.
+
+### What steps will reproduce the problem?
+
+*   Mount a remote file system via SMB v2, with symlinks enabled (eg mounting from a Samba share should be sufficient, as that's what is running on the Synology NAS)
+
+*   `git clone` an existing `git-annex` onto a directory on that SMB mounted filesystem
+
+*   `git remote ....` configure one or more remotes with the file data
+
+*   `git annex init "SMB share"`
+
+*   `git annex sync`
+
+*   `git annex get ....` of an existing file
+
+or alternatively after the init/sync, go to a source `git-annex` and do:
+
+*   `git remote add smb ....`
+
+*   `git annex sync`
+
+*   `git annex copy --to=smb ...` of an existing file with content in that `git-annex`
+
+### What version of git-annex are you using? On what operating system?
+
+`git-annex` 6.20170320 (downloaded 2017-04-23) on Mac OS X 10.11 (El Capitan):
+
+    ewen@ashram:~$ uname -a
+    Darwin ashram 15.6.0 Darwin Kernel Version 15.6.0: Fri Feb 17 10:21:18 PST 2017; root:xnu-3248.60.11.4.1~1/RELEASE_X86_64 x86_64
+    ewen@ashram:~$ git annex version
+    git-annex version: 6.20170320-g41c5d9d
+    build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+    ewen@ashram:~$ 
+
+### Please provide any additional information below.
+
+[[!format sh """
+# Example session
+ewen@ashram:/nas01/annex/purchased$ git clone /bkup/purchased/mediabytes
+Cloning into 'mediabytes'...
+done.
+ewen@ashram:/nas01/annex/purchased$ cd mediabytes/
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git remote add ashram-data /data/purchased/mediabytes
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git remote add ashram /bkup/purchased/mediabytes
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git remote remove origin
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git annex init 'nas01 file server'
+init nas01 file server 
+  Detected a filesystem without POSIX fcntl lock support.
+
+  Enabling annex.pidlock.
+
+  Detected a filesystem without fifo support.
+
+  Disabling ssh connection caching.
+
+  Detected a crippled filesystem.
+
+  Disabling core.symlinks.
+
+  Enabling direct mode.
+ok
+(recording state in git...)
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git annex sync
+commit  ok
+pull ashram 
+From /bkup/purchased/mediabytes
+ * [new branch]      git-annex  -> ashram/git-annex
+ * [new branch]      master     -> ashram/master
+ * [new branch]      synced/git-annex -> ashram/synced/git-annex
+ * [new branch]      synced/master -> ashram/synced/master
+ok
+pull ashram-data 
+From /data/purchased/mediabytes
+ * [new branch]      git-annex  -> ashram-data/git-annex
+ * [new branch]      master     -> ashram-data/master
+ * [new branch]      synced/git-annex -> ashram-data/synced/git-annex
+ * [new branch]      synced/master -> ashram-data/synced/master
+ok
+(merging ashram-data/git-annex into git-annex...)
+(recording state in git...)
+push ashram 
+Counting objects: 8, done.
+Delta compression using up to 8 threads.
+Compressing objects: 100% (6/6), done.
+Writing objects: 100% (8/8), 806 bytes | 0 bytes/s, done.
+Total 8 (delta 2), reused 1 (delta 0)
+To /bkup/purchased/mediabytes
+   02b7699..0edf575  git-annex -> synced/git-annex
+ok
+push ashram-data 
+Counting objects: 8, done.
+Delta compression using up to 8 threads.
+Compressing objects: 100% (6/6), done.
+Writing objects: 100% (8/8), 806 bytes | 0 bytes/s, done.
+Total 8 (delta 2), reused 1 (delta 0)
+To /data/purchased/mediabytes
+   02b7699..0edf575  git-annex -> synced/git-annex
+ok
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git annex sync
+commit  ok
+pull ashram 
+ok
+pull ashram-data 
+ok
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git annex get .
+get README (transfer already in progress, or unable to take transfer lock) 
+  Unable to access these remotes: ashram-data
+
+  Try making some of these repositories available:
+  	00420eb2-1c9a-47e8-abfa-675b9ffb0782 -- naos_wd_1 backup drive
+   	0ac1326e-9e03-4c1e-bd51-6cd664988caa -- Naos (colo) fileserver
+   	0e5fd428-ba5b-4817-8489-9780bc62e655 -- naos_wd_2 backup drive
+   	1144edcc-b18f-4cb1-a828-6c272d7788d6 -- naos_wd_3 backup drive
+   	6142cfd4-83ba-410f-bf54-fd4b79e49fdb -- ashram external data drive [ashram-data]
+   	77aeca6a-a4f8-4734-befc-2a0b50687c52 -- tv home fileserver
+   	cc1c45f3-7c96-4103-b9c0-cce0f5f2cc36 -- bethel_data_drive
+   	eb0ca2b6-e8c6-43b0-a805-de17a4d7eeae -- naos_wd_4 backup drive
+failed
+get alex-lindsay-1.m4a (transfer already in progress, or unable to take transfer lock) 
+  Unable to access these remotes: ashram-data
+
+  Try making some of these repositories available:
+  	00420eb2-1c9a-47e8-abfa-675b9ffb0782 -- naos_wd_1 backup drive
+   	0ac1326e-9e03-4c1e-bd51-6cd664988caa -- Naos (colo) fileserver
+   	0e5fd428-ba5b-4817-8489-9780bc62e655 -- naos_wd_2 backup drive
+   	1144edcc-b18f-4cb1-a828-6c272d7788d6 -- naos_wd_3 backup drive
+   	6142cfd4-83ba-410f-bf54-fd4b79e49fdb -- ashram external data drive [ashram-data]
+   	77aeca6a-a4f8-4734-befc-2a0b50687c52 -- tv home fileserver
+   	cc1c45f3-7c96-4103-b9c0-cce0f5f2cc36 -- bethel_data_drive
+   	eb0ca2b6-e8c6-43b0-a805-de17a4d7eeae -- naos_wd_4 backup drive
+failed
+[...]
+[...]
+failed
+git-annex: get: 59 failed
+ewen@ashram:/nas01/annex/purchased/mediabytes$ 
+ewen@ashram:/nas01/annex/purchased/mediabytes$ ls -l | head
+total 118
+lrwx------  1 ewen  staff  182 23 Apr 18:53 README -> .git/annex/objects/3M/8w/SHA256E-s273--1eb5c07e12d99c99af8c163b5e005162820518020cc9922152852bd1422858ae/SHA256E-s273--1eb5c07e12d99c99af8c163b5e005162820518020cc9922152852bd1422858ae
+lrwx------  1 ewen  staff  200 23 Apr 18:53 alex-lindsay-1.m4a -> .git/annex/objects/9M/xJ/SHA256E-s26905448--c8ecda8ca67c3ff8ab1061b34a2974be41eb4b44421e3c39d1dda63f4de6b910.m4a/SHA256E-s26905448--c8ecda8ca67c3ff8ab1061b34a2974be41eb4b44421e3c39d1dda63f4de6b910.m4a
+lrwx------  1 ewen  staff  194 23 Apr 18:53 alexlindsay.txt -> .git/annex/objects/P7/7g/SHA256E-s17467--2e7ae57eff6db5a832681443eb3787d1d7f9e57a52e4fedb2b693136a3df19f0.txt/SHA256E-s17467--2e7ae57eff6db5a832681443eb3787d1d7f9e57a52e4fedb2b693136a3df19f0.txt
+lrwx------  1 ewen  staff  198 23 Apr 18:53 bourne-1.mp3 -> .git/annex/objects/98/vg/SHA256E-s7560841--a24ce7acabcb0dd46b9a9df6df8aff50145d8b1d5253308a51573a7742652943.mp3/SHA256E-s7560841--a24ce7acabcb0dd46b9a9df6df8aff50145d8b1d5253308a51573a7742652943.mp3
+lrwx------  1 ewen  staff  192 23 Apr 18:53 bourne.txt -> .git/annex/objects/QK/2p/SHA256E-s1031--d45a0f0aba935126bf29d6265e9c252faae8b735abdc20edbec69eb3cc9edcd7.txt/SHA256E-s1031--d45a0f0aba935126bf29d6265e9c252faae8b735abdc20edbec69eb3cc9edcd7.txt
+lrwx------  1 ewen  staff  200 23 Apr 18:53 chasejarvis-1.m4a -> .git/annex/objects/M1/8z/SHA256E-s80616681--d1b63892a9d68c51ddef9eb5c0a56ced795a449afe13d4884663ca6ecc76cade.m4a/SHA256E-s80616681--d1b63892a9d68c51ddef9eb5c0a56ced795a449afe13d4884663ca6ecc76cade.m4a
+lrwx------  1 ewen  staff  194 23 Apr 18:53 chasejarvis.txt -> .git/annex/objects/35/jv/SHA256E-s50443--687b0cf63cc8f5724e3a694e1de6221ca4d1dd3449a6e6ad9b6f34501bfa30ce.txt/SHA256E-s50443--687b0cf63cc8f5724e3a694e1de6221ca4d1dd3449a6e6ad9b6f34501bfa30ce.txt
+lrwx------  1 ewen  staff  184 23 Apr 18:53 checksums -> .git/annex/objects/3Q/vv/SHA256E-s1672--31b3cb7ccbcafc96d5061775dc31aa974bb4ce2f16984b2ab0104eb66674d81b/SHA256E-s1672--31b3cb7ccbcafc96d5061775dc31aa974bb4ce2f16984b2ab0104eb66674d81b
+lrwx------  1 ewen  staff  200 23 Apr 18:53 chrismarquardt-1.m4a -> .git/annex/objects/k4/f3/SHA256E-s61861112--c14bacb5adfe3eca237547760183a986499808084dd1925a0202598981e4406b.m4a/SHA256E-s61861112--c14bacb5adfe3eca237547760183a986499808084dd1925a0202598981e4406b.m4a
+ewen@ashram:/nas01/annex/purchased/mediabytes$ 
+ewen@ashram:/nas01/annex/purchased/mediabytes$ git annex info
+repository mode: direct
+trusted repositories: 0
+semitrusted repositories: 13
+	00000000-0000-0000-0000-000000000001 -- web
+ 	00000000-0000-0000-0000-000000000002 -- bittorrent
+ 	00420eb2-1c9a-47e8-abfa-675b9ffb0782 -- naos_wd_1 backup drive
+ 	0ac1326e-9e03-4c1e-bd51-6cd664988caa -- Naos (colo) fileserver
+ 	0ba93724-85a8-4ac7-855b-e656a6abaf09 -- ashram (laptop) internal drive [ashram]
+ 	0e5fd428-ba5b-4817-8489-9780bc62e655 -- naos_wd_2 backup drive
+ 	1144edcc-b18f-4cb1-a828-6c272d7788d6 -- naos_wd_3 backup drive
+ 	4e5bfc02-1adc-44d2-a60a-885d9120ed5d -- nas01 file server [here]
+ 	6142cfd4-83ba-410f-bf54-fd4b79e49fdb -- ashram external data drive [ashram-data]
+ 	77aeca6a-a4f8-4734-befc-2a0b50687c52 -- tv home fileserver
+ 	bde87f8e-4740-411f-b068-3dbbd9db03d4 -- nas01 file server
+ 	cc1c45f3-7c96-4103-b9c0-cce0f5f2cc36 -- bethel_data_drive
+ 	eb0ca2b6-e8c6-43b0-a805-de17a4d7eeae -- naos_wd_4 backup drive
+untrusted repositories: 0
+transfers in progress: none
+available local disk space: 3.63 terabytes (+1 megabyte reserved)
+local annex keys: 0
+local annex size: 0 bytes
+annexed files in working tree: 59
+size of annexed files in working tree: 1.67 gigabytes
+bloom filter size: 32 mebibytes (0% full)
+backend usage: 
+	SHA256E: 59
+ewen@ashram:/nas01/annex/purchased/mediabytes$ 

(Diff truncated)
diff --git a/doc/forum/Lots_of_4k_symlinks.mdwn b/doc/forum/Lots_of_4k_symlinks.mdwn
index c9113d9..7f90f87 100644
--- a/doc/forum/Lots_of_4k_symlinks.mdwn
+++ b/doc/forum/Lots_of_4k_symlinks.mdwn
@@ -1,6 +1,6 @@
 Hi, 
 
-this is a minor issue and probably there is no better solution, but nevertheless I would like to point out it and maybe discuss a little about the issue.
+this is a minor issue and probably there is no better solution, but nevertheless I would like to point it out and maybe discuss a little about the issue.
 
 Given that the symlinks generated by annex are pretty large in size (they point to a file named by a large hash number), ext4 is using an entire block (4K) of storage instead of [embedding the symlink into the inode][inode] itself. For the "archivist use case" of annex, this might lead to tens or hundreds of MBs of disk occupied by symlinks which actually don't add up to more than a few MBs.
 

Added a comment: Why is it takins too long?
diff --git a/doc/forum/Synchronize_large_files___40__VM_images__41__/comment_2_bbd98d0b5d77dc7efc55ef8c2a18d612._comment b/doc/forum/Synchronize_large_files___40__VM_images__41__/comment_2_bbd98d0b5d77dc7efc55ef8c2a18d612._comment
new file mode 100644
index 0000000..bc04122
--- /dev/null
+++ b/doc/forum/Synchronize_large_files___40__VM_images__41__/comment_2_bbd98d0b5d77dc7efc55ef8c2a18d612._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="https://me.yahoo.com/a/hVbIabkhqO11.DpKUWBoztFSLD5q#8cbe8"
+ nickname="Murat"
+ avatar="http://cdn.libravatar.org/avatar/52d95e40aca820c1993077ef9aa676c75700a072511c143f6db6b78be6b1b212"
+ subject="Why is it takins too long?"
+ date="2017-04-18T16:26:06Z"
+ content="""
+Hi,
+due to my requirement I need to revert vm image every time before running it via \"git reset --hard\" which is really fast on the other hand \"git annex unlock\" takes really long, I run git-annex on Centos 6 and git-annex version git-annex-3.20120522-2.1.el6.x86_64, if I update git-annex version can it help to fasten \"unlock\"?
+
+thanks a lot
+
+"""]]

diff --git a/doc/forum/Lots_of_4k_symlinks.mdwn b/doc/forum/Lots_of_4k_symlinks.mdwn
new file mode 100644
index 0000000..c9113d9
--- /dev/null
+++ b/doc/forum/Lots_of_4k_symlinks.mdwn
@@ -0,0 +1,24 @@
+Hi, 
+
+this is a minor issue and probably there is no better solution, but nevertheless I would like to point out it and maybe discuss a little about the issue.
+
+Given that the symlinks generated by annex are pretty large in size (they point to a file named by a large hash number), ext4 is using an entire block (4K) of storage instead of [embedding the symlink into the inode][inode] itself. For the "archivist use case" of annex, this might lead to tens or hundreds of MBs of disk occupied by symlinks which actually don't add up to more than a few MBs.
+
+Here is a real world example:
+
+```
+(ins)carlos@carlos home$ du -hs music/
+56M	music/
+(ins)carlos@carlos home$ du -bhs music/
+3.3M	music/
+(ins)carlos@carlos home$ ln -s /tmp/x x
+(ins)carlos@carlos home$ du x
+0	x
+(ins)carlos@carlos home$ ln -s /tmp/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xx
+(ins)carlos@carlos home$ du xx
+4	xx
+```
+
+Cheers, Carlos
+
+[inode]: https://kernelnewbies.org/Linux_3.8#head-372b38979138cf2006bd0114ae97f889f67ef46a

Added a comment
diff --git a/doc/forum/Git_repos_in_git_annex__63__/comment_3_88e37911a0280d817559b2d51ddb1d4e._comment b/doc/forum/Git_repos_in_git_annex__63__/comment_3_88e37911a0280d817559b2d51ddb1d4e._comment
new file mode 100644
index 0000000..efd86d8
--- /dev/null
+++ b/doc/forum/Git_repos_in_git_annex__63__/comment_3_88e37911a0280d817559b2d51ddb1d4e._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="memeplex"
+ avatar="http://cdn.libravatar.org/avatar/84a611000e819ef825421de06c9bca90"
+ subject="comment 3"
+ date="2017-04-14T22:35:49Z"
+ content="""
+Another use case is when you use annex for backup. For example I keep most of my files (dotfiles, scripts, music, books, etc) in a \"home\" git annex repo. Part of them is stored in google drive, part in a pendrive, part in my home computer, part in my work computer. Every once in a while I sync everything into my home computer and into an additional external hard drive remote that I keep as a backup. Sadly, my archived git projects can't be managed like that. Nevertheless, it's not a big deal since they are already in the cloud (gitlab or github) and in my local filesystem. Besides, since they are archived, I can just create tarballs and add them to the annex (and in case annex allowed to store .git directories, I'd not be really comfortable with the huge amount of symlinks that would produce).
+
+That said, it's kind of a bathos that the illusion of a distributed, decentralized, redundant, versioned, annotated, etc. filesystem over git is broken by git itself.   
+"""]]

diff --git a/doc/tips/Faster_bash_autocompletion_with_big_annex_repos.mdwn b/doc/tips/Faster_bash_autocompletion_with_big_annex_repos.mdwn
index ded86df..61d4919 100644
--- a/doc/tips/Faster_bash_autocompletion_with_big_annex_repos.mdwn
+++ b/doc/tips/Faster_bash_autocompletion_with_big_annex_repos.mdwn
@@ -25,26 +25,26 @@ __git_index_files ()
 
 time __git_index_files > /dev/null
 
+real    0m0.830s
+user    0m0.597s
+sys    0m0.310s
 
 __git_index_files ()
 {
     local dir="$(__gitdir)" root="${2-.}" file;
     if [ -d "$dir" ]; then
         __git_ls_files_helper "$root" "$1" | \
-            sed -r 's@^"?([^/]+)/.*$@\1@' | sort | uniq
+            sed -r 's@/.*@@' | uniq | sort | uniq
     fi
 }
 
+
 time __git_index_files > /dev/null
 
-real    0m0.830s
-user    0m0.597s
-sys    0m0.310s
+real    0m0.075s
+user    0m0.083s
+sys    0m0.010s
 
-real    0m0.345s
-user    0m0.357s
-sys    0m0.000s
 ```
 
-So you might redefine `__git_index_files` as above in your .bashrc after sourcing the git autocomplete script. 
-
+10 times faster! So you might redefine `__git_index_files` as above in your .bashrc after sourcing the git autocomplete script. 

diff --git a/doc/tips/Faster_bash_autocompletion_with_big_annex_repos.mdwn b/doc/tips/Faster_bash_autocompletion_with_big_annex_repos.mdwn
new file mode 100644
index 0000000..ded86df
--- /dev/null
+++ b/doc/tips/Faster_bash_autocompletion_with_big_annex_repos.mdwn
@@ -0,0 +1,50 @@
+I'm currently using git annex to manage my entire file collection
+(including tons of music and books) and I noticed how slow
+autocompletion has become for files in the index (say for git add).
+The main offender is a while-read-case-echo bash loop in
+`__git_index_files` that can be readily substituted with a much faster
+sed invocation. Here is my benchmark:
+
+```
+__git_index_files ()
+{
+    local dir="$(__gitdir)" root="${2-.}" file;
+    if [ -d "$dir" ]; then
+        __git_ls_files_helper "$root" "$1" | while read -r file; do
+            case "$file" in
+                ?*/*)
+                    echo "${file%%/*}"
+                ;;
+                *)
+                    echo "$file"
+                ;;
+            esac;
+        done | sort | uniq;
+    fi
+}
+
+time __git_index_files > /dev/null
+
+
+__git_index_files ()
+{
+    local dir="$(__gitdir)" root="${2-.}" file;
+    if [ -d "$dir" ]; then
+        __git_ls_files_helper "$root" "$1" | \
+            sed -r 's@^"?([^/]+)/.*$@\1@' | sort | uniq
+    fi
+}
+
+time __git_index_files > /dev/null
+
+real    0m0.830s
+user    0m0.597s
+sys    0m0.310s
+
+real    0m0.345s
+user    0m0.357s
+sys    0m0.000s
+```
+
+So you might redefine `__git_index_files` as above in your .bashrc after sourcing the git autocomplete script. 
+

diff --git a/doc/forum/deploy.mdwn b/doc/forum/deploy.mdwn
new file mode 100644
index 0000000..b9e0974
--- /dev/null
+++ b/doc/forum/deploy.mdwn
@@ -0,0 +1,5 @@
+Greetings,
+
+I use the push-to-deploy pattern (as described in 4.1 http://gitolite.com/deploy.html). However, my git repo has large binary files that I'd like to annex. Is there an example of using git annex with a bare remote repository with the appropriate post-receive hook to accomplish the deploy?
+
+Thanks

diff --git a/doc/forum/removing_information_from_special_remote_file.mdwn b/doc/forum/removing_information_from_special_remote_file.mdwn
new file mode 100644
index 0000000..de740ee
--- /dev/null
+++ b/doc/forum/removing_information_from_special_remote_file.mdwn
@@ -0,0 +1,8 @@
+I'm working on a project using git-annex and Tahoe as a special remote. I want to encrypt that Tahoe capability for various file(s) (the capability is present in the git-annex branch), so that it is impossible for a malicious actor to access a file. However, due to the union-merge that happens on that branch, no matter what I try, the plain text capability keeps coming back.
+
+How can I either remove an entry from a file in the git-annex branch, or maybe remove all history for a file so that it has nothing to merge against, or otherwise remove the plain-text capability?
+
+This is very similar/same as https://git-annex.branchable.com/forum/removing_remote.log_information_completely/ but hopefully I can get a updated perspective, or any further guidance. Any help at all would be greatly appreciated.
+
+If there's a simple way to modify the git-annex source code to accommodate this, I would be perfectly happy with that solution. Unfortunately I am not well versed in Haskell, but if there was a way to say, for example, ignore *.rmt files when doing the union-merge, that would probably work. I'd be happy with any solution though.
+

Added a comment
diff --git a/doc/forum/whereis__58___Why_not_here__63__/comment_1_02a423918efcc760bb89dfb78586dc2a._comment b/doc/forum/whereis__58___Why_not_here__63__/comment_1_02a423918efcc760bb89dfb78586dc2a._comment
new file mode 100644
index 0000000..4f032f4
--- /dev/null
+++ b/doc/forum/whereis__58___Why_not_here__63__/comment_1_02a423918efcc760bb89dfb78586dc2a._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="Horus"
+ avatar="http://cdn.libravatar.org/avatar/8f0ee08b98ea5bba76c3fe112c08851c"
+ subject="comment 1"
+ date="2017-04-12T21:21:08Z"
+ content="""
+It's strange. I made a change to the file on Horus, it gets propagated to Marduk, but Marduk does not know about. whereis still shows the file is only at Horus. At Asaru the symlink is deleted (so the change info was propagated), but instead there is a symlink on invalid content (like in indirect mode)
+
+The connections are all over SSH: Horus -> Marduk <- Asaru
+
+Sorry, I'm utterly confused...
+"""]]

diff --git a/doc/forum/whereis__58___Why_not_here__63__.mdwn b/doc/forum/whereis__58___Why_not_here__63__.mdwn
new file mode 100644
index 0000000..fb2a325
--- /dev/null
+++ b/doc/forum/whereis__58___Why_not_here__63__.mdwn
@@ -0,0 +1,33 @@
+Hey,
+
+I have three repos, all are run by git-annex assistent. Files are synced just fine, but there is one oddity:
+
+```
+florian@horus ~/.synced_configuration (git)-[annex/direct/master] % git annex whereis .zshenv 
+whereis .zshenv (3 copies) 
+        69902190-d4c1-4abb-a000-f0712769f1cd -- Horus [here]
+        c7fc42ca-49f7-4688-b81d-c9cfd4a42564 -- Asaru
+        d5b0586f-32dd-4aae-a7a2-b23d9edd0c32 -- Marduk [marduk]
+ok
+florian@horus ~/.synced_configuration (git)-[annex/direct/master] % git annex whereis .emacs 
+whereis .emacs (2 copies) 
+        c7fc42ca-49f7-4688-b81d-c9cfd4a42564 -- Asaru
+        d5b0586f-32dd-4aae-a7a2-b23d9edd0c32 -- Marduk [marduk]
+ok
+```
+for `.zshenv` everything is fine, but why is `.emacs` not in Horus? Changes from and to Horus are propagated:
+
+```
+florian@horus ~/.synced_configuration (git)-[annex/direct/master] % git annex log .emacs
+- Wed, 12 Apr 2017 22:48:03 CEST .emacs | 69902190-d4c1-4abb-a000-f0712769f1cd -- Horus
++ Wed, 12 Apr 2017 22:02:55 CEST .emacs | 69902190-d4c1-4abb-a000-f0712769f1cd -- Horus
++ Wed, 12 Apr 2017 22:02:55 CEST .emacs | d5b0586f-32dd-4aae-a7a2-b23d9edd0c32 -- Marduk [marduk]
+- Wed, 12 Apr 2017 22:02:54 CEST .emacs | 69902190-d4c1-4abb-a000-f0712769f1cd -- Horus
+- Wed, 12 Apr 2017 22:02:53 CEST .emacs | d5b0586f-32dd-4aae-a7a2-b23d9edd0c32 -- Marduk [marduk]
++ Wed, 12 Apr 2017 22:00:40 CEST .emacs | d5b0586f-32dd-4aae-a7a2-b23d9edd0c32 -- Marduk [marduk]
++ Wed, 12 Apr 2017 22:00:36 CEST .emacs | 69902190-d4c1-4abb-a000-f0712769f1cd -- Horus
+```
+
+Anyone any idea what could be wrong there?
+
+Thanks!

Added a comment: sounds like the dumb backend, except not dumb
diff --git a/doc/todo/export/comment_1_cb87f7518da252b950d70c60352e848e._comment b/doc/todo/export/comment_1_cb87f7518da252b950d70c60352e848e._comment
new file mode 100644
index 0000000..9158d12
--- /dev/null
+++ b/doc/todo/export/comment_1_cb87f7518da252b950d70c60352e848e._comment
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ username="anarcat"
+ avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
+ subject="sounds like the dumb backend, except not dumb"
+ date="2017-04-08T20:21:41Z"
+ content="""
+This sounds a lot like what i was trying to do in [[todo/dumb, unsafe,
+human-readable_backend]], except done properly. :)
+
+I was wondering about that asymmetry recentrly, and it would seem like
+a good idea to fix this. the `--to remote` flag could especially be
+useful for directory, rsync or even webdav remotes. i am not sure how
+this could be implemented, but i would certainly use this.
+
+having addurl work would be an awesome bonus, especially with webdav,
+where the mapping can be easily guessed (like S3). could be trickier
+with rsync/directory because then the user would need to tell
+git-annex what the base url is somewhere, but would fix a ton of use
+cases i had for this, like [[forum/original filename on s3]],
+[[forum/Facilitate public pretty S3 URLs]], [[todo/hide_missing_files]]
+and, my favorite, [[forum/syncing music to my android device]].
+it would also automate, extend and simplify the
+[[tips/publishing_your_files_to_the_public/]] use case.
+
+so thanks for this effort, i think it's a great idea and i'm excited
+to test this! -- [[anarcat]]
+
+"""]]

update
diff --git a/doc/thanks/list b/doc/thanks/list
index 9151ccf..f38bfa2 100644
--- a/doc/thanks/list
+++ b/doc/thanks/list
@@ -65,3 +65,5 @@ Chris Lamb,
 Anton Grensjö, 
 Stan Yamane, 
 John Peloquin, 
+Thomas Schwinge, 
+Lars Wallenborn, 

Added a comment
diff --git a/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_5_b094509fe0194313666b5b1db0a68156._comment b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_5_b094509fe0194313666b5b1db0a68156._comment
new file mode 100644
index 0000000..4d904ca
--- /dev/null
+++ b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_5_b094509fe0194313666b5b1db0a68156._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 5"
+ date="2017-04-08T03:16:48Z"
+ content="""
+> How do you check if ssh has established a cached ssh connection?
+
+ssh -O check  -- somewhat of an additional overhead, but possible
+
+> GIT_SSH_COMMAND is used for every call to ssh in git-annex.
+
+so then theoretically we could implement \"may be ...\" strategy on our end in our sshrun.
+"""]]

version: Added "dependency versions" line.
This commit was sponsored by Anthony DeRobertis on Patreon.
diff --git a/Assistant/WebApp/Documentation.hs b/Assistant/WebApp/Documentation.hs
index cdb5d40..064f5db 100644
--- a/Assistant/WebApp/Documentation.hs
+++ b/Assistant/WebApp/Documentation.hs
@@ -12,7 +12,7 @@ module Assistant.WebApp.Documentation where
 import Assistant.WebApp.Common
 import Assistant.Install (standaloneAppBase)
 import Build.SysConfig (packageversion)
-import BuildFlags
+import BuildInfo
 
 {- The full license info may be included in a file on disk that can
  - be read in and displayed. -}
diff --git a/BuildFlags.hs b/BuildFlags.hs
deleted file mode 100644
index 68dfabb..0000000
--- a/BuildFlags.hs
+++ /dev/null
@@ -1,81 +0,0 @@
-{- git-annex build flags reporting
- -
- - Copyright 2013 Joey Hess <id@joeyh.name>
- -
- - Licensed under the GNU GPL version 3 or higher.
- -}
-
-{-# LANGUAGE CPP #-}
-
-module BuildFlags where
-
-buildFlags :: [String]
-buildFlags = filter (not . null)
-	[ ""
-#ifdef WITH_ASSISTANT
-	, "Assistant"
-#else
-#warning Building without the assistant.
-#endif
-#ifdef WITH_WEBAPP
-	, "Webapp"
-#else
-#warning Building without the webapp. You probably need to install Yesod..
-#endif
-#ifdef WITH_PAIRING
-	, "Pairing"
-#else
-#warning Building without local pairing.
-#endif
-#ifdef WITH_TESTSUITE
-	, "Testsuite"
-#else
-#warning Building without the testsuite.
-#endif
-#ifdef WITH_S3
-	, "S3"
-#if MIN_VERSION_aws(0,10,6)
-		++ "(multipartupload)"
-#endif
-#if MIN_VERSION_aws(0,13,0)
-		++ "(storageclasses)"
-#endif
-#else
-#warning Building without S3.
-#endif
-#ifdef WITH_WEBDAV
-	, "WebDAV"
-#else
-#warning Building without WebDAV.
-#endif
-#ifdef WITH_INOTIFY
-	, "Inotify"
-#endif
-#ifdef WITH_FSEVENTS
-	, "FsEvents"
-#endif
-#ifdef WITH_KQUEUE
-	, "Kqueue"
-#endif
-#ifdef WITH_DBUS
-	, "DBus"
-#endif
-#ifdef WITH_DESKTOP_NOTIFY
-	, "DesktopNotify"
-#endif
-#ifdef WITH_CONCURRENTOUTPUT
-	, "ConcurrentOutput"
-#else
-#warning Building without ConcurrentOutput
-#endif
-#ifdef WITH_TORRENTPARSER
-	, "TorrentParser"
-#endif
-#ifdef WITH_MAGICMIME
-	, "MagicMime"
-#endif
-	-- Always enabled now, but users may be used to seeing these flags
-	-- listed.
-	, "Feeds"
-	, "Quvi"
-	]
diff --git a/BuildInfo.hs b/BuildInfo.hs
new file mode 100644
index 0000000..29455f6
--- /dev/null
+++ b/BuildInfo.hs
@@ -0,0 +1,113 @@
+{- git-annex build info reporting
+ -
+ - Copyright 2013-2017 Joey Hess <id@joeyh.name>
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+{-# LANGUAGE CPP #-}
+
+module BuildInfo where
+
+import Data.List
+import qualified Data.CaseInsensitive as CI
+
+buildFlags :: [String]
+buildFlags = filter (not . null)
+	[ ""
+#ifdef WITH_ASSISTANT
+	, "Assistant"
+#else
+#warning Building without the assistant.
+#endif
+#ifdef WITH_WEBAPP
+	, "Webapp"
+#else
+#warning Building without the webapp. You probably need to install Yesod..
+#endif
+#ifdef WITH_PAIRING
+	, "Pairing"
+#else
+#warning Building without local pairing.
+#endif
+#ifdef WITH_TESTSUITE
+	, "Testsuite"
+#else
+#warning Building without the testsuite.
+#endif
+#ifdef WITH_S3
+	, "S3"
+#if MIN_VERSION_aws(0,10,6)
+		++ "(multipartupload)"
+#endif
+#if MIN_VERSION_aws(0,13,0)
+		++ "(storageclasses)"
+#endif
+#else
+#warning Building without S3.
+#endif
+#ifdef WITH_WEBDAV
+	, "WebDAV"
+#else
+#warning Building without WebDAV.
+#endif
+#ifdef WITH_INOTIFY
+	, "Inotify"
+#endif
+#ifdef WITH_FSEVENTS
+	, "FsEvents"
+#endif
+#ifdef WITH_KQUEUE
+	, "Kqueue"
+#endif
+#ifdef WITH_DBUS
+	, "DBus"
+#endif
+#ifdef WITH_DESKTOP_NOTIFY
+	, "DesktopNotify"
+#endif
+#ifdef WITH_CONCURRENTOUTPUT
+	, "ConcurrentOutput"
+#else
+#warning Building without ConcurrentOutput
+#endif
+#ifdef WITH_TORRENTPARSER
+	, "TorrentParser"
+#endif
+#ifdef WITH_MAGICMIME
+	, "MagicMime"
+#endif
+	-- Always enabled now, but users may be used to seeing these flags
+	-- listed.
+	, "Feeds"
+	, "Quvi"
+	]
+
+-- Not a complete list, let alone a listing transitive deps, but only
+-- the ones that are often interesting to know.
+dependencyVersions :: [String]
+dependencyVersions = map fmt $ sortOn (CI.mk . fst)
+	[ ("feed", VERSION_feed)
+	, ("uuid", VERSION_uuid)
+	, ("bloomfilter", VERSION_bloomfilter)
+	, ("http-client", VERSION_http_client)
+	, ("persistent-sqlite", VERSION_persistent_sqlite)

(Diff truncated)
add todo for lib versions
diff --git a/doc/todo/Display_the_version_of_a_library_corresponding_to_a_build_flag.mdwn b/doc/todo/Display_the_version_of_a_library_corresponding_to_a_build_flag.mdwn
new file mode 100644
index 0000000..eebb529
--- /dev/null
+++ b/doc/todo/Display_the_version_of_a_library_corresponding_to_a_build_flag.mdwn
@@ -0,0 +1,17 @@
+It would be great when diagnosing issues to know the version of a particular library that git-annex is compiled with.
+
+Because there are so many dependencies though, perhaps only the libraries corresponding to a build flag should be displayed, so instead of
+
+    ~ λ git annex version
+    git-annex version: 6.20170321-g4642912
+    build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify ConcurrentOutput TorrentParser Feeds Quvi
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+
+It would show:
+
+    ~ λ git annex version
+    git-annex version: 6.20170321-g4642912
+    build flags: ...etc... TorrentParser-1.2.1 Feeds-2.3.1 Quvi-1.0.0
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external

Added a comment
diff --git a/doc/forum/Trouble_setting_up_public_repo_cloneable_from_a_web_site/comment_2_409103144cf803ced9d81d3380ca76fb._comment b/doc/forum/Trouble_setting_up_public_repo_cloneable_from_a_web_site/comment_2_409103144cf803ced9d81d3380ca76fb._comment
new file mode 100644
index 0000000..bd2e5fb
--- /dev/null
+++ b/doc/forum/Trouble_setting_up_public_repo_cloneable_from_a_web_site/comment_2_409103144cf803ced9d81d3380ca76fb._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="lee@7614f42c1a6cc84dbc813df25d2f75ed54948e17"
+ nickname="lee"
+ avatar="http://cdn.libravatar.org/avatar/bcacd00a7f05c4772329cf9f446c7987"
+ subject="comment 2"
+ date="2017-04-07T21:10:34Z"
+ content="""
+> I wonder if you forgot to run git update-server-info?
+
+That was it! Thanks!
+
+I actually didn't forget to run this, but I did not realize that it needed to be run on the *server*, so I was running it on my local annex instead. Now that I've run it everything works!
+"""]]

response
diff --git a/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_3_daac1d424bc9e5b56772fa49707bc5a5._comment b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_3_daac1d424bc9e5b56772fa49707bc5a5._comment
new file mode 100644
index 0000000..26f1aed
--- /dev/null
+++ b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_3_daac1d424bc9e5b56772fa49707bc5a5._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2017-04-07T21:06:51Z"
+ content="""
+How do you check if ssh has established a cached ssh connection?
+"""]]
diff --git a/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_4_05bf20db275b911e2d89311182f289f6._comment b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_4_05bf20db275b911e2d89311182f289f6._comment
new file mode 100644
index 0000000..27d39a7
--- /dev/null
+++ b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_4_05bf20db275b911e2d89311182f289f6._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2017-04-07T21:07:44Z"
+ content="""
+`GIT_SSH_COMMAND` is used for *every* call to ssh in git-annex.
+"""]]

devblog
diff --git a/doc/devblog/day_456__digging_in.mdwn b/doc/devblog/day_456__digging_in.mdwn
new file mode 100644
index 0000000..024e0b9
--- /dev/null
+++ b/doc/devblog/day_456__digging_in.mdwn
@@ -0,0 +1,16 @@
+Digging in to some of the meatier backlog today. Backlog down to 225.
+
+A lot of fixes around using `git annex enableremote` to add new gpg keys to
+a gcrypt special remote. 
+
+Had to make git-annex's use of `GIT_SSH`/`GIT_SSH_COMMAND`
+contingent on `GIT_ANNEX_USE_GIT_SSH=1` being set. Unfortunate, but
+difference from git made at least one existing use of that environment
+variable break, and so it will need to be whitelisted in places where
+git-annex should use it.
+
+Added support for `git annex add --update`
+
+----
+
+Today's work was sponsored by Trenton Cronholm on Patreon.

Added a comment: may be?
diff --git a/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_2_c877de08f959dee4ace34e66f42c8615._comment b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_2_c877de08f959dee4ace34e66f42c8615._comment
new file mode 100644
index 0000000..c60ba6d
--- /dev/null
+++ b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_2_c877de08f959dee4ace34e66f42c8615._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="may be?"
+ date="2017-04-07T21:01:56Z"
+ content="""
+well, it kinda depends at either at which level parallelization is happening or how parallel jobs handling is done, or may be ...
+
+level of parallelization:
+I guess ATM annex just parallelizes at the level of \"get --key KEY\" jobs.
+But if central process decided to try to \"get --from=remote --key KEY\" -- call which it submits to parallel work pull -- then it could first check if remote is an ssh remote and connection caching is established, and if not -- establish it and then submit this and/or any subsequent get call.
+This would though over-complicate the design I guess considerably, so probably shouldn't be approached.
+
+jobs handling:
+if parallel jobs could 'yield' back to the original process (e.g. if there was some protocoled exchange between them and master process... somewhat similar to git annex special remotes in a way) demanding some action (e.g. - authenticate me to the host) and then proceed back with its dues, could work out I guess.
+But I guess that is also not current implementation
+
+
+may be...:
+since I guess (didn't check) GIT_SSH_COMMAND is used (or not yet but could be?) for ssh transfers, such activity as establishing shared ssh connection could be deferred to it (with some proper locking/waiting for parallel invocations)... or am I wrong?
+"""]]

remove stealth on-topic spam
diff --git a/doc/forum/recover_deleted_files___63__/comment_5_29ec08578bc45e4bbdecf76d1eb33826._comment b/doc/forum/recover_deleted_files___63__/comment_5_29ec08578bc45e4bbdecf76d1eb33826._comment
deleted file mode 100644
index a59b079..0000000
--- a/doc/forum/recover_deleted_files___63__/comment_5_29ec08578bc45e4bbdecf76d1eb33826._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="veron_veron@8e19f168a8da3dabcdbf28ccd3f27edfb40941ed"
- nickname="veron_veron"
- avatar="http://cdn.libravatar.org/avatar/eb7805af696010b4e8844aefeeb89a1b"
- subject="easy"
- date="2016-12-20T09:19:31Z"
- content="""
-Indeed the recycle bin, and using Windows Explorer to look in folders; the programs you used are not designed to search for files; especially programs like Elements, which uses a catalog is not what you need now. Moving files confuses PS Elements organiser enough to make it seem like you lost files, while they're actually there. When things go wrong, use the basic tools provided in Windows itself (explorer, search).
-For file recovery programs, I've use <a href=\"http://www.passwordmanagers.net/resources/Software-for-Recovering-Deleted-Files-71.html\">Software for Recovering Deleted Files</a> on a few occassions; it's free and it works as good as can be expected. The main thing is to stop using the external hard disk until you use this tool, and avoid writing any files to it.
-"""]]
diff --git a/doc/forum/recover_deleted_files___63__/comment_6_b6614dddceba94c94179572a8f59ee80._comment b/doc/forum/recover_deleted_files___63__/comment_6_b6614dddceba94c94179572a8f59ee80._comment
deleted file mode 100644
index 0806fc4..0000000
--- a/doc/forum/recover_deleted_files___63__/comment_6_b6614dddceba94c94179572a8f59ee80._comment
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- username="hobbie123"
- avatar="http://cdn.libravatar.org/avatar/757597b9cb81748641d047835128e96c"
- subject="windows password "
- date="2017-03-13T06:28:14Z"
- content="""
-<a href=\"http://www.passwordmanagers.net/resources/Necessary-Softwares-for-Work-and-Life-6.html\">Password Manager</a>
-is a free manager for XP and windows password issues and it works as good as can be expected. In addition,the main thing is to stop using the external hard disk until you use this tool, and avoid writing any files to it.
-"""]]

response
diff --git a/doc/forum/special_remote_which_applies_a_lossless_operation_in__47__out/comment_1_195f52aaaa36ce20955f2eb40af3c79a._comment b/doc/forum/special_remote_which_applies_a_lossless_operation_in__47__out/comment_1_195f52aaaa36ce20955f2eb40af3c79a._comment
new file mode 100644
index 0000000..ea497a5
--- /dev/null
+++ b/doc/forum/special_remote_which_applies_a_lossless_operation_in__47__out/comment_1_195f52aaaa36ce20955f2eb40af3c79a._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-07T20:34:38Z"
+ content="""
+Of course that will work fine, as long as the operation is reversable.
+
+Plenty of existing special remotes
+compress/encrypt/slice/dice/mangle/obfuscate/whatever the content of files
+stored in them, and undo it all when the contents are retreived.
+"""]]

comment
diff --git a/doc/forum/Trouble_setting_up_public_repo_cloneable_from_a_web_site/comment_1_34a858688c46faf624fa7d98b05e6519._comment b/doc/forum/Trouble_setting_up_public_repo_cloneable_from_a_web_site/comment_1_34a858688c46faf624fa7d98b05e6519._comment
new file mode 100644
index 0000000..e42a45a
--- /dev/null
+++ b/doc/forum/Trouble_setting_up_public_repo_cloneable_from_a_web_site/comment_1_34a858688c46faf624fa7d98b05e6519._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-07T20:31:26Z"
+ content="""
+Hmm, that all looks ok. I wonder if you forgot to run `git
+update-server-info`?
+
+<https://downloads.kitenet.net/.git/> is an example of such a repo, which
+you can clone. So it certianly works if you set it up right.
+"""]]

comment
diff --git a/doc/forum/Debugging_of_transfer_considerations/comment_3_508b5b4155f0f9c0c2a7d2e990a1bf92._comment b/doc/forum/Debugging_of_transfer_considerations/comment_3_508b5b4155f0f9c0c2a7d2e990a1bf92._comment
new file mode 100644
index 0000000..99a8be0
--- /dev/null
+++ b/doc/forum/Debugging_of_transfer_considerations/comment_3_508b5b4155f0f9c0c2a7d2e990a1bf92._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2017-04-07T20:26:54Z"
+ content="""
+Yes, files in archive/ directories are only dropped from clients once they
+reach an archive. Backup repositories are not considered to be archives.
+
+Of course, you can tweak the preferred content expressions to change this
+behavior.
+
+I don't know if the arhive vs backup distinction makes sense really but
+I have heard of some users doing things that depend on it.
+"""]]

comment
diff --git a/doc/bugs/annex_doesn__39__t_fixup_symlinks_when___34__git_commit_path__95__to__95__repo__34___is_used/comment_1_b90f22ce0ab931658856e949ac227985._comment b/doc/bugs/annex_doesn__39__t_fixup_symlinks_when___34__git_commit_path__95__to__95__repo__34___is_used/comment_1_b90f22ce0ab931658856e949ac227985._comment
new file mode 100644
index 0000000..e5a1ac2
--- /dev/null
+++ b/doc/bugs/annex_doesn__39__t_fixup_symlinks_when___34__git_commit_path__95__to__95__repo__34___is_used/comment_1_b90f22ce0ab931658856e949ac227985._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-07T20:16:45Z"
+ content="""
+There's nothing special about using the absolute path; "git commit ." 
+or "git commit thefile" will behave the same.
+
+This is the git false index problem. Since git commit in these situations
+runs the pre-commit hook with a false index file, changes made to that
+index file won't be visible after the commit.
+
+So, if `git annex pre-commit` fixes symlinks in this situation,
+the right thing will be committed, but then the old index will have the old
+symlinks staged, which will result in `git status` after the commit showing
+modification to the files you just staged and committed!
+
+Short of having a post-commit hook come along and fix up the index to match
+what was committed, I don't see anything git-annex can do better. Well, it
+could prevent such commits even being made, I suppose, or warn.
+
+It's a pity git uses this false index file in this situation.
+"""]]

comment
diff --git a/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_1_236d2b897a550e7db4b266814d4e778d._comment b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_1_236d2b897a550e7db4b266814d4e778d._comment
new file mode 100644
index 0000000..40e7ee2
--- /dev/null
+++ b/doc/bugs/get_-J_cannot_be_used_with_password-based_authentication/comment_1_236d2b897a550e7db4b266814d4e778d._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-07T19:58:41Z"
+ content="""
+Well let's see.. To fix this would need some way for ssh to outsource its
+password prompting to another program, which could then serialize
+concurrent password requests, and perhaps reuse the same password when
+reconnecting to the same host.
+
+Sounds an aweful lot like ssh-agent, doesn't it?
+
+Now, it does happen to be the case that without -J, the password is only
+prompted for once to download multiple files from the same host. That works
+because of ssh connection caching. But in the -J case, the
+connection caching does not help, because multiple sshed are started before
+there's a connection to reuse, so each tries to make a new connection and
+prompts.
+
+Even if connection caching worked with -J, the general problem would remain
+when it did concurrent downloads from different hosts.
+
+So I tend to feel that this is just not fixable; if the user wants to use
+-J, they ought to use ssh-agent so it doesn't prompt for passwords.
+"""]]

git annex add -u now supported, analagous to git add -u
Unlike git add -u, git annex add -u does not update the index for files
removed from the working tree. But then, "git add ." stages removals,
and "git annex add ." does not, so that's an existing divergence.
Seems that --update --batch would need to run git ls-files once per line of
batch input, which would surely be too slow, so just throw an error for
that.
This commit was supported by the NSF-funded DataLad project.
diff --git a/CHANGELOG b/CHANGELOG
index c53b01c..d988a96 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -19,6 +19,7 @@ git-annex (6.20170322) UNRELEASED; urgency=medium
     ignored, a reversion introduced in 6.20160527.
   * gcrypt: Support re-enabling to change eg, encryption parameters.
     This was never supported before.
+  * git annex add -u now supported, analagous to git add -u
 
  -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 
diff --git a/Command/Add.hs b/Command/Add.hs
index f9cfbb9..beea48e 100644
--- a/Command/Add.hs
+++ b/Command/Add.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2010, 2013 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2017 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -30,6 +30,7 @@ data AddOptions = AddOptions
 	{ addThese :: CmdParams
 	, includeDotFiles :: Bool
 	, batchOption :: BatchMode
+	, updateOnly :: Bool
 	}
 
 optParser :: CmdParamsDesc -> Parser AddOptions
@@ -40,6 +41,11 @@ optParser desc = AddOptions
 		<> help "don't skip dotfiles"
 		)
 	<*> parseBatchOption
+	<*> switch
+		( long "update"
+		<> short 'u'
+		<> help "only update tracked files"
+		)
 
 seek :: AddOptions -> CommandSeek
 seek o = allowConcurrentOutput $ do
@@ -52,10 +58,14 @@ seek o = allowConcurrentOutput $ do
 			)
 		)
 	case batchOption o of
-		Batch -> batchFiles gofile
+		Batch
+			| updateOnly o ->
+				giveup "--update --batch is not supported"
+			| otherwise -> batchFiles gofile
 		NoBatch -> do
 			let go a = a gofile (addThese o)
-			go (withFilesNotInGit (not $ includeDotFiles o))
+			unless (updateOnly o) $
+				go (withFilesNotInGit (not $ includeDotFiles o))
 			go withFilesMaybeModified
 			unlessM (versionSupportsUnlockedPointers <||> isDirect) $
 				go withFilesOldUnlocked
diff --git a/doc/git-annex-add.mdwn b/doc/git-annex-add.mdwn
index 15bb8a6..2ebbbac 100644
--- a/doc/git-annex-add.mdwn
+++ b/doc/git-annex-add.mdwn
@@ -56,6 +56,11 @@ annexed content, and other symlinks.
   Adds multiple files in parallel. This may be faster.
   For example: `-J4`  
 
+* `--update` `-u`
+
+  Like `git add --update`, this does not add new files, but any updates
+  to tracked files will be added to the index.
+
 * `--json`
 
   Enable JSON output. This is intended to be parsed by programs that use
diff --git a/doc/todo/annex_add___40__-u__124__--update__41___mode.mdwn b/doc/todo/annex_add___40__-u__124__--update__41___mode.mdwn
index 8331176..ab0eac1 100644
--- a/doc/todo/annex_add___40__-u__124__--update__41___mode.mdwn
+++ b/doc/todo/annex_add___40__-u__124__--update__41___mode.mdwn
@@ -1,3 +1,5 @@
 to supplement 'git add -u' behavior -- to add only updated (tracked only, no untracked) files to be committed.  ATM all files would be added, including untracked.
 
 [[!meta author=yoh]]
+
+> [[done]]
diff --git a/doc/todo/annex_add___40__-u__124__--update__41___mode/comment_1_bde2b1e2c45e110d56ce98b43dd77743._comment b/doc/todo/annex_add___40__-u__124__--update__41___mode/comment_1_bde2b1e2c45e110d56ce98b43dd77743._comment
new file mode 100644
index 0000000..e6a6329
--- /dev/null
+++ b/doc/todo/annex_add___40__-u__124__--update__41___mode/comment_1_bde2b1e2c45e110d56ce98b43dd77743._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-07T19:41:02Z"
+ content="""
+Good idea, adding.
+"""]]

gcrypt: Support re-enabling to change eg, encryption parameters.
This was never supported before. And it doesn't re-encrypt the
gcrypt repo to the new gcrypt-participants, but it does at least now not
crash, and set gcrypt-participants.
This commit was sponsored by andrea rota.
diff --git a/CHANGELOG b/CHANGELOG
index 3d9ed22..c53b01c 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -14,9 +14,11 @@ git-annex (6.20170322) UNRELEASED; urgency=medium
     the parameters that git passes.
   * enableremote: When enabling a non-special remote, param=value
     parameters can't be used, so error out if any are provided.
-  * enableremote: Fix re-enabling of existing gcrypt remotes, so
-    that eg, encryption key changes take effect. They were silently
+  * enableremote: Fix re-enabling of special remotes that have a git
+    url, so that eg, encryption key changes take effect. They were silently
     ignored, a reversion introduced in 6.20160527.
+  * gcrypt: Support re-enabling to change eg, encryption parameters.
+    This was never supported before.
 
  -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 
diff --git a/Remote/GCrypt.hs b/Remote/GCrypt.hs
index ea101a7..f1b48cd 100644
--- a/Remote/GCrypt.hs
+++ b/Remote/GCrypt.hs
@@ -31,7 +31,6 @@ import qualified Git.Command
 import qualified Git.Config
 import qualified Git.GCrypt
 import qualified Git.Construct
-import qualified Git.Types as Git ()
 import qualified Annex.Branch
 import Config
 import Config.Cost
@@ -176,11 +175,18 @@ gCryptSetup _ mu _ c gc = go $ M.lookup "gitrepo" c
 	go Nothing = giveup "Specify gitrepo="
 	go (Just gitrepo) = do
 		(c', _encsetup) <- encryptionSetup c gc
-		inRepo $ Git.Command.run 
-			[ Param "remote", Param "add"
-			, Param remotename
-			, Param $ Git.GCrypt.urlPrefix ++ gitrepo
-			]
+
+		let url = Git.GCrypt.urlPrefix ++ gitrepo
+		rs <- fromRepo Git.remotes
+		case filter (\r -> Git.remoteName r == Just remotename) rs of
+			[] -> inRepo $ Git.Command.run 
+				[ Param "remote", Param "add"
+				, Param remotename
+				, Param url
+				]
+			(r:_)
+				| Git.repoLocation r == url -> noop
+				| otherwise -> error "Another remote with the same name already exists."		
 
 		setGcryptEncryption c' remotename
 
diff --git a/doc/bugs/git_annex_enableremote_ignoring_encryption_changes.mdwn b/doc/bugs/git_annex_enableremote_ignoring_encryption_changes.mdwn
index e73b2ce..92a454c 100644
--- a/doc/bugs/git_annex_enableremote_ignoring_encryption_changes.mdwn
+++ b/doc/bugs/git_annex_enableremote_ignoring_encryption_changes.mdwn
@@ -116,3 +116,5 @@ Yes, definitively. I enjoy using annex to backup and manage my data. I would lov
 Thanks
 
 Jörn
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/git_annex_enableremote_ignoring_encryption_changes/comment_2_5b36cd7f8ec4dd4f8787b60959512157._comment b/doc/bugs/git_annex_enableremote_ignoring_encryption_changes/comment_2_5b36cd7f8ec4dd4f8787b60959512157._comment
new file mode 100644
index 0000000..22a6cdf
--- /dev/null
+++ b/doc/bugs/git_annex_enableremote_ignoring_encryption_changes/comment_2_5b36cd7f8ec4dd4f8787b60959512157._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2017-04-07T16:58:42Z"
+ content="""
+Huh, so it seems that for gcrypt remotes, enableremote just doesn't
+call their setup function at all!
+
+Ah, it's because it sees the remote has an url, so it is not treated
+as a special remote, but as a regular git remote, and so the
+special remote encryption changes are ignored. (Since 6.20160527)
+
+So, enableremote needs to fail when it thinks it's enabling a regular git
+remote and has been passed some parameters which cannot apply to such a
+remote. Done.
+
+And, enableremote needs fixed to treat existing gcrypt remotes as special
+remotes. Done.
+
+Also, gcrypt special remotes didn't actually support being re-enabled
+either. I made that work. When an encryption key is added, that
+automatically makes it change the gcrypt-participants, too.
+
+I suppose enableremote could even be made to do the `GCRYPT_FULL_REPACK`
+and forced push, but that seems like too much for it to do!
+"""]]

enableremote: When enabling a non-special remote, param=value parameters can't be used, so error out if any are provided.
This commit was sponsored by Riku Voipio.
diff --git a/CHANGELOG b/CHANGELOG
index f364455..fc0e651 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -12,6 +12,8 @@ git-annex (6.20170322) UNRELEASED; urgency=medium
     necessary because as feared, the extra -n parameter that git-annex
     passes breaks uses of these environment variables that expect exactly
     the parameters that git passes.
+  * enableremote: When enabling a non-special remote, param=value
+    parameters can't be used, so error out if any are provided.
 
  -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 
diff --git a/Command/EnableRemote.hs b/Command/EnableRemote.hs
index 96efce3..3addd87 100644
--- a/Command/EnableRemote.hs
+++ b/Command/EnableRemote.hs
@@ -39,7 +39,10 @@ start (name:rest) = go =<< filter matchingname <$> Annex.fromRepo Git.remotes
 	matchingname r = Git.remoteName r == Just name
 	go [] = startSpecialRemote name (Logs.Remote.keyValToConfig rest)
 		=<< Annex.SpecialRemote.findExisting name
-	go (r:_) = startNormalRemote name r
+	go (r:_)
+		| null rest = startNormalRemote name r
+		| otherwise = giveup $ 
+			"That is a normal git remote; passing these parameters does not make sense: " ++ unwords rest
 
 startNormalRemote :: Git.RemoteName -> Git.Repo -> CommandStart
 startNormalRemote name r = do
diff --git a/doc/bugs/git_annex_enableremote_ignoring_encryption_changes/comment_1_e3924c02186d962e4dc1e1af8e968891._comment b/doc/bugs/git_annex_enableremote_ignoring_encryption_changes/comment_1_e3924c02186d962e4dc1e1af8e968891._comment
new file mode 100644
index 0000000..9d0193d
--- /dev/null
+++ b/doc/bugs/git_annex_enableremote_ignoring_encryption_changes/comment_1_e3924c02186d962e4dc1e1af8e968891._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-07T16:54:19Z"
+ content="""
+This does not happen when I do the same thing with a directory special
+remote.
+
+I do reproduce it with gcrypt. It must be missing a call to the code that
+handles keyid+=.
+
+Thanks for an excellent bug report!
+"""]]

comment
diff --git a/doc/encryption/comment_8_66e81e89fd483ca95620522b0f63c4fd._comment b/doc/encryption/comment_8_66e81e89fd483ca95620522b0f63c4fd._comment
new file mode 100644
index 0000000..bca3286
--- /dev/null
+++ b/doc/encryption/comment_8_66e81e89fd483ca95620522b0f63c4fd._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2017-04-07T16:52:59Z"
+ content="""
+@joern.mankiewicz, I see you found a bug and filed it, so will answer in
+the but report.
+
+Barring bugs, adding another gpg key to a hybrid encryption special remote
+is as simple as `git annex enableremote $theremote keyid+=$newkey`
+"""]]

confused user, not a bug
diff --git a/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
index 7e4d81d..fc098bb 100644
--- a/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
+++ b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
@@ -80,3 +80,5 @@ upgrade supported from repository versions: 0 1 2 3 4 5
 operating system: linux x86_64
 
 </pre>
+
+> [[done]] appears to be a confused user, not a bug. --[[Joey]]
diff --git a/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly/comment_2_13989894486a9ff9f243c6b2071d788f._comment b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly/comment_2_13989894486a9ff9f243c6b2071d788f._comment
new file mode 100644
index 0000000..5ffd00d
--- /dev/null
+++ b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly/comment_2_13989894486a9ff9f243c6b2071d788f._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2017-04-07T16:42:11Z"
+ content="""
+This bug report is, to the best of my knowledge, totally wrong.
+
+Yes, git-annex sync pushes the git-annex branch to all git remotes.
+This is intentional, and not a problem. That is just a git branch.
+It's helpful to push it to eg, github, if you want to be able to pull it
+from there into another clone. Pushing the git-annex branch to github does
+*not* make git-annex think that github is holding the contents of annexed
+files.
+
+In your "reproduction" section, I think you forgot to run `git annex sync`
+in Zeta, so its working tree had not been updated with the files synced to
+it from Alpha, which is why `git annex list` didn't show anything. Or
+somethig like that. You did not provide quite enough information to guess
+what you were doing.
+
+I'm going to close this. If you think it is still a problem, please provide
+either a detailed transcript demontrating the problem or enough information
+to reporoduce the problem, starting with an empty directory and
+constructing the necessary git repositories to demonstrate the problem.
+"""]]

comment
diff --git a/doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes/comment_8_5aa63bf0d8e4145974702a86b36bd435._comment b/doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes/comment_8_5aa63bf0d8e4145974702a86b36bd435._comment
new file mode 100644
index 0000000..2406728
--- /dev/null
+++ b/doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes/comment_8_5aa63bf0d8e4145974702a86b36bd435._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""@ben: what?"""
+ date="2017-04-07T16:37:31Z"
+ content="""
+I don't know what you mean with `git annex init` in a v6 repo somehow
+doing something to a non-annexed file. That would be extemely surprising
+if it happened, and it does not happen when I try to follow your
+instructions.
+
+	joey@darkstar:~/tmp/v62>git annex init --version=6
+	init  (merging origin/git-annex into git-annex...)
+	(recording state in git...)
+	ok
+	(recording state in git...)
+	joey@darkstar:~/tmp/v62>ls
+	x  y@
+	joey@darkstar:~/tmp/v62>git annex get y
+	get y (from origin...) (checksum...) ok
+	(recording state in git...)
+	joey@darkstar:~/tmp/v62>git status
+	On branch master
+	Your branch is up-to-date with 'origin/master'.
+	nothing to commit, working tree clean
+"""]]

partial analysis
diff --git a/doc/bugs/committing_files_into_git_doesn__39__t_work_with_explicitly_given_paths_in_V6/comment_1_1922f38b3620e94e90b16e3c14f59add._comment b/doc/bugs/committing_files_into_git_doesn__39__t_work_with_explicitly_given_paths_in_V6/comment_1_1922f38b3620e94e90b16e3c14f59add._comment
new file mode 100644
index 0000000..5faf6b7
--- /dev/null
+++ b/doc/bugs/committing_files_into_git_doesn__39__t_work_with_explicitly_given_paths_in_V6/comment_1_1922f38b3620e94e90b16e3c14f59add._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-07T16:24:59Z"
+ content="""
+The root problem is, you have set annex.largefiles=nothing temporarily when
+adding the file. But, when git commit re-smudges the file, that is not set,
+so git-annex generates a v6 pointer, which is what gets committed.
+
+I don't think you will have these problems if use use .gitattributes to
+configure annex.largefiles.
+
+(There is something going on that I don't quite understand with why
+git status then thinks the file has changed. git diff --cached shows a diff
+between the pointer that got committed and the actual file contents; 
+I would have expected that git would run the smudge/clean filter then and
+not show that diff.)
+"""]]

Disable git-annex's support for GIT_SSH and GIT_SSH_COMMAND, unless GIT_ANNEX_USE_GIT_SSH=1 is also set in the environment.
This is necessary because as feared, the extra -n parameter that git-annex
passes breaks uses of these environment variables that expect exactly the
parameters that git passes.
For example, see https://github.com/datalad/datalad/issues/1456
It would of course be possible to pre-close stdin before running ssh so not
needing the -n, and I think that would not even break ssh's password
caching. But it would probably involve a lot of work, possibly would need
to deal with some layering violations, and would be error-prone. The really
clean fix would be to make all the ssh stuff return a CreateProcess, which
could have the handle closed when appropriate, but that would be a large
reworing of the code base.
This commit was supported by the NSF-funded DataLad project.
diff --git a/Annex/Ssh.hs b/Annex/Ssh.hs
index aa35754..bf13a02 100644
--- a/Annex/Ssh.hs
+++ b/Annex/Ssh.hs
@@ -52,10 +52,13 @@ data ConsumeStdin = ConsumeStdin | NoConsumeStdin
 
 {- Generates a command to ssh to a given host (or user@host) on a given
  - port. This includes connection caching parameters, and any ssh-options.
- - If GIT_SSH or GIT_SSH_COMMAND is set, they are used instead. -}
+ - If GIT_SSH or GIT_SSH_COMMAND is enabled, they are used instead. -}
 sshCommand :: ConsumeStdin -> (SshHost, Maybe SshPort) -> RemoteGitConfig -> SshCommand -> Annex (FilePath, [CommandParam])
-sshCommand cs (host, port) gc remotecmd = maybe go return
-	=<< liftIO (gitSsh' host port remotecmd (consumeStdinParams cs))
+sshCommand cs (host, port) gc remotecmd = ifM (liftIO safe_GIT_SSH)
+	( maybe go return
+		=<< liftIO (gitSsh' host port remotecmd (consumeStdinParams cs))
+	, go
+	)
   where
 	go = do
 		ps <- sshOptions cs (host, port) gc []
@@ -81,6 +84,12 @@ sshOptions cs (host, port) gc opts = go =<< sshCachingInfo (host, port)
 		, [Param "-T"]
 		]
 
+{- Due to passing -n to GIT_SSH and GIT_SSH_COMMAND, some settings
+ - of those that expect exactly git's parameters will break. So only
+ - use those if the user set GIT_ANNEX_USE_GIT_SSH to say it's ok. -}
+safe_GIT_SSH :: IO Bool
+safe_GIT_SSH = (== Just "1") <$> getEnv "GIT_ANNEX_USE_GIT_SSH"
+
 consumeStdinParams :: ConsumeStdin -> [CommandParam]
 consumeStdinParams ConsumeStdin = []
 consumeStdinParams NoConsumeStdin = [Param "-n"]
@@ -305,13 +314,13 @@ inRepoWithSshOptionsTo remote gc a =
  - to set GIT_SSH=git-annex, and set sshOptionsEnv when running git
  - commands.
  -
- - If GIT_SSH or GIT_SSH_COMMAND are set, this has no effect. -}
+ - If GIT_SSH or GIT_SSH_COMMAND are enabled, this has no effect. -}
 sshOptionsTo :: Git.Repo -> RemoteGitConfig -> Git.Repo -> Annex Git.Repo
 sshOptionsTo remote gc localr
 	| not (Git.repoIsUrl remote) || Git.repoIsHttp remote = unchanged
 	| otherwise = case Git.Url.hostuser remote of
 		Nothing -> unchanged
-		Just host -> ifM (liftIO gitSshEnvSet)
+		Just host -> ifM (liftIO $ safe_GIT_SSH <&&> gitSshEnvSet)
 			( unchanged
 			, do
 				(msockfile, _) <- sshCachingInfo (host, Git.Url.port remote)
diff --git a/CHANGELOG b/CHANGELOG
index 0a9e56f..f364455 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,11 @@ git-annex (6.20170322) UNRELEASED; urgency=medium
   * Added remote.<name>.annex-push and remote.<name>.annex-pull
     which can be useful to make remotes that don't get fully synced with
     local changes.
+  * Disable git-annex's support for GIT_SSH and GIT_SSH_COMMAND, unless
+    GIT_ANNEX_USE_GIT_SSH=1 is also set in the environment. This is
+    necessary because as feared, the extra -n parameter that git-annex
+    passes breaks uses of these environment variables that expect exactly
+    the parameters that git passes.
 
  -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 2bebd9f..56be4bc 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -1434,7 +1434,10 @@ These environment variables are used by git-annex when set:
   Handled similarly to the same as described in git(1).
   The one difference is that git-annex will sometimes pass an additional
   "-n" parameter to these, as the first parameter, to prevent ssh from
-  reading from stdin.
+  reading from stdin. Since that can break existing uses of these
+  environment variables that don't expect the extra parameter, you will
+  need to set `GIT_ANNEX_USE_GIT_SSH=1` to make git-annex support
+  these.
 
   Note that setting either of these environment variables prevents
   git-annex from automatically enabling ssh connection caching

Added a comment
diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_6_a7c02a4dfa74de8ad05bfaaee0b335b8._comment b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_6_a7c02a4dfa74de8ad05bfaaee0b335b8._comment
new file mode 100644
index 0000000..a92c9ac
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_6_a7c02a4dfa74de8ad05bfaaee0b335b8._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="woffs"
+ avatar="http://cdn.libravatar.org/avatar/198790d74209efe4896fd4cfc37ec2a6"
+ subject="comment 6"
+ date="2017-04-06T08:28:06Z"
+ content="""
+I was wrong. git-annex-sync DOES pull/push to the gcrypt rsync remote. So seems fine. Thank you. :)
+"""]]

Added a comment
diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_5_39d905d4577c9b2987bf5e6cdbace7f2._comment b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_5_39d905d4577c9b2987bf5e6cdbace7f2._comment
new file mode 100644
index 0000000..72de2a5
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_5_39d905d4577c9b2987bf5e6cdbace7f2._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="woffs"
+ avatar="http://cdn.libravatar.org/avatar/198790d74209efe4896fd4cfc37ec2a6"
+ subject="comment 5"
+ date="2017-04-06T07:29:14Z"
+ content="""
+I remember having tried this (gcrypt rsync:// git remote + rsync encrypted git-annex specialremote) but I was disappointed because git-annex-sync does not pull/push the git remote and I had to do this separately every time.
+
+Anyway, thank you for caring. :-)
+"""]]

Added a comment
diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_4_0f82673281494b1cb084dce702525a01._comment b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_4_0f82673281494b1cb084dce702525a01._comment
new file mode 100644
index 0000000..0245a0b
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_4_0f82673281494b1cb084dce702525a01._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="spwhitton"
+ avatar="http://cdn.libravatar.org/avatar/9c3f08f80e67733fd506c353239569eb"
+ subject="comment 4"
+ date="2017-04-05T22:28:10Z"
+ content="""
+git-remote-gcrypt maintainer here.
+
+As Joey recommends for the meantime, I am successfully using an `rsync://` gcrypt remote plus a separate encrypted git-annex rsync remote to work around these performance issues.
+
+git-remote-gcrypt relies on rsync to implement the incremental upload, so the README is wrong to suggest that using an `sftp://` remote would work around this issue -- git-remote-gcrypt invokes curl for the sftp transaction, which as far as I know does nothing incremental (that's presumably why rsync exists).  I've just updated the README.
+
+If we wanted the gitception gcrypt remote to be incremental, we would need to implement rsync-like incremental uploads on top of the structure of git commits, such that we could push a git commit that represents the changes to the gcrypt packfiles and manifest since the previous commit.  But I don't think this is possible for binary files -- I don't think git can represent the deltas efficiently.
+
+I've come to think that git-remote-gcrypt's gitception mode is not actually very useful, simply due to the design of git.  But perhaps there is an alternative way to represent the manifest and packfiles that would be compatible with incremental git pushes.
+"""]]

devblog
diff --git a/doc/devblog/day_455__semi-synchronized.mdwn b/doc/devblog/day_455__semi-synchronized.mdwn
new file mode 100644
index 0000000..2a53f06
--- /dev/null
+++ b/doc/devblog/day_455__semi-synchronized.mdwn
@@ -0,0 +1,9 @@
+Catching up on some recent backlog after a trip and post-trip flu.
+
+Anarcat wrote up an anlysis of [[tips/semi-synchronized_remotes]], and
+based on that I implemented `remote.<name>.annex-push` and
+`remote.<name>.annex-pull`
+
+Also fixed the Windows build.
+
+Today's work was sponsored by Thomas Hochstein on Patreon.

Added a comment
diff --git a/doc/forum/Split_annex_into_multiple/comment_2_eec6bc8c49c08411a4bb2cf7cc1c4697._comment b/doc/forum/Split_annex_into_multiple/comment_2_eec6bc8c49c08411a4bb2cf7cc1c4697._comment
new file mode 100644
index 0000000..1957eb2
--- /dev/null
+++ b/doc/forum/Split_annex_into_multiple/comment_2_eec6bc8c49c08411a4bb2cf7cc1c4697._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="archimedes"
+ avatar="http://cdn.libravatar.org/avatar/5b2ed40aabedcb1b8c2c0ce69f55a1e1"
+ subject="comment 2"
+ date="2017-04-05T20:01:44Z"
+ content="""
+So actually \"import --reinject-duplicates\" solves the issue of reinject not being recursive.
+
+
+That trick on .git/annex/objects/ is cunning, but the destruction of Global before everything has been finished is discomforting...if only there was a \"--reinject-duplicates --duplicate\" option for import.
+
+No chance that I can make a recursive copy of .git/annex/objects with hardlinks can I? To then use with reinject-duplicates? 
+"""]]

response
diff --git a/doc/forum/Split_annex_into_multiple/comment_1_e934b404f738cbd67df928fd6769e3ff._comment b/doc/forum/Split_annex_into_multiple/comment_1_e934b404f738cbd67df928fd6769e3ff._comment
new file mode 100644
index 0000000..84451b8
--- /dev/null
+++ b/doc/forum/Split_annex_into_multiple/comment_1_e934b404f738cbd67df928fd6769e3ff._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-05T19:09:03Z"
+ content="""
+Recent git-annex's support `git annex reinject --known`
+
+You can do this -- but do note that it leaves Global in an unusable state,
+so only do it if you don't plan to use that repository again:
+
+	chmod -R +w $Global/.git/annex/objects/
+	git annex reinject --known $(find $Global/.git/annex/objects/ -type f)
+
+BTW, splitting the files amoung several git branches is also a useful
+thing to do in this situation. Splitting amoung branches avoids most of
+git's problems with a repository having too many files in it.
+"""]]

diff --git a/doc/forum/Split_annex_into_multiple.mdwn b/doc/forum/Split_annex_into_multiple.mdwn
new file mode 100644
index 0000000..2ebdf63
--- /dev/null
+++ b/doc/forum/Split_annex_into_multiple.mdwn
@@ -0,0 +1,16 @@
+Lets say I have an annex, called Global in some location A. The plan is to mirror it to some location B.
+
+I suddenly decide that Global has become too unwieldy, and I wish to split it up. 
+
+I "rsync -Lhav" the entire annex over to B, so it turns back into raw files (as I'm a little worried about the broken symlinks some have mentioned when using uninit).
+I then reorganize all the files in B into five new annexes. Yay.
+
+Now, I clone those five repos back onto A. How can I use the content of the original repo "Global" to populate the new annexes?
+
+**Approach 1**: delete Global, use git-annex copy (slow, wasteful use of precious disk life)
+
+**Approach 2**: uninit Global, deal with whatever broken symlink mess *may* occur, and then use reinject. Reinject doesn't seem to work recursively afaik (ok, I can use find -type f -exec I guess...), and doesn't seem to want to work on files that are sitting inside an annex already (i.e. the uninit is not optional)
+
+So besides these two, are there any approaches I'm overlooking?
+
+Context: entire photo collection. split into e.g. parents digitized photos, my photos, etc

followup
diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest.mdwn b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest.mdwn
index f136b25..4f2abac 100644
--- a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest.mdwn
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest.mdwn
@@ -26,3 +26,5 @@ is running. The upload of the actual changeset starts after this, the processes
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 git-annex is great and revolutionized my file organization and backup structure (if they were even existing before)
+
+[[!meta tite="gcrypt special remotes should support rsync:// and perhaps also sftp://"]]
diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_3_83fd8643b988fdf689ef40b819b48299._comment b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_3_83fd8643b988fdf689ef40b819b48299._comment
new file mode 100644
index 0000000..2f46e29
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_3_83fd8643b988fdf689ef40b819b48299._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2017-04-05T18:35:30Z"
+ content="""
+It should be possible for git-annex to support rsync:// for gcrypt special
+remotes; that would just need to reuse the rsync special remote for the
+git-annex objects. Retitling this bug report appropriately.
+
+In the meantime, it should work to set up a gcrypt git remote (not a
+git-annex special remote) using rsync:// or sftp:// and then use a
+git-annex rsync special remote on the same server to store the annex
+objects.
+
+But that doesn't help with large pushes to gcrypt remotes
+when git hosting providers are being used, which is a main 
+use case for using gcrypt (though generally not the gcrypt special remote).
+The lack of incrementals there seems like something worth finding a way to
+fix in gcrypt.
+"""]]

Added remote.<name>.annex-push and remote.<name>.annex-pull
The former can be useful to make remotes that don't get fully synced with
local changes, which comes up in a lot of situations.
The latter was mostly added for symmetry, but could be useful (though less
likely to be).
Implementing `remote.<name>.annex-pull` was a bit tricky, as there's no one
place where git-annex pulls/fetches from remotes. I audited all
instances of "fetch" and "pull". A few cases were left not checking this
config:
* Git.Repair can try to pull missing refs from a remote, and if the local
repo is corrupted, that seems a reasonable thing to do even though
the config would normally prevent it.
* Assistant.WebApp.Gpg and Remote.Gcrypt and Remote.Git do fetches
as part of the setup process of a remote. The config would probably not
be set then, and having the setup fail seems worse than honoring it if it
is already set.
I have not prevented all the code that does a "merge" from merging branches
from remotes with remote.<name>.annex-pull=false. That could perhaps
be done, but it would need a way to map from branch name to remote name,
and the way refspecs work makes that hard to get really correct. So if the
user fetches manually, the git-annex branch will get merged, for example.
Anther way of looking at/justifying this is that the setting is called
"annex-pull", not "annex-merge".
This commit was supported by the NSF-funded DataLad project.
diff --git a/Assistant/Sync.hs b/Assistant/Sync.hs
index 702f1e9..8f30aa4 100644
--- a/Assistant/Sync.hs
+++ b/Assistant/Sync.hs
@@ -110,8 +110,14 @@ reconnectRemotes rs = void $ do
 pushToRemotes :: [Remote] -> Assistant [Remote]
 pushToRemotes remotes = do
 	now <- liftIO getCurrentTime
-	let remotes' = filter (not . remoteAnnexReadOnly . Remote.gitconfig) remotes
+	let remotes' = filter (wantpush . Remote.gitconfig) remotes
 	syncAction remotes' (pushToRemotes' now)
+  where
+	wantpush gc
+		| remoteAnnexReadOnly gc = False
+		| not (remoteAnnexPush gc) = False
+		| otherwise = True
+
 pushToRemotes' :: UTCTime -> [Remote] -> Assistant [Remote]
 pushToRemotes' now remotes = do
 	(g, branch, u) <- liftAnnex $ do
@@ -195,16 +201,20 @@ manualPull :: Command.Sync.CurrBranch -> [Remote] -> Assistant ([Remote], Bool)
 manualPull currentbranch remotes = do
 	g <- liftAnnex gitRepo
 	let (_xmppremotes, normalremotes) = partition Remote.isXMPPRemote remotes
-	failed <- forM normalremotes $ \r -> do
-		g' <- liftAnnex $ sshOptionsTo (Remote.repo r) (Remote.gitconfig r) g
-		ifM (liftIO $ Git.Command.runBool [Param "fetch", Param $ Remote.name r] g')
-			( return Nothing
-			, return $ Just r
-			)
+	failed <- forM normalremotes $ \r -> if wantpull $ Remote.gitconfig r
+		then do
+			g' <- liftAnnex $ sshOptionsTo (Remote.repo r) (Remote.gitconfig r) g
+			ifM (liftIO $ Git.Command.runBool [Param "fetch", Param $ Remote.name r] g')
+				( return Nothing
+				, return $ Just r
+				)
+		else return Nothing
 	haddiverged <- liftAnnex Annex.Branch.forceUpdate
 	forM_ normalremotes $ \r ->
 		liftAnnex $ Command.Sync.mergeRemote r currentbranch Command.Sync.mergeConfig
 	return (catMaybes failed, haddiverged)
+  where
+	wantpull gc = remoteAnnexPull gc
 
 {- Start syncing a remote, using a background thread. -}
 syncRemote :: Remote -> Assistant ()
diff --git a/CHANGELOG b/CHANGELOG
index f69aade..0a9e56f 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -4,6 +4,9 @@ git-annex (6.20170322) UNRELEASED; urgency=medium
     about it once, not every time git-annex is run.
   * multicast: New command, uses uftp to multicast annexed files, for eg
     a classroom setting.
+  * Added remote.<name>.annex-push and remote.<name>.annex-pull
+    which can be useful to make remotes that don't get fully synced with
+    local changes.
 
  -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 
diff --git a/Command/Sync.hs b/Command/Sync.hs
index f2c1945..85bb8c1 100644
--- a/Command/Sync.hs
+++ b/Command/Sync.hs
@@ -360,7 +360,7 @@ updateBranch syncbranch updateto g =
 		] g
 
 pullRemote :: SyncOptions -> [Git.Merge.MergeConfig] -> Remote -> CurrBranch -> CommandStart
-pullRemote o mergeconfig remote branch = stopUnless (pure $ pullOption o) $ do
+pullRemote o mergeconfig remote branch = stopUnless (pure $ pullOption o && wantpull) $ do
 	showStart "pull" (Remote.name remote)
 	next $ do
 		showOutput
@@ -370,6 +370,7 @@ pullRemote o mergeconfig remote branch = stopUnless (pure $ pullOption o) $ do
 	fetch = inRepoWithSshOptionsTo (Remote.repo remote) (Remote.gitconfig remote) $
 		Git.Command.runBool
 			[Param "fetch", Param $ Remote.name remote]
+	wantpull = remoteAnnexPull (Remote.gitconfig remote)
 
 {- The remote probably has both a master and a synced/master branch.
  - Which to merge from? Well, the master has whatever latest changes
@@ -400,7 +401,7 @@ pushRemote o remote (Just branch, _) = stopUnless (pure (pushOption o) <&&> need
 	showStart "push" (Remote.name remote)
 	next $ next $ do
 		showOutput
-		ok <- inRepoWithSshOptionsTo (Remote.repo remote) (Remote.gitconfig remote) $
+		ok <- inRepoWithSshOptionsTo (Remote.repo remote) gc $
 			pushBranch remote branch
 		if ok
 			then postpushupdate
@@ -410,7 +411,8 @@ pushRemote o remote (Just branch, _) = stopUnless (pure (pushOption o) <&&> need
 				return ok
   where
 	needpush
-		| remoteAnnexReadOnly (Remote.gitconfig remote) = return False
+		| remoteAnnexReadOnly gc = return False
+		| not (remoteAnnexPush gc) = return False
 		| otherwise = anyM (newer remote) [syncBranch branch, Annex.Branch.name]
 	-- Do updateInstead emulation for remotes on eg removable drives
 	-- formatted FAT, where the post-update hook won't run.
@@ -426,6 +428,7 @@ pushRemote o remote (Just branch, _) = stopUnless (pure (pushOption o) <&&> need
 					, return True
 					)
 		| otherwise = return True
+	gc = Remote.gitconfig remote
 
 {- Pushes a regular branch like master to a remote. Also pushes the git-annex
  - branch.
diff --git a/RemoteDaemon/Common.hs b/RemoteDaemon/Common.hs
index 711771f..366f6aa 100644
--- a/RemoteDaemon/Common.hs
+++ b/RemoteDaemon/Common.hs
@@ -8,7 +8,7 @@
 module RemoteDaemon.Common
 	( liftAnnex
 	, inLocalRepo
-	, checkNewShas
+	, checkShouldFetch
 	, ConnectionStatus(..)
 	, robustConnection
 	) where
@@ -35,6 +35,13 @@ liftAnnex (TransportHandle _ annexstate) a = do
 inLocalRepo :: TransportHandle -> (Git.Repo -> IO a) -> IO a
 inLocalRepo (TransportHandle (LocalRepo g) _) a = a g
 
+-- Check if some shas should be fetched from the remote,
+-- and presumably later merged.
+checkShouldFetch :: RemoteGitConfig -> TransportHandle -> [Git.Sha] -> IO Bool
+checkShouldFetch gc transporthandle shas
+	| remoteAnnexPull gc = checkNewShas transporthandle shas
+	| otherwise = return False
+
 -- Check if any of the shas are actally new in the local git repo,
 -- to avoid unnecessary fetching.
 checkNewShas :: TransportHandle -> [Git.Sha] -> IO Bool
diff --git a/RemoteDaemon/Transport/Ssh.hs b/RemoteDaemon/Transport/Ssh.hs
index fdb75e8..772ae97 100644
--- a/RemoteDaemon/Transport/Ssh.hs
+++ b/RemoteDaemon/Transport/Ssh.hs
@@ -36,7 +36,7 @@ transportUsingCmd cmd params rr@(RemoteRepo r gc) url h@(TransportHandle (LocalR
 	transportUsingCmd' cmd params rr url transporthandle ichan ochan
 
 transportUsingCmd' :: FilePath -> [CommandParam] -> Transport
-transportUsingCmd' cmd params (RemoteRepo r _) url transporthandle ichan ochan =
+transportUsingCmd' cmd params (RemoteRepo r gc) url transporthandle ichan ochan =
 	robustConnection 1 $ do
 		(Just toh, Just fromh, Just errh, pid) <-
 			createProcess (proc cmd (toCommand params))
@@ -74,7 +74,7 @@ transportUsingCmd' cmd params (RemoteRepo r _) url transporthandle ichan ochan =
 				send (CONNECTED url)
 				handlestdout fromh
 			Just (SshRemote.CHANGED (ChangedRefs shas)) -> do
-				whenM (checkNewShas transporthandle shas) $
+				whenM (checkShouldFetch gc transporthandle shas) $
 					fetch
 				handlestdout fromh
 			-- avoid reconnect on protocol error
diff --git a/RemoteDaemon/Transport/Tor.hs b/RemoteDaemon/Transport/Tor.hs
index afa249b..b0fa3c1 100644
--- a/RemoteDaemon/Transport/Tor.hs
+++ b/RemoteDaemon/Transport/Tor.hs
@@ -129,7 +129,7 @@ serveClient th u r q = bracket setup cleanup start
 
 -- Connect to peer's tor hidden service.
 transport :: Transport
-transport (RemoteRepo r _) url@(RemoteURI uri) th ichan ochan =
+transport (RemoteRepo r gc) url@(RemoteURI uri) th ichan ochan =
 	case unformatP2PAddress (show uri) of
 		Nothing -> return ()
 		Just addr -> robustConnection 1 $ do
@@ -168,7 +168,7 @@ transport (RemoteRepo r _) url@(RemoteURI uri) th ichan ochan =
 		v <- runNetProto conn P2P.notifyChange
 		case v of
 			Right (Just (ChangedRefs shas)) -> do
-				whenM (checkNewShas th shas) $
+				whenM (checkShouldFetch gc th shas) $
 					fetch
 				handlepeer conn
 			_ -> return ConnectionClosed
diff --git a/Types/GitConfig.hs b/Types/GitConfig.hs
index af699a7..da548d4 100644
--- a/Types/GitConfig.hs
+++ b/Types/GitConfig.hs
@@ -181,6 +181,8 @@ data RemoteGitConfig = RemoteGitConfig
 	, remoteAnnexCostCommand :: Maybe String
 	, remoteAnnexIgnore :: Bool
 	, remoteAnnexSync :: Bool
+	, remoteAnnexPull :: Bool
+	, remoteAnnexPush :: Bool
 	, remoteAnnexReadOnly :: Bool
 	, remoteAnnexVerify :: Bool
 	, remoteAnnexTrustLevel :: Maybe String
@@ -218,6 +220,8 @@ extractRemoteGitConfig r remotename = RemoteGitConfig
 	, remoteAnnexCostCommand = notempty $ getmaybe "cost-command"
 	, remoteAnnexIgnore = getbool "ignore" False
 	, remoteAnnexSync = getbool "sync" True
+	, remoteAnnexPull = getbool "pull" True
+	, remoteAnnexPush = getbool "push" True
 	, remoteAnnexReadOnly = getbool "readonly" False

(Diff truncated)
Added a comment: git-remote-gcrypt recommends rsync:// or sftp:// transports
diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_2_02ae97849e2d9fc6d3d996500f264455._comment b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_2_02ae97849e2d9fc6d3d996500f264455._comment
new file mode 100644
index 0000000..d9174ce
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_2_02ae97849e2d9fc6d3d996500f264455._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="git-annex@6f13b739194f758abc0b86556b7ce966c1bf3c00"
+ nickname="git-annex"
+ avatar="http://cdn.libravatar.org/avatar/198790d74209efe4896fd4cfc37ec2a6"
+ subject="git-remote-gcrypt recommends rsync:// or sftp:// transports"
+ date="2017-04-05T16:16:25Z"
+ content="""
+spwhitton says on <https://github.com/spwhitton/git-remote-gcrypt>:
+
+> \"Using an arbitrary <giturl> requires uploading the entire repository history with
+> each push. If your repository history is large or you are pushing over a slow link, consider using
+> either the rsync:// or sftp:// transports, which perform incremental pushes\"
+
+So it's a known performance. Would be great if rsync:// could be used when combining git-annex with
+git-remote-gcrypt?
+
+"""]]

comment
diff --git a/doc/forum/library__44___backup_and_music_player/comment_1_a6e38a1678c433e5a5d96afae4ae4332._comment b/doc/forum/library__44___backup_and_music_player/comment_1_a6e38a1678c433e5a5d96afae4ae4332._comment
new file mode 100644
index 0000000..edf4c4e
--- /dev/null
+++ b/doc/forum/library__44___backup_and_music_player/comment_1_a6e38a1678c433e5a5d96afae4ae4332._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-05T15:52:30Z"
+ content="""
+Good question! I don't have a perfect solution for this, 
+because what you really probably want is to be able to `git annex copy`
+files to the music player and have only the present files show up in
+its repository.
+
+One approach is to use a [[directory_special_remote|special_remotes/directory]]
+on the music player. This way, only present files will show up there,
+but they'll have git-annex's internal filenames. If the music player
+doesn't care about filenames, this can work ok.
+
+Another approach is to use a v6 git-annex repository
+on the media player, and use [[git-annex-adjust]] to generate
+a version of a branch where all files that are present are unlocked
+(and thus regular files with the expected filenames). Files
+that are not present will still show up in the working tree though,
+either as symlinks or as the small text files git uses when a filesystem
+does not support symlinks. So this is much like direct mode, but it has
+the benefit that you can use all the normal git branching features
+to create a trimmed down branch for the media player with just the files
+you want.
+"""]]

comment
diff --git a/doc/tips/peer_to_peer_network_with_tor/comment_4_917eee70600e13470f70555abe06597d._comment b/doc/tips/peer_to_peer_network_with_tor/comment_4_917eee70600e13470f70555abe06597d._comment
new file mode 100644
index 0000000..a9326db
--- /dev/null
+++ b/doc/tips/peer_to_peer_network_with_tor/comment_4_917eee70600e13470f70555abe06597d._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2017-04-05T15:47:55Z"
+ content="""
+@anarcat, I'm not sure I follow how git-annex would use that. It
+does not currently use the tor control port at all when setting up
+its tor hidden service. But please file a todo item if I'm wrong
+and there's something to be improved.
+
+(What would be useful is if tor came with the control port enabled
+by default, and perhaps protected by something like onion-grater.
+Then git-annex could use it when it installs its hidden service,
+instead of its current approach of prompting for the root password
+and modifying the torrc file.)
+"""]]

comment
diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_1_45982ced836d2e0f41a5ddd7edd59936._comment b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_1_45982ced836d2e0f41a5ddd7edd59936._comment
new file mode 100644
index 0000000..065ca65
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_1_45982ced836d2e0f41a5ddd7edd59936._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-04-05T15:44:47Z"
+ content="""
+I don't think this is a bug in git-annex. That is the behavior of
+`git-remote-gcrypt`. Could you please file a bug on that program instead?
+
+(I've also wondered about this; it could be that it's re-uploading the
+manifest every time for good security reasons, or it could be that a better
+file structure would allow more incremental uploads. It certianly
+seems like it could avoid re-uploading when nothing has changed!)
+"""]]

comment
diff --git a/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/.comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment.swp b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/.comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment.swp
deleted file mode 100644
index 24597d8..0000000
Binary files a/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/.comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment.swp and /dev/null differ
diff --git a/doc/tips/Systemd_unit/comment_2_389a2b36c3115eb3342429b0b68ddef2._comment b/doc/tips/Systemd_unit/comment_2_389a2b36c3115eb3342429b0b68ddef2._comment
new file mode 100644
index 0000000..ed61b19
--- /dev/null
+++ b/doc/tips/Systemd_unit/comment_2_389a2b36c3115eb3342429b0b68ddef2._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2017-04-05T15:43:21Z"
+ content="""
+@oberix, the --autostart --foreground combination is only supported
+properly since git-annex version 6.20170214. Before that, the --forground
+was ignored when using --autostart.
+"""]]

comment
diff --git a/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/.comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment.swp b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/.comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment.swp
new file mode 100644
index 0000000..24597d8
Binary files /dev/null and b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/.comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment.swp differ
diff --git a/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment
new file mode 100644
index 0000000..5ea68b6
--- /dev/null
+++ b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_8_a6c330f5ad2f64d86d8a24c23c49115a._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2017-04-05T15:26:59Z"
+ content="""
+git-annex does use CoW in a few situations; it does so by running `cp
+--reflink` and letting it use CoW features when available. However,
+I don't see where `git annex add` would use that, and of course I don't see
+why that feature would break for you even if it did run cp that way.
+
+git also uses CoW, at least it uses `mmap` with `MAP_PRIVATE`. I'm not
+clear if/how that involves the filesystem layer.
+
+This remains puzzling, but knowing it's limited to btrfs on Arch Linux
+with CoW is certianly a good start.
+
+It seems like a bug in btrfs would not be out of the question.
+Trying some different kernel versions might be useful.
+
+It would perhaps be useful to get `strace -ff` logs.
+"""]]

comment
diff --git a/doc/devblog/day_454__multicast/comment_2_09908606fc362cbee516999cd696f718._comment b/doc/devblog/day_454__multicast/comment_2_09908606fc362cbee516999cd696f718._comment
new file mode 100644
index 0000000..b7b5279
--- /dev/null
+++ b/doc/devblog/day_454__multicast/comment_2_09908606fc362cbee516999cd696f718._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2017-04-05T15:20:20Z"
+ content="""
+Multicast is only being used to send git-annex objects around, not git
+objects. There's assumed to be some way to sync git repositories, which is
+how the encryption keys for uftp are distributed.
+
+`git annex multicast --send` operates on files in the working tree.
+It would be possible to make it support `--all`.
+
+I'm not sure if uftp can send outside the local LAN.
+
+It would certianly be possible to have a special remote backed by uftp
+that thus only sends to a single host. Since multicast does not send
+to any particular remote, it did not make sense to implement it
+as a special remote.
+"""]]

Added a comment
diff --git a/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_7_fc053d15c8a634fc7be744ee51470fc6._comment b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_7_fc053d15c8a634fc7be744ee51470fc6._comment
new file mode 100644
index 0000000..e2d331e
--- /dev/null
+++ b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_7_fc053d15c8a634fc7be744ee51470fc6._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="t.z.mates"
+ avatar="http://cdn.libravatar.org/avatar/90f15fad216078fd08d62cc676487925"
+ subject="comment 7"
+ date="2017-04-02T04:17:55Z"
+ content="""
+Thanks for the suggestions! Just to clarrify, I've tried this on three of my machines: machines 1 and 2 both run arch linux and both experience the same error (even in `/tmp/newrepo`); machine 3 runs opensuse but does not experience the error.
+
+However, all three machines are using BTRFS for the underlying filesystem. I next tried running the script (on machine 1) in a partition that has Copy-on-Write turned off, and it completed successfully. So it seems that CoW has something to do with it (though the script runs fine even with CoW enabled on machine 3). So it seems we got a bit closer, but I'm still not sure what to make of it.
+"""]]

Added a comment: point to point?
diff --git a/doc/devblog/day_454__multicast/comment_1_137ea34cd11004513a03f24955719756._comment b/doc/devblog/day_454__multicast/comment_1_137ea34cd11004513a03f24955719756._comment
new file mode 100644
index 0000000..5f842ee
--- /dev/null
+++ b/doc/devblog/day_454__multicast/comment_1_137ea34cd11004513a03f24955719756._comment
@@ -0,0 +1,38 @@
+[[!comment format=mdwn
+ username="anarcat"
+ avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
+ subject="point to point?"
+ date="2017-03-31T13:08:59Z"
+ content="""
+I wonder: could this be used to send blobs to arbitrary hosts as well? I've been looking at the problem of sharing blobs without sharing the git repository (in [[tips/semi-synchronized_remotes]]), and while we have the tor/wormhole and SSH remotes that can be (ab)used for this purpose, uftp seems like a much better fit, feature-wise.
+
+When you run: `git annex multicast --gen-address; git annex sync`, does the `sync` command exchange git refs over multicast?
+
+When you run: `git annex multicast --send`, will that work for all git-annex blobs, or just the ones that are in the local checkout?
+
+Looking at the [server usage](http://uftp-multicast.sourceforge.net/server_usage.txt), i see that multicast is the default, but that you can also unicast to specific hosts:
+
+       Or in unicast mode:
+
+	    uftp -M host_or_ip file
+
+       Where host_or_ip is the hostname or unicast IP address of the  host  to
+       send to.
+
+       To send only to certain hosts:
+
+	    uftp -H client_id_1,client_id_2,client_id_3 file_to_send
+
+       or:
+
+	    uftp -H @file_containing_list_of_clients file_to_send
+
+Could this be used to send to arbitrary, non-local hosts on the internet?
+
+And how about tor support? Could this simplify the encrypted setup?
+
+This amazing feature just brings up more questions for me, but i'm really glad to see this come. It certainly addresses parts of [[todo/Bittorrent-like_features]], and is probably the first remote that supports bandwidth optimizations for multiple downloads, or rather, in this case, uploads. :)
+
+Thanks so much for your hard work!
+The feature set of git-annex never ceases to amaze me - new features just keep on coming, this is great!
+"""]]

devblog
diff --git a/doc/devblog/day_454__multicast.mdwn b/doc/devblog/day_454__multicast.mdwn
new file mode 100644
index 0000000..1972c5e
--- /dev/null
+++ b/doc/devblog/day_454__multicast.mdwn
@@ -0,0 +1,15 @@
+Earlier this week I had the opportunity to sit in on a workshop at MIT where
+students were taught how to use git-annex as part of a stack of tools for
+reproducible scientific data research. That was great!
+
+One thing we noticed there is, it can be hard to distribute files to such a
+class; downloading them individually wastes network bandwidth. Today, I
+added [[git annex multicast|git-annex-multicast]] which uses `uftp`
+to multicast files to other clones of a repository on a LAN.
+An "easy" 500 lines of code and 7 hour job.
+
+There is encryption and authentication, but the key management for this
+turned out to be simple, since the public key fingerprints can be stored on
+the git-annex branch, and easily synced around that way. So, I expect
+this should be not hard to use in a classroom setting such as the one
+I was in earlier this week.

multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting.
This commit was supported by the NSF-funded DataLad project.
diff --git a/Annex/Multicast.hs b/Annex/Multicast.hs
new file mode 100644
index 0000000..16aa1bd
--- /dev/null
+++ b/Annex/Multicast.hs
@@ -0,0 +1,41 @@
+{- git-annex multicast receive callback
+ -
+ - Copyright 2017 Joey Hess <id@joeyh.name>
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Annex.Multicast where
+
+import Config.Files
+import Utility.Env
+import Utility.PartialPrelude
+
+import System.Process
+import System.IO
+import GHC.IO.Handle.FD
+
+multicastReceiveEnv :: String
+multicastReceiveEnv = "GIT_ANNEX_MULTICAST_RECEIVE"
+
+multicastCallbackEnv :: IO (FilePath, [(String, String)], Handle)
+multicastCallbackEnv = do
+	gitannex <- readProgramFile
+	(rfd, wfd) <- createPipeFd
+	rh <- fdToHandle rfd
+	environ <- addEntry multicastReceiveEnv (show wfd) <$> getEnvironment
+	return (gitannex, environ, rh)
+
+-- This is run when uftpd has received a file. Rather than move
+-- the file into the annex here, which would require starting up the
+-- Annex monad, parsing git config, and verifying the content, simply
+-- output to the specified FD the filename. This keeps the time
+-- that uftpd is not receiving the next file as short as possible.
+runMulticastReceive :: [String] -> String -> IO ()
+runMulticastReceive ("-I":_sessionid:fs) hs = case readish hs of
+	Just fd -> do
+		h <- fdToHandle fd
+		mapM_ (hPutStrLn h) fs
+		hClose h
+	Nothing -> return ()
+runMulticastReceive _ _ = return ()
diff --git a/CHANGELOG b/CHANGELOG
index c58c3cf..f69aade 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -2,6 +2,8 @@ git-annex (6.20170322) UNRELEASED; urgency=medium
 
   * When a http remote does not expose an annex.uuid config, only warn
     about it once, not every time git-annex is run.
+  * multicast: New command, uses uftp to multicast annexed files, for eg
+    a classroom setting.
 
  -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 
diff --git a/CmdLine/GitAnnex.hs b/CmdLine/GitAnnex.hs
index 0e47200..be5f56b 100644
--- a/CmdLine/GitAnnex.hs
+++ b/CmdLine/GitAnnex.hs
@@ -14,6 +14,7 @@ import CmdLine
 import Command
 import Utility.Env
 import Annex.Ssh
+import Annex.Multicast
 import Types.Test
 
 import qualified Command.Help
@@ -53,6 +54,7 @@ import qualified Command.Describe
 import qualified Command.InitRemote
 import qualified Command.EnableRemote
 import qualified Command.EnableTor
+import qualified Command.Multicast
 import qualified Command.Expire
 import qualified Command.Repair
 import qualified Command.Unused
@@ -144,6 +146,7 @@ cmds testoptparser testrunner =
 	, Command.InitRemote.cmd
 	, Command.EnableRemote.cmd
 	, Command.EnableTor.cmd
+	, Command.Multicast.cmd
 	, Command.Reinject.cmd
 	, Command.Unannex.cmd
 	, Command.Uninit.cmd
@@ -242,4 +245,5 @@ run testoptparser testrunner args = go envmodes
 	envmodes =
 		[ (sshOptionsEnv, runSshOptions args)
 		, (sshAskPassEnv, runSshAskPass)
+		, (multicastReceiveEnv, runMulticastReceive args)
 		]
diff --git a/Command/Multicast.hs b/Command/Multicast.hs
new file mode 100644
index 0000000..cd74c3e
--- /dev/null
+++ b/Command/Multicast.hs
@@ -0,0 +1,253 @@
+{- git-annex command
+ -
+ - Copyright 2017 Joey Hess <id@joeyh.name>
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+{-# LANGUAGE CPP #-}
+
+module Command.Multicast where
+
+import Command
+import Logs.Multicast
+import Annex.Multicast
+import Annex.WorkTree
+import Annex.Content
+import Annex.UUID
+#ifndef mingw32_HOST_OS
+import Creds
+import Annex.Perms
+import Utility.FileMode
+#endif
+import qualified Limit
+import Types.FileMatcher
+import qualified Git.LsFiles as LsFiles
+import Utility.Hash
+import Utility.Tmp
+import Config
+
+import Data.Char
+import qualified Data.ByteString.Lazy.UTF8 as B8
+import qualified Data.Map as M
+import Control.Concurrent.Async
+
+cmd :: Command
+cmd = command "multicast" SectionCommon "multicast file distribution"
+	paramNothing (seek <$$> optParser)
+
+data MultiCastAction
+	= GenAddress
+	| Send
+	| Receive
+	deriving (Show)
+
+data MultiCastOptions = MultiCastOptions MultiCastAction [CommandParam] [FilePath]
+	deriving (Show)
+
+optParser :: CmdParamsDesc -> Parser MultiCastOptions
+optParser _ = MultiCastOptions 
+	<$> (genaddressp <|> sendp <|> receivep)
+	<*> many uftpopt
+	<*> cmdParams paramPaths
+  where
+	genaddressp = flag' GenAddress
+		( long "gen-address"
+		<> help "generate multicast encryption key and store address in git-annex branch"
+		)
+	sendp = flag' Send
+		( long "send"
+		<> help "multicast files"
+		)
+	receivep = flag' Receive
+		( long "receive"
+		<> help "listen for multicast files and store in repository"
+		)
+	uftpopt = Param <$> strOption
+		( long "uftp-opt"
+		<> short 'U'
+		<> help "passed on to uftp/uftpd"
+		<> metavar "OPTION"
+		)
+
+seek :: MultiCastOptions -> CommandSeek
+seek (MultiCastOptions GenAddress _ _) = commandAction genAddress 
+seek (MultiCastOptions Send ups fs) = commandAction $ send ups fs
+seek (MultiCastOptions Receive ups []) = commandAction $ receive ups
+seek (MultiCastOptions Receive _ _) = giveup "Cannot specify list of files with --receive; this receives whatever files the sender chooses to send."
+
+genAddress :: CommandStart
+genAddress = do
+	showStart "gen-address" ""
+	k <- uftpKey
+	(s, ok) <- case k of
+		KeyContainer s -> liftIO $ genkey (Param s)
+		KeyFile f -> do
+			createAnnexDirectory (takeDirectory f)
+			liftIO $ nukeFile f
+			liftIO $ protectedOutput $ genkey (File f)
+	case (ok, parseFingerprint s) of
+		(False, _) -> giveup $ "uftp_keymgt failed: " ++ s
+		(_, Nothing) -> giveup $ "Failed to find fingerprint in uftp_keymgt output: " ++ s
+		(True, Just fp) -> next $ next $ do
+			recordFingerprint fp =<< getUUID
+			return True
+  where
+ 	-- Annoyingly, the fingerprint is output to stderr.
+	genkey p = processTranscript "uftp_keymgt" ps Nothing
+	  where
+		ps = toCommand $
+			[ Param "-g"

(Diff truncated)
fix format
diff --git a/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net/comment_1_5353dc46ccecc6077f2643d281d58f99._comment b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net/comment_1_5353dc46ccecc6077f2643d281d58f99._comment
index 5c932cb..a38457d 100644
--- a/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net/comment_1_5353dc46ccecc6077f2643d281d58f99._comment
+++ b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net/comment_1_5353dc46ccecc6077f2643d281d58f99._comment
@@ -89,3 +89,4 @@ The process of setting up a multicast classroom would then be:
    files going by and stores them.
 5. Once the students are listening (*ahem*), teacher runs
    `git annex multicast-send [file]` to distribute files.
+"""]]

design
diff --git a/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net/comment_1_5353dc46ccecc6077f2643d281d58f99._comment b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net/comment_1_5353dc46ccecc6077f2643d281d58f99._comment
new file mode 100644
index 0000000..5c932cb
--- /dev/null
+++ b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net/comment_1_5353dc46ccecc6077f2643d281d58f99._comment
@@ -0,0 +1,91 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-03-30T16:51:24Z"
+ content="""
+Using a remote for this seems problimatic, because the remote is not
+pointing at a single well-defined data store, but instead whatever peers
+happen to exist on the network.
+
+For one, `copy --to=udp-multicast` would not try to send files that it thinks
+are already located in that remote. I suppose this could be dealt with by
+making the transfers always seem to fail, so it never thinks that the
+multicast remote has any files.
+
+But then, `copy --from=udp-multicast` would not try to receive files,
+unless it thinks they're in that remote. And we just established it should
+not think any files are in that remote. So that's a problem.
+
+Also, the copy from/to would need to be operating on the same file at the
+same time, which seems problimatic. If a receiving git-annex is a little
+slower than the sender, or is operating on a slightly different set of
+files, it would then miss a file being broadcast by the sender.
+
+These issues seem to point to this needing to use some other, more
+special-purpose commands for muticast.
+
+----
+
+It probably needs encryption, both for privacy and to ensure that files
+are being received from the sender you intended, and not someone else
+who might be broadcasting the contents of a different repository.
+
+Here's how to set up encryption and authentication with uftp,
+so that both client and server actually encrypt and check that they're
+talking with a known entity. It took a while to figure out.
+
+Client:
+
+	uftp_keymgt -g rsa:512 ~/client_key
+	# Parse the fingerprint value from its output; does not
+	# seem to be a better way, except perhaps using openssl to examine
+	# the key file. This is CLIENT_FINGERPRINT
+
+	# Pick a UID for the client. This is an 8-diget hex number,
+	# which needs to be unique across clients. Default is based on IP
+	# adddres, but for git-annex it could be derived from the git-annex
+	# UUID.	This is CLIENT_UID.
+
+Server:
+
+	uftp_keymgt -g rsa:512 ~/servant_key
+	# Parse the SERVER_FINGERPRINT from its output.
+	
+	# Pick a SERVER_UID for the server.
+
+Client:
+
+	# create a file "authlist" that contains "$SERVER_UID|$SERVER_FINGERPRINT"
+
+	uftpd -E -d -D/tmp/xx -k ~/client_key -U $CLIENT_UID -S '@authlist'
+
+Server:
+
+	# create file "authlist" that contains "$CLIENT_UID|$CLIENT_FINGERPRINT"
+	# lines for each allowed client
+
+	uftp -c -Y aes256-cbc -h sha1 -k ~/server_key -U $SERVER_UID -H '@authlist' file_to_send
+
+----
+
+Notice that the client and server UID and key generation steps above 
+are the same. So, a command like `git annex multicast --gen-address`
+could be run on both the server and clients, and could store
+the addresses in the git-annex branch. 
+The uftp authlist file would be derived from all known such addresses.
+
+(Unlike `p2p --gen-address`, where the address allows connecting with
+and authentication with a remote over TOR, these multicast addresses
+are safe to make public.)
+
+The process of setting up a multicast classroom would then be:
+
+1. Teacher runs `git annex multicast --gen-address; git annex sync`
+2. Students each run `git annex multicast --gen-address; git annex sync`
+3. Teacher runs `git annex sync` once all the students have generated addresses.  
+   (Now the students all have received the teacher's address, and the teacher
+   has received all the student's addresses.)
+4. Students each run `git annex multicast-receive`, which listens for 
+   files going by and stores them.
+5. Once the students are listening (*ahem*), teacher runs
+   `git annex multicast-send [file]` to distribute files.

diff --git a/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net.mdwn b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net.mdwn
index c2894ef..fa240bd 100644
--- a/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net.mdwn
+++ b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net.mdwn
@@ -1,6 +1,6 @@
 Hi Joey,
 
-Although you haven't remembered that "hash thing" to perform the job, we looked around for other ways to accomplish the mission:  quickly/simultaneously distribute annexed files to multiple hosts on the same network.  So, there are such tools as uftp to efficiently multicast files to multiple recipients.  Initial setup takes a bit but I wondered how feasible it could be to e.g. establish some "custom annex remote" (if to avoid coming up with a new command eg. "annex serve") so e.g. I could e.g. `git annex copy --to=udp-multicast ` on the host A which has all the keys, and then run `git annex get --from=udp-multicast` on the clients (B) which want to get it all.  To make it even more efficient, A could may be provide/announce on the local net a service to receive requests for desired keys to be transmitted.  But even if it just multicasts all the content of the current tree, while those clients suck them all in, it could be super useful.
+Although I haven't remembered that "hash thing" to perform the job, we looked around for other ways to accomplish the mission:  quickly/simultaneously distribute annexed files to multiple hosts on the same network.  So, there are such tools as uftp to efficiently multicast files to multiple recipients.  Initial setup takes a bit but I wondered how feasible it could be to e.g. establish some "custom annex remote" (if to avoid coming up with a new command eg. "annex serve") so e.g. I could e.g. `git annex copy --to=udp-multicast ` on the host A which has all the keys, and then run `git annex get --from=udp-multicast` on the clients (B) which want to get it all.  To make it even more efficient, A could may be provide/announce on the local net a service to receive requests for desired keys to be transmitted.  But even if it just multicasts all the content of the current tree, while those clients suck them all in, it could be super useful.
 
 What do you think?
 

initial idea
diff --git a/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net.mdwn b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net.mdwn
new file mode 100644
index 0000000..c2894ef
--- /dev/null
+++ b/doc/todo/multicast___34__broadcasting__34___of_content_on_local_net.mdwn
@@ -0,0 +1,7 @@
+Hi Joey,
+
+Although you haven't remembered that "hash thing" to perform the job, we looked around for other ways to accomplish the mission:  quickly/simultaneously distribute annexed files to multiple hosts on the same network.  So, there are such tools as uftp to efficiently multicast files to multiple recipients.  Initial setup takes a bit but I wondered how feasible it could be to e.g. establish some "custom annex remote" (if to avoid coming up with a new command eg. "annex serve") so e.g. I could e.g. `git annex copy --to=udp-multicast ` on the host A which has all the keys, and then run `git annex get --from=udp-multicast` on the clients (B) which want to get it all.  To make it even more efficient, A could may be provide/announce on the local net a service to receive requests for desired keys to be transmitted.  But even if it just multicasts all the content of the current tree, while those clients suck them all in, it could be super useful.
+
+What do you think?
+
+[[!meta name=yoh]]

Added a comment: onion-grater
diff --git a/doc/tips/peer_to_peer_network_with_tor/comment_3_9fc514cf5273dd4a29dfb3ab81346425._comment b/doc/tips/peer_to_peer_network_with_tor/comment_3_9fc514cf5273dd4a29dfb3ab81346425._comment
new file mode 100644
index 0000000..5834959
--- /dev/null
+++ b/doc/tips/peer_to_peer_network_with_tor/comment_3_9fc514cf5273dd4a29dfb3ab81346425._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="anarcat"
+ avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
+ subject="onion-grater"
+ date="2017-03-30T14:49:12Z"
+ content="""
+you may want to consider using [onion-grater][] to limit possible escalations in the use of that control port:
+
+ [onion-grater]: https://github.com/Whonix/onion-grater
+
+RFP: [[!debbug 859125]]
+"""]]

Added a comment: autostart and foreground together doesn't seem to work
diff --git a/doc/tips/Systemd_unit/comment_1_06399a293f08401032bef1b94f05547c._comment b/doc/tips/Systemd_unit/comment_1_06399a293f08401032bef1b94f05547c._comment
new file mode 100644
index 0000000..7af170e
--- /dev/null
+++ b/doc/tips/Systemd_unit/comment_1_06399a293f08401032bef1b94f05547c._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="oberix@c7a19cddb1663df0c612a979b9d13b0d67f1f69a"
+ nickname="oberix"
+ avatar="http://cdn.libravatar.org/avatar/e8b871f3d0bf96df9a3fc8cdca7abe09"
+ subject="autostart and foreground together doesn't seem to work"
+ date="2017-03-30T10:43:18Z"
+ content="""
+With systemd using `--autostart --foreground` either ignore foreground or quit immediatelly.
+
+I managed to have the process stay alive with `RemainAfterExit=on`:
+
+    [Service]
+    User=%i
+    ExecStart=/usr/bin/git-annex assistant --autostart --foreground
+    ExecStop=/usr/bin/git-annex assistant --autostop
+    RemainAfterExit=on
+    Restart=on-failure
+    RestartSec=5
+
+but git-annex processes does not maintain the `--foreground` option which is causing a lot of zombies in the long period (not totally clear why).
+
+My current solution is to have a service for each annex repository and avoid `--autosart` but this is annoying because it require to pass the path as `%I` and wrap git-annex in bash script to get the repo owner as the user.
+
+
+"""]]

diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest.mdwn b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest.mdwn
new file mode 100644
index 0000000..f136b25
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest.mdwn
@@ -0,0 +1,28 @@
+### Please describe the problem.
+
+every sync (without --content) to a gcrypt remote uploads huge amount of data (>100MB) when doing
+
+    gcrypt: Requesting manifest signature
+
+It seems to upload a special git object every time, even if this object is apparently unchanged. An unencrypted, regular git remote is much faster and does not transfer such amounts of data.
+
+I wonder if this can be changed, because it renders that gcrypt remote almost unusable via ADSL upstream. In my case the sync duration was 36 Minutes, uploading ~250MB.
+
+### What steps will reproduce the problem?
+
+Have a (bare) gcrypt remote and a rather big (mine has 77668 keys, annexing 769GB of files) git-annex repository. Sync with the gcrypt remote. When pushing, the message "gcrypt: Requesting manifest signature" appears, and a very large amount of data is transferred to the remote, while the process chain
+
+    git-remote-gcrypt mygcrypt ssh://mygcrypt/home/my/annex
+     git push --quiet -f ssh://mygcrypt/home/my/annex refs/gcrypt/gitception+:refs/heads/master
+      ssh mygcrypt git-receive-pack '/home/my/annex'
+       git pack-objects --all-progress-implied --revs --stdout --thin --delta-base-offset -q
+
+is running. The upload of the actual changeset starts after this, the processes look the same, transferring again a more or less big amount of data (depending on the changeset size, I guess).
+
+### What version of git-annex are you using? On what operating system?
+
+6.20170101-1 on Debian Stretch (9.0)
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+git-annex is great and revolutionized my file organization and backup structure (if they were even existing before)

When a http remote does not expose an annex.uuid config, only warn about it once, not every time git-annex is run.
Same behavior as for a ssh remote.
diff --git a/CHANGELOG b/CHANGELOG
index 2b467e7..c58c3cf 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,8 +1,9 @@
 git-annex (6.20170322) UNRELEASED; urgency=medium
 
-  * stack.yaml: Update to lts-8.6.
+  * When a http remote does not expose an annex.uuid config, only warn
+    about it once, not every time git-annex is run.
 
- -- Joey Hess <id@joeyh.name>  Mon, 27 Mar 2017 19:48:31 -0400
+ -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 
 git-annex (6.20170321) unstable; urgency=medium
 
diff --git a/Remote/Git.hs b/Remote/Git.hs
index e5d85d2..78213f4 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -248,18 +248,15 @@ tryGitConfigRead autoinit r
 				, return Nothing
 				)
 		case v of
-			Nothing -> do
-				warning $ "Failed to get annex.uuid configuration of repository " ++ Git.repoDescribe r
-				return r
-			Just (Left _) -> do
-				set_ignore "not usable by git-annex" False
-				return r
 			Just (Right r') -> do
 				-- Cache when http remote is not bare for
 				-- optimisation.
 				unless (Git.Config.isBare r') $
 					setremote setRemoteBare False
 				return r'
+			_ -> do
+				set_ignore "not usable by git-annex" False
+				return r
 
 	store = observe $ \r' -> do
 		g <- gitRepo
diff --git a/doc/bugs/http_git_remote_uuid_discovery_repeated.mdwn b/doc/bugs/http_git_remote_uuid_discovery_repeated.mdwn
index a158996..7b3aa03 100644
--- a/doc/bugs/http_git_remote_uuid_discovery_repeated.mdwn
+++ b/doc/bugs/http_git_remote_uuid_discovery_repeated.mdwn
@@ -1,3 +1,5 @@
 When cloning eg <https://github.com/datalad/ds000114>, each time git-annex
 is run, it tries to get the uuid, fails, prints a warning. It should set
 annex-ignore instead, so that only happens once. --[[Joey]]
+
+> [[fixed|done]] --[[Joey]] 

diff --git a/doc/forum/library__44___backup_and_music_player.mdwn b/doc/forum/library__44___backup_and_music_player.mdwn
new file mode 100644
index 0000000..0248998
--- /dev/null
+++ b/doc/forum/library__44___backup_and_music_player.mdwn
@@ -0,0 +1,11 @@
+Hello, I'm not sure on how to proceed with a fairly simple workflow. I keep my music library in a debian box with plenty of space. I run the plex media server on it so it should be a direct repository to avoid strange player behavior.
+I also perform a monthly (more or less) backup on an external hard drive. This should be simple too, just add a remote and sync.
+My problem now is: I have also a portable music player that I want to keep in sync with the music library. Not the whole music library but only a small subset. This needs to be a direct repository too (the best would be also without the .git/ folder but I don't believe it's possible). How to create a small subset of files to keep in sync with a direct repository? I've tried with branches but they are not simple to use in direct mode.
+
+So, to recap: a (direct) main repository, a standard backup repository and another (direct) repository with only a subset of files.
+
+Anyone with a similar workflow wants to share some tips?
+
+Thanks,
+
+F

WSL can now run git-annex
diff --git a/doc/todo/windows_support.mdwn b/doc/todo/windows_support.mdwn
index deb0e1e..5389011 100644
--- a/doc/todo/windows_support.mdwn
+++ b/doc/todo/windows_support.mdwn
@@ -1,17 +1,6 @@
 The git-annex Windows port is beta, but rapidly becoming polished and
 usable!
 
-## do we need this port anymore?
-
-See <http://blog.dustinkirkland.com/2016/03/ubuntu-on-windows.html>
-
-If windows has transparent support for running linux executables, and those
-executables can access files in "." which are on the windows system, then
-you could just use this to run linux git-annex on windows. No port needed.
-
-That would be great!
-
-Seems like this would need Windows 10.
 
 ## status
 
@@ -84,3 +73,43 @@ seems unreliable/broken on Windows.
   it and files can be transferred to it and back
 * Does stopping in progress transfers work in the webapp?
 
+## do we need this port anymore?
+
+See <http://blog.dustinkirkland.com/2016/03/ubuntu-on-windows.html>
+
+If windows has transparent support for running linux executables, and those
+executables can access files in "." which are on the windows system, then
+you could just use this to run linux git-annex on windows. No port needed.
+
+That would be great!
+
+Seems like this would need Windows 10.
+
+> The latest builds of Windows 10 (build 15063) can run git-annex in the
+> Windows Subsystem for Linux. After following the instructions at
+> <https://msdn.microsoft.com/en-us/commandline/wsl/about>, run:
+> `sudo apt-get install git-annex`
+> 
+> git-annex in WSL passes its full test suite, and it avoids all
+> the problems discussed in sections above.
+> 
+> git-annex can access Windows files in eg `/mnt/c`, so a git-annex
+> repository can be stored there. However, if the git-annex repository uses
+> indirect mode, the symlinks used by git-annex won't be usable by Windows
+> programs. Use either direct mode, or v6 mode to avoid the symlink
+> problem.
+> 
+> Also, see this important caveat:
+> <https://blogs.msdn.microsoft.com/commandline/2016/11/17/do-not-change-linux-files-using-windows-apps-and-tools/>
+> 
+> WSL is currently rather annoying to enable. *If* it became easy enough
+> to enable, note that "bash -c git-annex" works from a windows command
+> prompt, and would probably work in a .bat file as well, so git-annex from
+> the WSL could be transparently used on the windows side.
+> 
+> The webapp does not currently work. It doesn't know how to open a web
+> browser from the linux side. There are also what look like some emulation
+> problems around the daemonization code. `git annex assistant
+> --foreground` does run, but while it notices when new files are added, it
+> does not notice when existing files get modified. Probably an inotify
+> emulation bug. --[[Joey]]

expand
diff --git a/doc/todo/export.mdwn b/doc/todo/export.mdwn
index bb7aa37..e729b0c 100644
--- a/doc/todo/export.mdwn
+++ b/doc/todo/export.mdwn
@@ -1,7 +1,9 @@
 `git annex export` corresponding to import. This might be useful for eg,
 datalad. There are some requests to make eg a S3 bucket mirror the
-filenames in the git annex repository, which seem out of scope, but
-something simpler like this could be worth doing.
+filenames in the git annex repository with incremental updates, 
+which seem out of scope (and there are many tools to do stuff like that
+search "deploy files to S3 bucket"), 
+but something simpler like `git annex export` could be worth doing.
 
 `git annex export --to remote files` would copy the files to the remote,
 using the names in the working tree. For remotes like S3, it could add the

idea
diff --git a/doc/todo/export.mdwn b/doc/todo/export.mdwn
new file mode 100644
index 0000000..bb7aa37
--- /dev/null
+++ b/doc/todo/export.mdwn
@@ -0,0 +1,14 @@
+`git annex export` corresponding to import. This might be useful for eg,
+datalad. There are some requests to make eg a S3 bucket mirror the
+filenames in the git annex repository, which seem out of scope, but
+something simpler like this could be worth doing.
+
+`git annex export --to remote files` would copy the files to the remote,
+using the names in the working tree. For remotes like S3, it could add the
+url of the exported file, so that another clone of the repo could use the
+exported data.
+
+Would this be able to reuse the existing `storeKey` interface, or would
+there need to be a new interface in supported remotes?
+
+--[[Joey]]

add annex-sync setting sample
diff --git a/doc/tips/semi-synchronized_remotes.mdwn b/doc/tips/semi-synchronized_remotes.mdwn
index c38177b..c91eabf 100644
--- a/doc/tips/semi-synchronized_remotes.mdwn
+++ b/doc/tips/semi-synchronized_remotes.mdwn
@@ -77,8 +77,10 @@ I need some special setting. There are the options I considered, in
 [.gitconfig](https://manpages.debian.org/git-config.1.en.html) or [[git-annex]]'s config options:
 
  * `remote.<name>.annex-ignore=true`: `sync` and `assistant` will not
-   sync to the repository, but explicit `get --from=repoB` will still
-   work. unclear if `sync repoB` will also push.
+   sync *content* to the repository, but explicit `get --from=repoB`
+   will still work.
+ * `remote.<name>.annex-sync=false`: `sync` (and `assistant`?) will
+   not sync the git repository with the remote
  * `remote.<name>.push=nothing`: git won't push by default, unless
    branches are explicitly given, which may actually be the case for
    git-annex, so unlikely to work.
@@ -94,12 +96,34 @@ I need some special setting. There are the options I considered, in
    instead. crude hack and may confuse the hell out of git-annex, but
    at least doesn't yield errors.
 
-I've settled for the `pushurl=/dev/null` hack for now. A similar
-approach is to make `repoB` read-only to the user. This however, may
-trigger the activation of `annex-ignore` by git-annex and will
-otherwise yield the same warnings as the `pushurl=/dev/null` hack.
-
-Therefore, the best approach may be to have git-annex respect the
+A similar approach to hacking the `pushurl` is to make `repoB`
+read-only to the user. This however, may trigger the activation of
+`annex-ignore` by git-annex and will otherwise yield the same warnings
+as the `pushurl=/dev/null` hack.
+
+Right now, I am using `annex-sync = false` in `.git/config`. I have
+also configured the repository to be in the "manual" [[standard
+group|preferred_content/standard_groups]] which will avoid copying
+files into that repository:
+
+    $ git annex group repoB manual
+    group repoB ok
+    (recording state in git...)
+    $ git annex wanted repoB standard
+    wanted repoB ok
+    (recording state in git...)
+
+This is roughly equivalent to setting `annex-ignore = true`, yet it
+allows for more flexibility. I could, for example, create custom
+content expressions to sync certain folders automatically.
+
+A disadvantage of the `annex-sync` settings is that it affects both
+ways (push and pull), not just push, which is what I am interested
+in. Although it could be argued that restricting both is fine here
+because we want to manually review changes when we pull changes from
+those remotes anyways.
+
+The best approach may be to have git-annex respect the
 `remote.<name>.push=nothing` setting. Another approach would be to add
 `remote.<name>.annex-push` and `remote.<name>.annex-pull` settings
 that would match the `sync --[no-]push --[no-]pull` flags.

bug report from Hands-on Reproducible and Scalable Brain Imaging Analysis with Nipype
diff --git a/doc/bugs/http_git_remote_uuid_discovery_repeated.mdwn b/doc/bugs/http_git_remote_uuid_discovery_repeated.mdwn
new file mode 100644
index 0000000..a158996
--- /dev/null
+++ b/doc/bugs/http_git_remote_uuid_discovery_repeated.mdwn
@@ -0,0 +1,3 @@
+When cloning eg <https://github.com/datalad/ds000114>, each time git-annex
+is run, it tries to get the uuid, fails, prints a warning. It should set
+annex-ignore instead, so that only happens once. --[[Joey]]

link to torrent page
diff --git a/doc/tips/semi-synchronized_remotes.mdwn b/doc/tips/semi-synchronized_remotes.mdwn
index fcd8952..c38177b 100644
--- a/doc/tips/semi-synchronized_remotes.mdwn
+++ b/doc/tips/semi-synchronized_remotes.mdwn
@@ -104,5 +104,11 @@ Therefore, the best approach may be to have git-annex respect the
 `remote.<name>.annex-push` and `remote.<name>.annex-pull` settings
 that would match the `sync --[no-]push --[no-]pull` flags.
 
+Note that this is similar in concept to
+[[todo/Bittorrent-like_features]], although here we assumes you
+already have some transport to share anything you need, yet still have
+to address the question of semi-synchronized git repositories in some
+way.
+
 I would obviously welcome additional comments and questions on this
 approach. -- [[anarcat]]

some issue i have come up with a few times and workarounds
diff --git a/doc/tips/semi-synchronized_remotes.mdwn b/doc/tips/semi-synchronized_remotes.mdwn
new file mode 100644
index 0000000..fcd8952
--- /dev/null
+++ b/doc/tips/semi-synchronized_remotes.mdwn
@@ -0,0 +1,108 @@
+In general, git-annex repositories that are "synchronized" (e.g. with
+the [[git-annex-sync]] command, whatever the backend) have a global
+namespace. Repositories will eventually converge to have very exactly
+the same content, generally using git's push/pull/merge
+mechanisms.
+
+What if we do *not* wish to exactly have the same content across all
+repositories, but still want to share some objects?
+
+An example use case here is content (e.g. `.git/annex/objects` blobs)
+sharing, without having to deliberately collaborate over a globally
+consistent set of objects in the `master` branch. Think of a
+decentralized [conference proceedings][] repository where each
+conference could add their own content to a conference-specific
+repository, while at the same time allowing a unified view in another,
+more centralized repository, or allowing users to pick and choose
+which conference they would want content from.
+
+ [conference proceedings]: https://github.com/RichiH/conference_proceedings
+
+While each repository could have its own distinct branch, all
+repositories will see all those branches and this may affect content
+retention, as git-annex may consider files to be "in use" because they
+are on some remote branch, for example. Furthermore, I consider git
+branching to be a rather advanced topic in git usage. While git-annex
+uses those mechanisms (e.g. the `git-annex` and `sync/*` branches),
+those are generally hidden from the user until something goes
+wrong. Therefore I looked into providing a more straightforward
+approach to this problem for my users and myself.
+
+In my use case, I have the following repositories:
+
+* repoA: my own curated media collection
+* repoB: a third-party media collection
+
+I do not wish for my local curated collection (repoA) to be completely
+synchronized with the third-party collection (repoB). This is because
+we may have different tastes and retention policies: while I archive
+everything, there are certain media I am not interested in. On the
+other hand repoB might keep only (say) the last month of media and
+disard older content but have a more varied collection, which only a
+subset is interesting to me. Yet I still want to access some of that
+content!
+
+So I did the following to add the third party repository:
+
+    git remote add repoB example.net:repoB
+    git annex sync --no-push repoB
+    git annex get --from=repoB
+
+This works well: I get the files from repoB locally. Of course, if
+repoB expires some files, this will be impacted locally, but I can
+always revert those choices without conflict, because I do not push
+those back.
+
+The downside of the `--no-push` option in [[git-annex-sync]] is that
+it needs to be made explicit at each invocation of the
+command. Furthermore, this option is not supported by the assistant,
+which will happily sync the master branch to all remotes by default.
+
+An alternative is to manually fetch and merge content:
+
+    git fetch repoB
+    git annex merge repoB
+    git reset HEAD^
+    # revert any possible changes upstream we don't want
+    git commit
+
+Needless to say this quickly becomes quite messy, but it's the amazing
+level of control git and git-annex provides, which obviously comes
+with its price in complexity. Such a method will also be ignored by
+the assistant and further `sync` commands.
+
+To make sure those principles are respected in the assistant or a
+plain `git annex sync` that may mistakenly be ran in that repository,
+I need some special setting. There are the options I considered, in
+[.gitconfig](https://manpages.debian.org/git-config.1.en.html) or [[git-annex]]'s config options:
+
+ * `remote.<name>.annex-ignore=true`: `sync` and `assistant` will not
+   sync to the repository, but explicit `get --from=repoB` will still
+   work. unclear if `sync repoB` will also push.
+ * `remote.<name>.push=nothing`: git won't push by default, unless
+   branches are explicitly given, which may actually be the case for
+   git-annex, so unlikely to work.
+ * `remote.<name>.pushurl=/dev/null`: will completely disable any push
+   functionality to that remote. any sync will yield the following
+   error:
+   
+       fatal: '/dev/null' does not appear to be a git repository
+       [...]
+       git-annex: sync: 1 failed
+
+ * `remote.<name>.pushurl=.`: will push to the local repo
+   instead. crude hack and may confuse the hell out of git-annex, but
+   at least doesn't yield errors.
+
+I've settled for the `pushurl=/dev/null` hack for now. A similar
+approach is to make `repoB` read-only to the user. This however, may
+trigger the activation of `annex-ignore` by git-annex and will
+otherwise yield the same warnings as the `pushurl=/dev/null` hack.
+
+Therefore, the best approach may be to have git-annex respect the
+`remote.<name>.push=nothing` setting. Another approach would be to add
+`remote.<name>.annex-push` and `remote.<name>.annex-pull` settings
+that would match the `sync --[no-]push --[no-]pull` flags.
+
+I would obviously welcome additional comments and questions on this
+approach. -- [[anarcat]]

diff --git a/doc/bugs/git-annex-sync_sometimes_fails_in_submodule_in_V6_adjusted_branch.mdwn b/doc/bugs/git-annex-sync_sometimes_fails_in_submodule_in_V6_adjusted_branch.mdwn
new file mode 100644
index 0000000..8a188bb
--- /dev/null
+++ b/doc/bugs/git-annex-sync_sometimes_fails_in_submodule_in_V6_adjusted_branch.mdwn
@@ -0,0 +1,51 @@
+### Please describe the problem.
+I have a V6 annex in adjusted branch with a submodule, which also is a V6 annex in adjusted branch.
+There staged changes in the superproject, while the submodule is clean. Calling git-annex-sync in the superproject now leads to:
+
+[[!format sh """
+% git annex sync
+commit  (recording state in git...)
+
+[adjusted/master(unlocked) 3c8d350] git-annex in me@mycomputer:/tmp/datalad_temp_test_AnnexRepo_statusGmqM5E
+ 4 files changed, 4 insertions(+), 2 deletions(-)
+ create mode 100644 fifth
+ create mode 100644 sub/third
+ok
+fatal: entry 'submod' object type (blob) doesn't match mode type (commit)
+
+git-annex: user error (git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","mktree","--batch","-z"] exited 128)
+failed
+git-annex: sync: 1 failed
+"""]]
+
+### What steps will reproduce the problem?
+
+Sadly I wasn't able to reproduce it outside the datalad test it is happening in.
+I will provide the steps to achieve that situation, whenever I figured them out.
+But may be just the error message is sufficient to address the problem ...
+
+### What version of git-annex are you using? On what operating system?
+[[!format sh """
+% git annex version
+git-annex version: 6.20170307+gitg24ade8a25-1~ndall+1
+build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+local repository version: 6
+supported repository versions: 3 5 6
+upgrade supported from repository 
+"""]]
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+
+[[ben]]

diff --git a/doc/bugs/committing_files_into_git_doesn__39__t_work_with_explicitly_given_paths_in_V6.mdwn b/doc/bugs/committing_files_into_git_doesn__39__t_work_with_explicitly_given_paths_in_V6.mdwn
new file mode 100644
index 0000000..0f160bb
--- /dev/null
+++ b/doc/bugs/committing_files_into_git_doesn__39__t_work_with_explicitly_given_paths_in_V6.mdwn
@@ -0,0 +1,47 @@
+### Please describe the problem.
+If there are staged changes in two files (one in git, one in annex) and I want to commit them by explicitly giving git-commit both paths (in order to exclude other changes that are possibly staged), the not annexed file stays staged, while committing everything staged by not giving any path to git-commit works just fine.
+
+### What steps will reproduce the problem?
+[[!format sh """
+mkdir testv6
+cd testv6
+git init
+git annex init --version=6
+echo some > file_in_git
+git -c annex.largefiles=nothing annex add file_in_git
+echo more > file_in_annex
+git annex add file_in_annex
+git commit -m "files added" file_in_git file_in_annex
+git status
+"""]]
+
+
+### What version of git-annex are you using? On what operating system?
+[[!format sh """
+% git annex version  
+git-annex version: 6.20170307+gitg24ade8a25-1~ndall+1
+build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+local repository version: 6
+supported repository versions: 3 5 6
+upgrade supported from repository versions: 0 1 2 3 4 5
+operating system: linux x86_64
+
+"""]]
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+
+
+[[ben]]

Added a comment
diff --git a/doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes/comment_7_6b2a763f3d07f0cdc035008638e08194._comment b/doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes/comment_7_6b2a763f3d07f0cdc035008638e08194._comment
new file mode 100644
index 0000000..7c7d1b5
--- /dev/null
+++ b/doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes/comment_7_6b2a763f3d07f0cdc035008638e08194._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="benjamin.poldrack@d09ccff6d42dd20277610b59867cf7462927b8e3"
+ nickname="benjamin.poldrack"
+ avatar="http://cdn.libravatar.org/avatar/5c1a901caa7c2cfeeb7e17e786c5230d"
+ subject="comment 7"
+ date="2017-03-27T09:03:10Z"
+ content="""
+You are right, if we stage that file again with annex.largefiles option, it stays in git.
+But still keeping a file in git is troublesome. For example: Take a repository with an annexed file and a file in git, clone it and the call git annex init --version=6 on the clone.
+This will lead to a dirty repository, where git status as well as git annex status are stating, that the file in git has unstaged modifications.
+I'm not sure, whether this is actually related but I guess it is. Causing fresh clones to be dirty is at least a strange consequence for having files directly in git.
+
+[[ben]]
+"""]]

Added a comment: knowing about annex-ignore and git-annex shell helps
diff --git a/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly/comment_1_6e85b0122fca46b9232f8fd34e8c9f6d._comment b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly/comment_1_6e85b0122fca46b9232f8fd34e8c9f6d._comment
new file mode 100644
index 0000000..16ab942
--- /dev/null
+++ b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly/comment_1_6e85b0122fca46b9232f8fd34e8c9f6d._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="Cyberthal"
+ avatar="http://cdn.libravatar.org/avatar/1c619d65ee07d2343295c8f70f23c9df"
+ subject="knowing about annex-ignore and git-annex shell helps"
+ date="2017-03-26T10:06:35Z"
+ content="""
+When I wrote the above,
+I was unaware of the annex-ignore flag
+which can avert the above issue.
+
+This flag should've been automatically set.
+
+Also, I learned about the need for git-annex shell
+which must be available at the default path
+of remote servers.
+"""]]

diff --git a/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
index 4db907c..7e4d81d 100644
--- a/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
+++ b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
@@ -7,7 +7,8 @@ the annex-less git repo will gain a v5 annex branch
 it knows it can't store binary files
 but all the properly initialized annex repos in the network don't know that
 
-when I run "sync --content", the initialized annex repos think that the uninitialized repo contains the binary files.
+when I run "sync --content",
+the initialized annex repos think that the uninitialized repo contains copies.
 
 I suspect this results in inaccurate "copies" count
 I know it results in an inaccurate "list files' location" graph
@@ -35,12 +36,14 @@ git annex whereis
 
 Hypothesis:
 
-maybe the issue is that I cloned the repo rather than creating it normally,
-leaving it in a half-annexed state?
+maybe the issue is that I cloned the repo
+rather than creating it normally,
+thus leaving it in a half-annexed state?
 
 Experiment:
 
-tested by creating an independent git repo and then adding it as a remote to Alpha.
+tested by creating an independent git repo
+and then adding it as a remote to Alpha.
 
 then ran sync --content
 
@@ -55,11 +58,15 @@ git-annex sync --content
 
 ***** ramifications
 
-It would appear this error can cause data loss due to a false numcopies count.
+It would appear this error can cause data loss
+due to a false numcopies count.
 
-Yet GitHub is supposed to work. So this error should've already been noticed. Contradiction detected.
+Yet GitHub is supposed to work.
+So this error should've already been noticed.
+Contradiction detected.
 
-Positivity: I am planning on becoming a git-annex evangelist as part of a larger project. Emacs offers ergonomic control via magit and a dired plugin.
+Positivity: I am planning on becoming a git-annex evangelist as part of a larger project.
+Emacs offers ergonomic control via magit and a dired plugin.
 
 ***** environment
 

diff --git a/doc/bugs/__34__byte-progress__34___could_jump_down_upon_initiating_re-download_--_report_actual_one_first__63__.mdwn b/doc/bugs/__34__byte-progress__34___could_jump_down_upon_initiating_re-download_--_report_actual_one_first__63__.mdwn
new file mode 100644
index 0000000..6b0cb62
--- /dev/null
+++ b/doc/bugs/__34__byte-progress__34___could_jump_down_upon_initiating_re-download_--_report_actual_one_first__63__.mdwn
@@ -0,0 +1,28 @@
+### Please describe the problem.
+
+someone kinda could consider it a 'feature' but it complicates use of the output since then progressbar should jump down and some progressbar libraries 
+do not "support" that
+
+### What steps will reproduce the problem?
+
+initiate download, interrupt it, try to redownload it... I guess in some cases redownload doesn't start at the point where it was previously interrupted but somewhat before, or restarts altogether.  But annex first reports in --json-progress the size of previously downloaded portion and then goes down.  see below
+
+so, my life would be easier, if annex did not report "unconfirmed initial progress" at all I guess
+
+### What version of git-annex are you using? On what operating system?
+
+6.20170307+gitg24ade8a25-1~ndall+1
+
+### Please provide any additional information below.
+
+[[!format sh """
+$> ls -l .git/annex/tmp; k=$(/bin/ls .git/annex/tmp | head -n 1); [ -z "$k" ] || git annex get --json --json-progress --key $k
+total 3796
+-rw------- 1 yoh yoh 3887104 Mar 24 17:06 MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c
+{"byte-progress":3887104,"action":{"command":"get","note":"from origin...","key":"MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c","file":null},"total-size":4108657,"percent-progress":"94.61%"}
+{"byte-progress":2068480,"action":{"command":"get","note":"from origin...","key":"MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c","file":null},"total-size":4108657,"percent-progress":"50.34%"}
+{"command":"get","note":"checksum...","success":true,"key":"MD5E-s4108657--e055fc250b37b313c0904f3687bbed1c","file":null}
+
+"""]]
+
+

diff --git a/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
new file mode 100644
index 0000000..4db907c
--- /dev/null
+++ b/doc/bugs/cause__58___fake_annex_repo___124___result__58___inflated_numcopies___124___workaround__58___initialize_properly.mdwn
@@ -0,0 +1,75 @@
+<pre>
+
+***** summary
+
+if I run sync, and there is an annex-less git remote in the network
+the annex-less git repo will gain a v5 annex branch
+it knows it can't store binary files
+but all the properly initialized annex repos in the network don't know that
+
+when I run "sync --content", the initialized annex repos think that the uninitialized repo contains the binary files.
+
+I suspect this results in inaccurate "copies" count
+I know it results in an inaccurate "list files' location" graph
+and also inaccurate "whereis" readout
+
+***** reproduction
+
+repo "Alpha" is a git-annex repo with file "music"
+repo "Zeta" is a git repo cloned from Alpha
+
+@Alpha:
+git annex sync --content
+git annex list
+[shows music present on Zeta]
+git annex whereis
+[shows music present on Zeta]
+
+@Zeta
+git annex list
+[shows nothing]
+git annex whereis
+[shows music present on Zeta]
+
+***** proving that the problem is "sync --content"
+
+Hypothesis:
+
+maybe the issue is that I cloned the repo rather than creating it normally,
+leaving it in a half-annexed state?
+
+Experiment:
+
+tested by creating an independent git repo and then adding it as a remote to Alpha.
+
+then ran sync --content
+
+Result:
+
+no different than before.
+
+Conclusion:
+
+The problem lies with 
+git-annex sync --content
+
+***** ramifications
+
+It would appear this error can cause data loss due to a false numcopies count.
+
+Yet GitHub is supposed to work. So this error should've already been noticed. Contradiction detected.
+
+Positivity: I am planning on becoming a git-annex evangelist as part of a larger project. Emacs offers ergonomic control via magit and a dired plugin.
+
+***** environment
+
+git-annex version: 6.20170301-ga9e1e17d40
+build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+local repository version: unknown
+supported repository versions: 3 5 6
+upgrade supported from repository versions: 0 1 2 3 4 5
+operating system: linux x86_64
+
+</pre>