Recent changes to this wiki:

Added a comment
diff --git a/doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository/comment_2_97d5a2571679d5f6dc90694e5be89eaa._comment b/doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository/comment_2_97d5a2571679d5f6dc90694e5be89eaa._comment
new file mode 100644
index 000000000..8eec02fa4
--- /dev/null
+++ b/doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository/comment_2_97d5a2571679d5f6dc90694e5be89eaa._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="yves.noirjean@3f9b06d19a920fbf5c82340c362e5971b00d4af2"
+ nickname="yves.noirjean"
+ avatar="http://cdn.libravatar.org/avatar/2f1ad9d443c037337d91f29781560344"
+ subject="comment 2"
+ date="2018-06-22T07:53:26Z"
+ content="""
+I stumbled over the same problem. With \"git annex add\" I can add the file. But in unlocked mode, I get the \"out of memory\" error when trying to commit. Is this also a known problem? Is there a workaround?
+"""]]

Added a comment
diff --git a/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_3_e6bd4eb070a094116e13064dca6d5bb5._comment b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_3_e6bd4eb070a094116e13064dca6d5bb5._comment
new file mode 100644
index 000000000..94ae963a3
--- /dev/null
+++ b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_3_e6bd4eb070a094116e13064dca6d5bb5._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="yves.noirjean@3f9b06d19a920fbf5c82340c362e5971b00d4af2"
+ nickname="yves.noirjean"
+ avatar="http://cdn.libravatar.org/avatar/2f1ad9d443c037337d91f29781560344"
+ subject="comment 3"
+ date="2018-06-21T16:15:05Z"
+ content="""
+Actually, I am still interested in a response.
+
+I would prefer to use annex.addunlocked = true, which would use hardlinks.
+"""]]

Added a comment: Unable to access public s3 remote without S3 credentials.
diff --git a/doc/tips/public_Amazon_S3_remote/comment_1_a2684b49dbc2de45b644523f47b3c891._comment b/doc/tips/public_Amazon_S3_remote/comment_1_a2684b49dbc2de45b644523f47b3c891._comment
new file mode 100644
index 000000000..bc94d7e93
--- /dev/null
+++ b/doc/tips/public_Amazon_S3_remote/comment_1_a2684b49dbc2de45b644523f47b3c891._comment
@@ -0,0 +1,48 @@
+[[!comment format=mdwn
+ username="jared@ce91556d9548d318ec3f690b5f9bc33721256e4d"
+ nickname="jared"
+ avatar="http://cdn.libravatar.org/avatar/09bdb433e2a9448d780086be1e3d4f10"
+ subject="Unable to access public s3 remote without S3 credentials."
+ date="2018-06-21T13:47:27Z"
+ content="""
+I'm setting up a repository using the following commands:
+
+    git annex init
+    export AWS_ACCESS_KEY_ID=\"[MY KEY]\"
+    export AWS_SECRET_ACCESS_KEY=\"[MY SECRET]\"
+    git annex initremote publics3 type=S3 encryption=none bucket=[BUCKET] exporttree=yes public=yes encryption=none
+    git annex export --tracking master --to publics3
+
+I then add something and export it to s3 using:
+
+    git annex sync --content
+
+I then go on to a different computer and clone the repository and run:
+
+    git annex init
+    git annex enableremote publics3
+    git annex get .
+
+and receive this message:
+
+    # git annex get .
+    get IMG_2714.MOV (from publics3...) 
+    Set both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to use S3
+
+    No S3 credentials configured
+
+    Unable to access these remotes: publics3
+
+    Try making some of these repositories available:
+         6bc91baa-5250-41e2-8195-d3c1ef3c86e3 -- .
+         7f84dcff-5a74-42b0-b422-5b74490e4aa7 -- [publics3]
+
+     (Note that these git remotes have annex-ignore set: origin)
+failed
+    git-annex: get: 1 failed
+
+
+It is my understanding that setting the remote to \"public=yes\" should allow me to download the files from the remote repository without entering the credentials again. Am I missing something?
+
+Thanks!
+"""]]

diff --git a/doc/forum/Trying_to_setup_2_full_backup_repos_that_sync_with_eachother.mdwn b/doc/forum/Trying_to_setup_2_full_backup_repos_that_sync_with_eachother.mdwn
new file mode 100644
index 000000000..95d748348
--- /dev/null
+++ b/doc/forum/Trying_to_setup_2_full_backup_repos_that_sync_with_eachother.mdwn
@@ -0,0 +1,30 @@
+So I'm liking git-annex alot but trying to setup something up
+by using the webapp wasn't very successful.
+
+I have serverA machineB laptopC laptopD
+
+Now what I want to setup is:
+
+serverA       machineB
+repo   <sync> repo
+
+Basically I want A and B to constantly sync with each
+other for a fullbackup repo on each. But I also want both A and B to
+have the full checkout local.
+
+Then I have C and D just be clients and pull whatever they need from A or B.
+
+What I've gotten working atm:
+On A I setup the repo(not bare - I want access to the contents)). On B I ran
+a clone from A. That way B always syncs to A.
+
+Setup both repos to be fullbackup. I've tried to setup A to also sync
+from B but the webapp fails at doing it - and when I retry it fails
+again but this time with an error in the log that the remote already exists.
+
+Would I need to setup things manually for this scenario? Or is it not even possible?
+
+Do I just setup a normal remote? Or will this loop break things?
+
+When I'm done setting it all up I'd like to have the assistant
+running on both systems to do the syncs for me.

Error trying to run git-annex on Android version 8.1
diff --git a/doc/forum/Termux_Android_8.1_Proot_Error.mdwn b/doc/forum/Termux_Android_8.1_Proot_Error.mdwn
new file mode 100644
index 000000000..91e7cf453
--- /dev/null
+++ b/doc/forum/Termux_Android_8.1_Proot_Error.mdwn
@@ -0,0 +1,17 @@
+Hi,
+
+I'm trying to get git-annex working on my Google Pixel running Android version 8.1 through Termux but I'm having some difficulty.
+
+I copy-pasta'd the instructions found here: <https://git-annex.branchable.com/tips/install_on_Android_in_Termux/> until I got the following error when trying to run git-annex.linux/runshell
+
+    Running on Android.. Adding git-annex to PATH for you, and tuning for optimal behavior
+    proot error: execve("/data/data/com.termux/files/home/git-annex.linux/bin/sh"): No such file or directory
+    proot info: possible causes
+      * the program is a script but its interpreter (eg. /bin/sh) was not found;
+      * the program is an ELF but its interpreter (eg. ld-linux.so) was not found;
+      * the program is a foreign binary but qemu was not specified;
+      * qemu does not work correctly (if specified);
+      * the loader was not found or doesn't work.
+    fatal error: see `proot --help`.
+
+If I could get some help troubleshooting this that would be great. Thanks!

some issues with anonymous pushes
diff --git a/doc/forum/failed_to_push_anonymously.mdwn b/doc/forum/failed_to_push_anonymously.mdwn
new file mode 100644
index 000000000..196f6f71c
--- /dev/null
+++ b/doc/forum/failed_to_push_anonymously.mdwn
@@ -0,0 +1,19 @@
+I have pushed changes to this wiki in the past using this URL as specified in the `Branchable` tab:
+
+    git://git-annex.branchable.com/
+
+Yet after writing the [[tips/hashdeep integration]] page, I wasn't able to push it to this git repo, and got the following error:
+
+    [1023]anarcat@curie:git-annex$ git push 
+    Décompte des objets: 5, fait.
+    Delta compression using up to 4 threads.
+    Compression des objets: 100% (5/5), fait.
+    Écriture des objets: 100% (5/5), 3.30 KiB | 3.30 MiB/s, fait.
+    Total 5 (delta 3), reused 0 (delta 0)
+    remote: fatal: Not a git repository (or any of the parent directories): .git
+    remote: 'git log --pretty=raw --raw --abbrev=40 --always -c -r d8de48ddee630f546bd71adff6024f875f6db4c2..5bce64b25e53849a6cd7f5973fed849898428efa --no-renames -- .' failed: 
+    To git://git-annex.branchable.com/
+     ! [remote rejected]     master -> master (pre-receive hook declined)
+    error: impossible de pousser des références vers 'git://git-annex.branchable.com/'
+
+Seems like there's something wrong on that end, or did I do something wrong? -- [[anarcat]]

hashdeep integration
diff --git a/doc/tips/hashdeep_integration.mdwn b/doc/tips/hashdeep_integration.mdwn
new file mode 100644
index 000000000..90de1f49c
--- /dev/null
+++ b/doc/tips/hashdeep_integration.mdwn
@@ -0,0 +1,114 @@
+## What is hashdeep
+
+[hashdeep](http://md5deep.sourceforge.net/) is a handy tool that allows you to check file integrity
+across whole directory trees. It can detect renames and missing files,
+for example.
+
+## How to use it with git-annex
+
+The general working principle of hashdeep is that it iterates over a
+set of files and produces a manifest that looks like this:
+
+    $ hashdeep -r *
+    %%%% HASHDEEP-1.0
+    %%%% size,md5,sha256,filename
+    ## Invoked from: /home/jessek
+    ## $ hashdeep -r archives bin lib doc
+    21508,6178d221a1714b7e2089565e997d6ad1,92caa3f5754b22ca792e4f8626362d2ef39596b080abfcfed951a86bee82bec3,/home/jessek/archives/foo-1.2.1.tar.gz
+    12292,116e77a5dc6af0996597f7bc1b9252a2,c2afc6aa8d5c094a7226db1695d99a37fa858548f5d09aad9e41badfc62b1d27,/home/jessek/archives/bar-0.9.tar.bz2
+    145684,4409c1e0b5995c290c2fc3d1d6d74bac,f56881fb277358c95ed3ddf64f28c4ff3f3937e636e17d6a26d42822b16fd4ed,/home/jessek/bin/ls
+
+Then this manifest can be used to check consistency of the files
+later. Because git-annex also uses hashes to identify files, it fits
+nicely with this pattern and I have used it to verify files that were
+outside of git-annex's control yet still from the repository. First,
+we produce the manifest file:
+
+    (
+    echo '%%%% HASHDEEP-1.0'
+    echo '%%%% size,sha256,filename'
+    git annex find --format '${bytesize},${keyname},${file}\n' | sed 's/\.[^,]*,/,/'
+    ) > manifest.txt
+
+Then this can be used to verify an external fileset with the following
+command:
+
+    hashdeep -k manifest.txt -a -vv -e /mnt/ > result
+
+This will create a listing of every file that was moved, that is
+missing and so on. I have used this to audit corrupted files on my
+phone's microSD card as it turned out that about half of the files
+were corrupted for some mysterious reason:
+
+    hashdeep: Audit failed
+       Input files examined: 0
+      Known files expecting: 0
+              Files matched: 0
+    Files partially matched: 0
+                Files moved: 3411
+            New files found: 2179
+      Known files not found: 42117
+
+The non-zero numbers are interesting: 3411 files were detected as
+being sane and just the filenames had changed. 2179 files were "new"
+which means that they were not in the original set. Since files were
+supposed to *only* come from the original set, this means those files
+were corrupt. Actually, that's not completely true: some files (JPG
+image files, namely) *were* created in the external fileset so I had
+to be careful to exclude those false positives by hand. The 42117
+"known files not found" were files that were simply not transferred
+over to the phone for lack of space.
+
+This way, I was able to quickly find which files were corrupt and
+remove them. This created a list of files to remove:
+
+    grep 'No match' result  | grep -v '.jpg' | sed 's/: No match$//'
+
+And I used the following loop to remove the files one by one:
+
+    grep 'No match' result  | grep -v '.jpg' | sed 's/: No match$//' | while read file; do rm "$file" ; done
+
+Note the above is actually quite dangerous and you might want to
+insert an `echo` in there to avoid shenanigans, especially if you do
+not trust the filesystem.
+
+## How else this might work
+
+Naturally, I could have imported all the files into git-annex and work
+only with git-annex to operate this. But because the files were
+renamed to some canonical version by the software transferring the file
+([dSub](https://f-droid.org/en/packages/github.daneren2005.dsub/) and [Airsonic](https://airsonic.github.io/)), it would have been difficult to make a
+diff with the original set. This is on a (ex)fat filesystem too, which
+might make git-annex operation difficult. Yet I can't help but think
+this is something that [[git-annex-export]] should be able to do, but
+I am not sure it could deal with the renames. And I must say I have
+found it a little inconvenient to have to `initremote` to be able to
+use what are essentially ephemeral storage mountpoints.
+
+The above procedure reuses the best of both world: hashdeep does the
+fuzzy matching and git-annex provides the catalog of files.
+
+## Future improvements
+
+It would be nice if [[git-annex-find]] would allow listing only the
+checksum, which would remove a potentially error-prone pattern
+substitution above (`sed 's/\.[^,]*,/,/'`). This is necessary because
+`${keyname}` includes the file extension which is expected with the
+`SHA256E` backend, but it is somewhat inconvenient to deal with. Of
+course, it would be pretty awesome if git-annex could output
+hashdeep-compatible catalogs out of the box: it would improve
+interoperability here... And the icing on cake would be a git-annex
+command (a variation of [[git-annex-import]]?) that would audit an
+external, non-annexed repository for consistency in the same way.
+
+Also note that hashdeep can operate in "chunk" mode which means that
+it can work across file boundaries, detecting partial matches, for
+example. This is something that, as far as I know, is impossible in
+git-annex as checksums are only file-based. This would be useful in
+eliminating the false positives by distinguishing the "this file is
+completely new" and "this file is corrupt" cases.
+
+## Comments
+
+Those notes were provided by [[anarcat]] but would gladly welcome
+corrections and improvements.

Added a comment
diff --git a/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_2_cf8065e697a0a83cf3461672cb6421e0._comment b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_2_cf8065e697a0a83cf3461672cb6421e0._comment
new file mode 100644
index 000000000..51edcafd5
--- /dev/null
+++ b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_2_cf8065e697a0a83cf3461672cb6421e0._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="yves.noirjean@3f9b06d19a920fbf5c82340c362e5971b00d4af2"
+ nickname="yves.noirjean"
+ avatar="http://cdn.libravatar.org/avatar/2f1ad9d443c037337d91f29781560344"
+ subject="comment 2"
+ date="2018-06-18T11:25:18Z"
+ content="""
+The issue has been resolved. The unix file permissions were not applied, but the NFS4 permissions. So git annex did not trust that files without write flags could not be changed. Therefore, it didn't move the files and create symlinks. This somehow lead to git allocating a lot of memory during commit.
+"""]]

removed
diff --git a/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin.mdwn b/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin.mdwn
deleted file mode 100644
index 52b37bb3b..000000000
--- a/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin.mdwn
+++ /dev/null
@@ -1,22 +0,0 @@
-### Please describe the problem.
-
-symlinking git-annex from a unpacked tarball to the 
-
-### What steps will reproduce the problem?
-
-symlink git-annex somewhere, call that symline
-
-###
-
-known fix
-
-[[!format sh """
-#!/bin/sh
-GIT_ANNEX_PROGRAMPATH=$(realpath "$0")
-export GIT_ANNEX_PROGRAMPATH
-exec "$GIT_ANNEX_DIR/exe/git-annex" --library-path "$GIT_ANNEX_LD_LIBRARY_PATH" "$GIT_ANNEX_DIR/shimmed/git-annex/git-annex" "$@"
-
-
-"""]]
-
-the realpath call is new

Added a comment: this one is invalid
diff --git a/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin/comment_1_b201c92969d0ffdac2c6b738d4abae55._comment b/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin/comment_1_b201c92969d0ffdac2c6b738d4abae55._comment
new file mode 100644
index 000000000..640460a0b
--- /dev/null
+++ b/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin/comment_1_b201c92969d0ffdac2c6b738d4abae55._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="git-annex.branchable.com@07c0f8919010cc703ae7eea746d9b494c153291f"
+ nickname="git-annex.branchable.com"
+ avatar="http://cdn.libravatar.org/avatar/c5379a3fe2188b7571858c49f9db63c6"
+ subject="this one is invalid"
+ date="2018-06-16T09:48:53Z"
+ content="""
+i retested a few times, and the actual tool i should have used is a link to the git-annex script in the root
+
+sorry for the noise
+"""]]

diff --git a/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin.mdwn b/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin.mdwn
new file mode 100644
index 000000000..52b37bb3b
--- /dev/null
+++ b/doc/bugs/prebuild_tarball_breaks_on_symlinking_sig-annex_to_bin.mdwn
@@ -0,0 +1,22 @@
+### Please describe the problem.
+
+symlinking git-annex from a unpacked tarball to the 
+
+### What steps will reproduce the problem?
+
+symlink git-annex somewhere, call that symline
+
+###
+
+known fix
+
+[[!format sh """
+#!/bin/sh
+GIT_ANNEX_PROGRAMPATH=$(realpath "$0")
+export GIT_ANNEX_PROGRAMPATH
+exec "$GIT_ANNEX_DIR/exe/git-annex" --library-path "$GIT_ANNEX_LD_LIBRARY_PATH" "$GIT_ANNEX_DIR/shimmed/git-annex/git-annex" "$@"
+
+
+"""]]
+
+the realpath call is new

posted issue
diff --git a/doc/todo/Exporting_with_exporttree_should_sync_files_deleted_from_the_remote.mdwn b/doc/todo/Exporting_with_exporttree_should_sync_files_deleted_from_the_remote.mdwn
new file mode 100644
index 000000000..98ddef9f5
--- /dev/null
+++ b/doc/todo/Exporting_with_exporttree_should_sync_files_deleted_from_the_remote.mdwn
@@ -0,0 +1,3 @@
+If git-tracked files are removed from the remote, they don't get synced over after a "git annex fsck" and "git annex export".
+
+Is there some way that they could make it to the remote? I'm imagining an rsync-like behavior to copy over files that have different time stamps or file sizes. Would such a feature be welcome in git annex?

remove broken link
diff --git a/doc/upgrades.mdwn b/doc/upgrades.mdwn
index dda32c848..c7ade4a9c 100644
--- a/doc/upgrades.mdwn
+++ b/doc/upgrades.mdwn
@@ -18,8 +18,7 @@ therefore needs write access to the parent directory of the
 Note that "upgrading" from a distribution-based package to the
 [[install/Linux_standalone/]] version may cause weird problems, as an
 unexpected version of git-annex (e.g. the old one from packages) may be
-ran, see [[bugs/git-annex-shell_doesn__39__t_work_as_expected/]] for a full
-discussion.
+ran.
 
 # Repository upgrades
 

Added a comment
diff --git a/doc/forum/Dealing_with_crippled_Android_file_system/comment_8_a2bafe08ff3e7a236ba560dafc9c4d85._comment b/doc/forum/Dealing_with_crippled_Android_file_system/comment_8_a2bafe08ff3e7a236ba560dafc9c4d85._comment
new file mode 100644
index 000000000..87faebd6c
--- /dev/null
+++ b/doc/forum/Dealing_with_crippled_Android_file_system/comment_8_a2bafe08ff3e7a236ba560dafc9c4d85._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="comment 8"
+ date="2018-06-15T15:59:04Z"
+ content="""
+Oh, Awesome, thanks! Great find. I'll write out some of the things I have tried:
+
+Create a text file called `hi.txt` from Termux in `/data/data/com.termux/files/home` AKA `$HOME`. Launch `Turbo Editor` enable `Use the Storage Access Framework` in Preferences menu. Click `Open a file` | `Open from` `Termux`, click `hi.txt`, edit file, save. Now I can see changes in Termux. Nice.
+
+From Termux create a new soft link, `cd`, `ln -s hi.txt hilink.txt`, then in Turbo Editor I can see both files, I can edit and save either, and see changes correctly in Termux.
+
+I wasn't able to find the Termux `$HOME` folder in any File Manager I tried, nor any Gallery App.
+
+
+
+
+"""]]

improve wording
diff --git a/doc/todo/import_tree.mdwn b/doc/todo/import_tree.mdwn
index f86f878db..4bb44d939 100644
--- a/doc/todo/import_tree.mdwn
+++ b/doc/todo/import_tree.mdwn
@@ -12,12 +12,12 @@ that has the modifications in it.
 Updating the working copy is then done by merging the import treeish.
 This way, conflicts will be detected and handled as normal by git.
 
-The remote interface needs one new method, to list the changed/new and
+----
+
+The remote interface could have a new method, to list the changed/new and
 deleted files. It will be up to remotes to implement that if they can
 support importing.
 
-----
-
 One way for a remote to do it, assuming it has mtimes, is to export
 files to the remote with their mtime set to the date of the treeish
 being exported (when the treeish is a commit, which has dates, and not
@@ -38,8 +38,7 @@ Where to store that data?
 
 The data could be stored in a file/files on the remote, or perhaps
 the remote has a way to store some arbitrary metadata about a file
-that could be used. Note that's basically the same as implementing the git
-index, on a per-remote basis.
+that could be used.
 
 It could be stored in git-annex branch per-remote state. However,
 that state is per-key, not per-file. The export database could be
@@ -58,18 +57,31 @@ masters, which can be reconciled as usual. It would mean extra downloads
 of content from the remote, since each import would download its own copy.
 Perhaps this is acceptable?
 
+This feels like it's reimplementing the git index, on a per-remote basis.
+So perhaps this is not the right interface.
+
 ----
 
-Following the thoughts above, how about this design: The remote
-is responsible for collecting a list of files currently in it, along with
-some content identifier. That data is sent to git-annex. git-annex stores
-the content identifiers locally, and compares old and new lists to determine
-when a file on the remote has changed or is new.
+Alternate interface: The remote is responsible for collecting a list of
+files currently in it, along with some content identifier. That data is
+sent to git-annex. git-annex keep track of which content identifier(s) map
+to which keys, and uses the information to determine when a file on the
+remote has changed or is new.
 
 This way, each special remote doesn't have to reimplement the equivilant of
 the git index, or comparing lists of files, it only needs a way to list
 files, and a good content identifier.
 
+This also simplifies implementation in git-annex, because it does not
+even need to look for changed/new/deleted files compared with the
+old tree. Instead, it can simply build git tree objects as the file list
+comes in, looking up the key corresponding to each content identifier
+(or downloading the content from the remote and adding it to the annex
+when there's no corresponding key yet). It might be possible to avoid
+git-annex buffering much tree data in memory.
+
+----
+
 A good content identifier needs to:
 
 * Be stable, so when a file has not changed, the content identifier
@@ -92,15 +104,16 @@ Do remotes need to tell git-annex about the properties of content
 identifiers they use, or does git-annex assume a minimum bar, and pay the
 price with some unncessary transfers of renamed files etc?
 
-Note that git-annex will need a way to get the content identifiers of files
-that it stores on the remote when exporting a tree to it. There's a race
-here, since a file could be modified on the remote while it's being
-exported, and if the remote then uses its mtime in the content identifier,
-the modification would never be noticed.
+----
+
+git-annex will need a way to get the content identifiers of files
+that it stores on the remote when exporting a tree to it, so it can later
+know if those files have changed.
 
-(Does git have this same race when updating the work tree after a merge?
-There's also a race where a file is modified and then immediately replaced
-with an exported update. Does git have the equivilant race?)
+There's a race here, since a file could be modified on the remote while
+it's being exported, and if the remote then uses its mtime in the content
+identifier, the modification would never be noticed.
+(Does git have this same race when updating the work tree after a merge?)
 
 Some remotes could avoid that race, if they sent back the content
 identifier in response to the TRANSFEREXPORT message, and kept the file
@@ -109,12 +122,18 @@ probably can't avoid the race. Is it worth changing the TRANSFEREXPORT
 interface to include the content identifier in the reply if it doesn't
 always avoid the race?
 
-Since exporttree remotes don't have content identifier information yet,
-it needs to be collected the first time import tree is used. (Or
-import everything, but that is probably too expensive). Any modifications
-made before the first import tree would not be noticed. Seems acceptible
-as long as this only affects exporttree remotes created before this feature
-was added.
+There's also a race where a file gets changed on the remote after an
+import tree, and an export then overwrites it with something else. This
+race seems impossible to avoid. Does git have the equivilant race?
+
+----
+
+Since exporttree remotes don't have content identifier information yet, it
+needs to be collected the first time import tree is used. (Or import
+everything, but that is probably too expensive). Any modifications made to
+exported files before the first import tree would not be noticed. Seems
+acceptible as long as this only affects exporttree remotes created before
+this feature was added.
 
 What if repo A is being used to import tree from R for a while, and the
 user gets used to editing files on R and importing them. Then they stop
@@ -122,7 +141,8 @@ using A and switch to clone B. It would not have the content identifier
 information that A did (unless it's stored in git-annex branch rather than
 locally). It seems that in this case, B needs to re-download everything,
 since anything could have changed since the last time A imported.
-That seems too expensive! 
+That seems too expensive!
+
 Would storing content identifiers in the git-annex branch be too expensive?
 
 ----

more thoughts
diff --git a/doc/todo/import_tree.mdwn b/doc/todo/import_tree.mdwn
index e6e2c0471..f86f878db 100644
--- a/doc/todo/import_tree.mdwn
+++ b/doc/todo/import_tree.mdwn
@@ -96,14 +96,34 @@ Note that git-annex will need a way to get the content identifiers of files
 that it stores on the remote when exporting a tree to it. There's a race
 here, since a file could be modified on the remote while it's being
 exported, and if the remote then uses its mtime in the content identifier,
-the modification would never be noticed. (Does git have this same race when
-updating the work tree after a merge?)
+the modification would never be noticed.
+
+(Does git have this same race when updating the work tree after a merge?
+There's also a race where a file is modified and then immediately replaced
+with an exported update. Does git have the equivilant race?)
 
 Some remotes could avoid that race, if they sent back the content
 identifier in response to the TRANSFEREXPORT message, and kept the file
 quarentined until they had generated the content identifier. Other remotes
 probably can't avoid the race. Is it worth changing the TRANSFEREXPORT
-interface to include the content identifier in the reply?
+interface to include the content identifier in the reply if it doesn't
+always avoid the race?
+
+Since exporttree remotes don't have content identifier information yet,
+it needs to be collected the first time import tree is used. (Or
+import everything, but that is probably too expensive). Any modifications
+made before the first import tree would not be noticed. Seems acceptible
+as long as this only affects exporttree remotes created before this feature
+was added.
+
+What if repo A is being used to import tree from R for a while, and the
+user gets used to editing files on R and importing them. Then they stop
+using A and switch to clone B. It would not have the content identifier
+information that A did (unless it's stored in git-annex branch rather than
+locally). It seems that in this case, B needs to re-download everything,
+since anything could have changed since the last time A imported.
+That seems too expensive! 
+Would storing content identifiers in the git-annex branch be too expensive?
 
 ----
 

some open questions
diff --git a/doc/todo/import_tree.mdwn b/doc/todo/import_tree.mdwn
index e47374b57..e6e2c0471 100644
--- a/doc/todo/import_tree.mdwn
+++ b/doc/todo/import_tree.mdwn
@@ -78,15 +78,32 @@ A good content identifier needs to:
 * Be reasonably unique, but not necessarily fully unique.  
   For example, if the mtime of a file is used as the content identifier, then
   a rename that swaps two files would be noticed, except for in the
-  unusual case where they have the same mtime. If a new file (or a copy)
+  unusual case where they have the same mtime. If a new file
   is added with the same mtime as some other file in the tree though,
-  git-annex will see that the file is new, and so can still import it, even
-  though it's seen that content identifier before. Of course, that might
-  result in unncessary downloads, so a more unique content identifer would
-  be better.
+  git-annex will see that the filename is new, and so can still import it,
+  even though it's seen that content identifier before. Of course, that might
+  result in unncessary downloads (eg of a renamed file), so a more unique
+  content identifer would be better.
 
 A (size, mtime, inode) tuple is as good a content identifier as git uses in
-its index. That or a hash of the content would be ideal.
+its index. That or a hash of the content would be ideal. 
+
+Do remotes need to tell git-annex about the properties of content
+identifiers they use, or does git-annex assume a minimum bar, and pay the
+price with some unncessary transfers of renamed files etc?
+
+Note that git-annex will need a way to get the content identifiers of files
+that it stores on the remote when exporting a tree to it. There's a race
+here, since a file could be modified on the remote while it's being
+exported, and if the remote then uses its mtime in the content identifier,
+the modification would never be noticed. (Does git have this same race when
+updating the work tree after a merge?)
+
+Some remotes could avoid that race, if they sent back the content
+identifier in response to the TRANSFEREXPORT message, and kept the file
+quarentined until they had generated the content identifier. Other remotes
+probably can't avoid the race. Is it worth changing the TRANSFEREXPORT
+interface to include the content identifier in the reply?
 
 ----
 

more thoughts
diff --git a/doc/todo/import_tree.mdwn b/doc/todo/import_tree.mdwn
index 8c40d089a..e47374b57 100644
--- a/doc/todo/import_tree.mdwn
+++ b/doc/todo/import_tree.mdwn
@@ -23,17 +23,70 @@ files to the remote with their mtime set to the date of the treeish
 being exported (when the treeish is a commit, which has dates, and not
 a raw tree). Then the remote can simply enumerate all files,
 with their mtimes, and look for files that have mtimes
-newer than the last exported treeish's date, as well as noticing
-deleted and newly added/renamed files.
+newer than the last exported treeish's date.
 
-> Hmm, but if files on the remote are being changed at the same time
-> as the export, then they could have older mtimes, and be missed.
-> --[[Joey]]
+> But: If files on the remote are being changed at around the time
+> of the export, they could have older mtimes than the exported treeish's
+> date, and so be missed.
+> 
+> Also, a rename that swaps two files would be missed if mtimes
+> are only compared to the treeish's date.
+
+A perhaps better way is for the remote to keep track of the mtime,
+size, etc of all exported files, and use that state to find changes.
+Where to store that data?
+
+The data could be stored in a file/files on the remote, or perhaps
+the remote has a way to store some arbitrary metadata about a file
+that could be used. Note that's basically the same as implementing the git
+index, on a per-remote basis.
+
+It could be stored in git-annex branch per-remote state. However,
+that state is per-key, not per-file. The export database could be
+used to convert a ExportLocation to a Key, which could be used
+to access the per-remote state. Querying the database for each file
+in the export could be a bottleneck without the right interface.
+
+If only one repository will ever access the remote, it could be stored
+in eg a local database. But access from only one repository is a 
+hard invariant to guarantee.
+
+Would local storage pose a problem when multiple repositories import from
+the same remote? In that case, perhaps different trees would be imported,
+and merged into master. So the two repositories then have differing
+masters, which can be reconciled as usual. It would mean extra downloads
+of content from the remote, since each import would download its own copy.
+Perhaps this is acceptable?
+
+----
+
+Following the thoughts above, how about this design: The remote
+is responsible for collecting a list of files currently in it, along with
+some content identifier. That data is sent to git-annex. git-annex stores
+the content identifiers locally, and compares old and new lists to determine
+when a file on the remote has changed or is new.
+
+This way, each special remote doesn't have to reimplement the equivilant of
+the git index, or comparing lists of files, it only needs a way to list
+files, and a good content identifier.
+
+A good content identifier needs to:
+
+* Be stable, so when a file has not changed, the content identifier
+  remains the same.
+* Change when a file is modified.
+* Be reasonably unique, but not necessarily fully unique.  
+  For example, if the mtime of a file is used as the content identifier, then
+  a rename that swaps two files would be noticed, except for in the
+  unusual case where they have the same mtime. If a new file (or a copy)
+  is added with the same mtime as some other file in the tree though,
+  git-annex will see that the file is new, and so can still import it, even
+  though it's seen that content identifier before. Of course, that might
+  result in unncessary downloads, so a more unique content identifer would
+  be better.
 
-A similar approach is for the remote to preserve object file timestamps,
-but keep a list somewhere (eg a file on the remote) of the timestamps of
-each exported file, and then it can later look for files with newer
-timestamps.
+A (size, mtime, inode) tuple is as good a content identifier as git uses in
+its index. That or a hash of the content would be ideal.
 
 ----
 

thoughts
diff --git a/doc/todo/import_tree.mdwn b/doc/todo/import_tree.mdwn
index 91aa630be..8c40d089a 100644
--- a/doc/todo/import_tree.mdwn
+++ b/doc/todo/import_tree.mdwn
@@ -26,6 +26,10 @@ with their mtimes, and look for files that have mtimes
 newer than the last exported treeish's date, as well as noticing
 deleted and newly added/renamed files.
 
+> Hmm, but if files on the remote are being changed at the same time
+> as the export, then they could have older mtimes, and be missed.
+> --[[Joey]]
+
 A similar approach is for the remote to preserve object file timestamps,
 but keep a list somewhere (eg a file on the remote) of the timestamps of
 each exported file, and then it can later look for files with newer
@@ -37,6 +41,18 @@ If multiple repos can access the remote at the same time, then there's a
 potential problem when one is exporting a new tree, and the other one is
 importing from the remote.
 
+> This can be reduced to the same problem as exports of two
+> different trees to the same remote, which is already handled with the
+> export log.
+> 
+> Once a tree has been imported from the remote, it's
+> in the same state as exporting that same tree to the remote, so
+> update the export log to say that the remote has that treeish exported
+> to it. A conflict between two export log entries will be handled as
+> usual, with the user being prompted to re-export the tree they want
+> to be on the remote. (May need to reword that prompt.)
+> --[[Joey]]
+
 ----
 
 See also, [[adb_special_remote]]

Added a comment
diff --git a/doc/forum/Multiple_group_groupwanted_settings_help/comment_2_b3d95d5c03c7b985d39a4d5a45ba3dec._comment b/doc/forum/Multiple_group_groupwanted_settings_help/comment_2_b3d95d5c03c7b985d39a4d5a45ba3dec._comment
new file mode 100644
index 000000000..b981e34b3
--- /dev/null
+++ b/doc/forum/Multiple_group_groupwanted_settings_help/comment_2_b3d95d5c03c7b985d39a4d5a45ba3dec._comment
@@ -0,0 +1,27 @@
+[[!comment format=mdwn
+ username="ghen1"
+ avatar="http://cdn.libravatar.org/avatar/efd0e92b6198291138f0cd7aedbf86b6"
+ subject="comment 2"
+ date="2018-06-13T17:18:49Z"
+ content="""
+Thank you for clarifying. I've worked out a setup that seems to do what I want. The rules are a little messier, but not too bad:
+
+    group a1 = a 1 a1store
+    group a2 = a 2 a2store
+    group b1 = b 1 b1store
+    group b2 = b 2 b2store
+    
+    wanted a1 = groupwanted
+    wanted a2 = groupwanted
+    wanted b1 = groupwanted
+    wanted b2 = groupwanted
+    
+    groupwanted a1store = not copies=1:1 and not copies=a:1
+    groupwanted a2store = not copies=2:1 and not copies=a:1
+    groupwanted b1store = not copies=1:1 and not copies=b:1
+    groupwanted b2store = not copies=2:1 and not copies=b:1
+    
+    numcopies default = 2
+    config annex.synccontent = true
+
+"""]]

Added a comment: Update on using SAF
diff --git a/doc/forum/Dealing_with_crippled_Android_file_system/comment_7_087fdae633144482473c7391ea338c5c._comment b/doc/forum/Dealing_with_crippled_Android_file_system/comment_7_087fdae633144482473c7391ea338c5c._comment
new file mode 100644
index 000000000..12c667154
--- /dev/null
+++ b/doc/forum/Dealing_with_crippled_Android_file_system/comment_7_087fdae633144482473c7391ea338c5c._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="https://christian.amsuess.com/chrysn"
+ nickname="chrysn"
+ avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc"
+ subject="Update on using SAF"
+ date="2018-06-13T14:57:10Z"
+ content="""
+I've looked around for how this whole SAF thing works and where it is supported; it seems to me that it is indeed the way to go:
+
+* Termux added this with the very intention of making the home directory usable ([initial issue](https://github.com/termux/termux-app/issues/79)).
+  * That .JPG files are not image/jpeg is probably the fault of a library termux uses ([reported](https://github.com/termux/termux-app/issues/721))
+* Some applications can find files from a termux-home out-of-the-box, like the [billthefarmer editor](https://f-droid.org/en/packages/org.billthefarmer.editor/) -- just that if it was just about editing text files, we could just have used ~/storage/shared and plain git.
+* Many applications accept files from there when opened from the file manager (VLC, Simple Gallery, Ghost Commander's picture viewer); I have not found one yet that does not.
+* A good way forward would be to encourage the authors of apps we care about to allow opening SAF locations directly; it appears to me that they sometimes add SAF support for a particular function (eg. to support USB-OTG or writing to external SD cards) and then limit it to that path because the usual other options appear to confusing. [Requested support for it in Simple Gallery](https://github.com/SimpleMobileTools/Simple-Gallery/issues/806), let's see where that takes this.
+"""]]

Added a comment
diff --git a/doc/bugs/git-annex_doesn__39__t_find_adb_device/comment_2_7e59d3e5a6273e737626cf73013c334d._comment b/doc/bugs/git-annex_doesn__39__t_find_adb_device/comment_2_7e59d3e5a6273e737626cf73013c334d._comment
new file mode 100644
index 000000000..843a74ceb
--- /dev/null
+++ b/doc/bugs/git-annex_doesn__39__t_find_adb_device/comment_2_7e59d3e5a6273e737626cf73013c334d._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="lykos@d125a37d89b1cfac20829f12911656c40cb70018"
+ nickname="lykos"
+ avatar="http://cdn.libravatar.org/avatar/085df7b04d3408ba23c19f9c49be9ea2"
+ subject="comment 2"
+ date="2018-06-13T10:46:25Z"
+ content="""
+Thanks! I think it's independent from adb, as it's the same serial number shown by dmesg.
+"""]]

Added a comment: Re: comment 5
diff --git a/doc/forum/Dealing_with_crippled_Android_file_system/comment_6_91f81669519dbf1dfbad2259f48ab99d._comment b/doc/forum/Dealing_with_crippled_Android_file_system/comment_6_91f81669519dbf1dfbad2259f48ab99d._comment
new file mode 100644
index 000000000..95a08e122
--- /dev/null
+++ b/doc/forum/Dealing_with_crippled_Android_file_system/comment_6_91f81669519dbf1dfbad2259f48ab99d._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="https://christian.amsuess.com/chrysn"
+ nickname="chrysn"
+ avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc"
+ subject="Re: comment 5"
+ date="2018-06-12T19:01:09Z"
+ content="""
+ad SDCardFS / FAT32: It seems that both the fuse wrapper around the SD card (be it actually formatted FAT32 because it's the external one, or ext4 as is the encrypted internal one on my device) and SDCardFS (which, it appears to me, is just Google moving the fuse wrapper into the kernel) won't expose the functionality we want.
+
+ad adoptable storage: I think that's what's happening on my device (but am not sure), so if it is, the there's still the fuse wrapper / SDCardFS inbetween.
+
+ad specific app: That'd still mean that git-annex needs to operate on crippled file system, and I don't want to rely on special solutions for each app.
+
+---
+
+What I've seen when just setting up a new device is that the Storage Access Framework (looks like Android's idea of gvfs / kvfs with a spike of flatpak-portals to me: a userland file system that operates on `saf:/` and `content://` URIs rather than files, probably does file operations by means of IPC and receiving file handlers) might help:
+
+In the LineageOS built-in file manager, I don't see two \"virtual SD cards\" (the internal and the external one), but also a \"Termux\" \"device\" with proper logo and \"0B free\", which allows me to peek into the termux home directory (but not its usr directory). From that file manager I even see the photos inside a regular (symlinked) checkout, and can open them with the Simple Gallery app. (Provided they end with '.jpg' and not '.JPG', but that's probably a detail of the file manager).
+
+I did not yet find another app that sees this \"Termux\" device; Ghost Commander only sees \"Documents\" and the internal and external SD cards; the \"Dir\" file manager exposes a SAF selection only when creating directories on external SD (and then shows the same selection as Ghost Commander), the Simple tools (Simple File Manager, Simple gallery) show \"Internal\", \"SD Card\" and \"Root\" in their own storage selection (might be pre-SAF; similar situation also for the Amaze file manager).
+
+If we could reproduce what the builtin file manager does to access the files, that might be a solution here. Termux obviously already helps in this by exporting the home directory ([since 0.34](https://termux.com/changelog.html)), but maybe they also can do more (like exporting XDG user directories for quick access?).
+"""]]

display p2pstdio stderr after auth
Display error messages that come from git-annex-shell when the p2p protocol
is used, so that diskreserve messages, IO errors, etc from the remote side
are visible again.
Felt like it should perhaps use outputError, so --json-error-messages would
include these, but as an async IO action, it can't, and this would need
MessageState to be converted to a tvar. Anyway, when not using p2pstdio,
that's not done; nor is it done for stderr from external special remotes
or other commands, so punted on the idea for now.
This commit was sponsored by mo on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index 45ed553d9..c927c9762 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -11,6 +11,9 @@ git-annex (6.20180530) UNRELEASED; urgency=medium
     by replying to the GETINFO message.
   * adb: Android serial numbers are not all 16 characters long, so accept
     other lengths.
+  * Display error messages that come from git-annex-shell when the p2p
+    protocol is used, so that diskreserve messages, IO errors, etc from
+    the remote side are visible again.
 
  -- Joey Hess <id@joeyh.name>  Wed, 30 May 2018 11:49:08 -0400
 
diff --git a/Remote/Helper/Ssh.hs b/Remote/Helper/Ssh.hs
index 2c5e204ef..021e6a087 100644
--- a/Remote/Helper/Ssh.hs
+++ b/Remote/Helper/Ssh.hs
@@ -27,6 +27,8 @@ import qualified P2P.IO as P2P
 import qualified P2P.Annex as P2P
 
 import Control.Concurrent.STM
+import Control.Concurrent.Async
+import qualified Data.ByteString as B
 
 toRepo :: ConsumeStdin -> Git.Repo -> RemoteGitConfig -> SshCommand -> Annex (FilePath, [CommandParam])
 toRepo cs r gc remotecmd = do
@@ -186,12 +188,15 @@ contentLockedMarker = "OK"
 
 -- A connection over ssh to git-annex shell speaking the P2P protocol.
 type P2PSshConnection = P2P.ClosableConnection
-	(P2P.RunState, P2P.P2PConnection, ProcessHandle)
+	(P2P.RunState, P2P.P2PConnection, ProcessHandle, TVar StderrHandlerState)
+
+data StderrHandlerState = DiscardStderr | DisplayStderr | EndStderrHandler
 
 closeP2PSshConnection :: P2PSshConnection -> IO (P2PSshConnection, Maybe ExitCode)
 closeP2PSshConnection P2P.ClosedConnection = return (P2P.ClosedConnection, Nothing)
-closeP2PSshConnection (P2P.OpenConnection (_st, conn, pid)) = do
+closeP2PSshConnection (P2P.OpenConnection (_st, conn, pid, stderrhandlerst)) = do
 	P2P.closeConnection conn
+	atomically $ writeTVar stderrhandlerst EndStderrHandler
 	exitcode <- waitForProcess pid
 	return (P2P.ClosedConnection, Just exitcode)
 
@@ -245,14 +250,12 @@ openP2PSshConnection r connpool = do
 			return Nothing
 		Just (cmd, params) -> start cmd params =<< getRepo r
   where
-	start cmd params repo = liftIO $ withNullHandle $ \nullh -> do
-		-- stderr is discarded because old versions of git-annex
-		-- shell always error
-		(Just from, Just to, Nothing, pid) <- createProcess $
+	start cmd params repo = liftIO $ do
+		(Just from, Just to, Just err, pid) <- createProcess $
 			(proc cmd (toCommand params))
 				{ std_in = CreatePipe
 				, std_out = CreatePipe
-				, std_err = UseHandle nullh
+				, std_err = CreatePipe
 				}
 		let conn = P2P.P2PConnection
 			{ P2P.connRepo = repo
@@ -260,14 +263,17 @@ openP2PSshConnection r connpool = do
 			, P2P.connIhdl = to
 			, P2P.connOhdl = from
 			}
+		stderrhandlerst <- newStderrHandler err
 		runst <- P2P.mkRunState P2P.Client
-		let c = P2P.OpenConnection (runst, conn, pid)
+		let c = P2P.OpenConnection (runst, conn, pid, stderrhandlerst)
 		-- When the connection is successful, the remote
 		-- will send an AUTH_SUCCESS with its uuid.
 		let proto = P2P.postAuth $
 			P2P.negotiateProtocolVersion P2P.maxProtocolVersion
 		tryNonAsync (P2P.runNetProto runst conn proto) >>= \case
-			Right (Right (Just theiruuid)) | theiruuid == uuid r ->
+			Right (Right (Just theiruuid)) | theiruuid == uuid r -> do
+				atomically $ 
+					writeTVar stderrhandlerst DisplayStderr
 				return $ Just c
 			_ -> do
 				(cclosed, exitcode) <- closeP2PSshConnection c
@@ -286,6 +292,33 @@ openP2PSshConnection r connpool = do
 		modifyTVar' connpool $
 			maybe (Just P2PSshUnsupported) Just
 
+newStderrHandler :: Handle -> IO (TVar StderrHandlerState)
+newStderrHandler errh = do
+	-- stderr from git-annex-shell p2pstdio is initially discarded
+	-- because old versions don't support the command. Once it's known
+	-- to be running, this is changed to DisplayStderr.
+	v <- newTVarIO DiscardStderr
+	p <- async $ go v
+	void $ async $ ender p v
+	return v
+  where
+	go v = do
+		l <- B.hGetLine errh
+		atomically (readTVar v) >>= \case
+			DiscardStderr -> go v
+			DisplayStderr -> do
+				B.hPut stderr l
+				go v
+			EndStderrHandler -> return ()
+	
+	ender p v = do
+		atomically $ do
+			readTVar v >>= \case
+				EndStderrHandler -> return ()
+				_ -> retry
+		hClose errh
+		cancel p
+
 -- Runs a P2P Proto action on a remote when it supports that,
 -- otherwise the fallback action.
 runProto :: Remote -> P2PSshConnectionPool -> a -> Annex a -> P2P.Proto a -> Annex (Maybe a)
@@ -304,7 +337,7 @@ runProto r connpool bad fallback proto = Just <$>
 
 runProtoConn :: P2P.Proto a -> P2PSshConnection -> Annex (P2PSshConnection, Maybe a)
 runProtoConn _ P2P.ClosedConnection = return (P2P.ClosedConnection, Nothing)
-runProtoConn a conn@(P2P.OpenConnection (runst, c, _pid)) = do
+runProtoConn a conn@(P2P.OpenConnection (runst, c, _, _)) = do
 	P2P.runFullProto runst c a >>= \case
 		Right r -> return (conn, Just r)
 		-- When runFullProto fails, the connection is no longer
diff --git a/doc/bugs/new_git-annex-shell_protocol_hides_remote_error_messages.mdwn b/doc/bugs/new_git-annex-shell_protocol_hides_remote_error_messages.mdwn
index e4e9d6cda..8c27d4296 100644
--- a/doc/bugs/new_git-annex-shell_protocol_hides_remote_error_messages.mdwn
+++ b/doc/bugs/new_git-annex-shell_protocol_hides_remote_error_messages.mdwn
@@ -1,3 +1,21 @@
 `git-annex-shell p2pstdio` hides error messages that were transported over
 ssh to display to the user before. For example, diskreserve problems or IO
 errors. --[[Joey]]
+
+git-annex discards stderr from the command because old
+versions of git-annex-shell don't support the command and error out.
+
+Simply letting stderr through seems like the best solution though,
+if a way can be found to do it.
+Otherwise, all errors would have to be trapped (easy), and all stderr
+output also trapped (hard!), to be sent over the protocol using ERROR.
+And, there'd be a problem with sending messages atomically; if a message is
+being sent and an exception is thrown, that message needs to somehow be
+ended before an ERROR message can be sent.
+
+Hmm, it negotiates the protocol version after opening the connection.
+Any error at that point would make it not use the p2p protocol, 
+so can be excluded. Then, after version negotiation is complete, it
+could start displaying stderr.
+
+> [[fixed|done]] --[[Joey]]

adb: Android serial numbers are not all 16 characters long, so accept other lengths.
I can't find any documentation of how long it should be. Hard to imagine
it being shorter than 4 characters though, so put that in as a conservative
lower bound.
This commit was sponsored by Nick Piper on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index f70200913..45ed553d9 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -9,6 +9,8 @@ git-annex (6.20180530) UNRELEASED; urgency=medium
     is configured.
   * External special remotes can now add info to `git annex info $remote`,
     by replying to the GETINFO message.
+  * adb: Android serial numbers are not all 16 characters long, so accept
+    other lengths.
 
  -- Joey Hess <id@joeyh.name>  Wed, 30 May 2018 11:49:08 -0400
 
diff --git a/Remote/Adb.hs b/Remote/Adb.hs
index 32bbe605f..f1e87013d 100644
--- a/Remote/Adb.hs
+++ b/Remote/Adb.hs
@@ -236,7 +236,7 @@ enumerateAdbConnected =
   where
 	parse l = 
 		let (serial, desc) = separate (== '\t') l
-		in if null desc || length serial /= 16
+		in if null desc || length serial < 4
 			then Nothing 
 			else Just (AndroidSerial serial)
 
diff --git a/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn b/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
index ff8600ce1..5a37559cc 100644
--- a/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
+++ b/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
@@ -20,3 +20,4 @@ git-annex claims that adb does not list any devices. `adb devices` however shows
     upgrade supported from repository versions: 0 1 2 3 4 5
     operating system: linux x86_64
 
+> [[fixed|done]] --[[Joey]] 
diff --git a/doc/bugs/git-annex_doesn__39__t_find_adb_device/comment_1_56070e29ea7f19d24d4ff0f56adf08a0._comment b/doc/bugs/git-annex_doesn__39__t_find_adb_device/comment_1_56070e29ea7f19d24d4ff0f56adf08a0._comment
new file mode 100644
index 000000000..f030295f5
--- /dev/null
+++ b/doc/bugs/git-annex_doesn__39__t_find_adb_device/comment_1_56070e29ea7f19d24d4ff0f56adf08a0._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-06-12T17:39:51Z"
+ content="""
+It's expecting a serial number 16 digits long, 
+and your adb is using a 12 digit one.  I suppose this means that the length
+can vary. Hopefully because of a difference in the android device and not
+the version of adb. I'll change it to accept shorter serials.
+"""]]

response
diff --git a/doc/bugs/non-git-annex_symlinks_not_perserved_when_exporting_tree_via_rsync/comment_1_b92c67f7d2f0424ce399ca4659e6c805._comment b/doc/bugs/non-git-annex_symlinks_not_perserved_when_exporting_tree_via_rsync/comment_1_b92c67f7d2f0424ce399ca4659e6c805._comment
new file mode 100644
index 000000000..1ad0e16d5
--- /dev/null
+++ b/doc/bugs/non-git-annex_symlinks_not_perserved_when_exporting_tree_via_rsync/comment_1_b92c67f7d2f0424ce399ca4659e6c805._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-06-12T16:46:43Z"
+ content="""
+Yes, neither are execute bits preserved. Most kinds of special remotes that
+an export could be done to don't support such things, and it seemed like it
+would unduely complicate the export interface to support them.
+"""]]

diff --git a/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn b/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
index 2f46a35c6..ff8600ce1 100644
--- a/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
+++ b/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
@@ -1,5 +1,5 @@
 ### Please describe the problem.
-git-annex claims that adb does not list any files. `adb devices` however shows the device.
+git-annex claims that adb does not list any devices. `adb devices` however shows the device.
 
 ### What steps will reproduce the problem?
     $ git annex initremote phone type=adb androiddirectory=/storage/external/annex encryption=none exporttree=yes        

diff --git a/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn b/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
new file mode 100644
index 000000000..2f46a35c6
--- /dev/null
+++ b/doc/bugs/git-annex_doesn__39__t_find_adb_device.mdwn
@@ -0,0 +1,22 @@
+### Please describe the problem.
+git-annex claims that adb does not list any files. `adb devices` however shows the device.
+
+### What steps will reproduce the problem?
+    $ git annex initremote phone type=adb androiddirectory=/storage/external/annex encryption=none exporttree=yes        
+    initremote handy git-annex: adb does not list any connected android devices. Plug in an Android device, or configure adb, and try again..
+    $ adb devices                                          
+    List of devices attached
+    6c12eba7d440	device
+
+
+### What version of git-annex are you using? On what operating system?
+    git-annex version: 6.20180529-g33834140e
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.2 feed-1.0.0.0 ghc-8.4.3 http-client-0.5.12.1 persistent-sqlite-2.8.1.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E           BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
+    local repository version: 5
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+

document that multiple groupwanted are not combined
diff --git a/doc/forum/Multiple_group_groupwanted_settings_help/comment_1_e78d0aa6b23225c509b9e97549fe915b._comment b/doc/forum/Multiple_group_groupwanted_settings_help/comment_1_e78d0aa6b23225c509b9e97549fe915b._comment
new file mode 100644
index 000000000..134f074a3
--- /dev/null
+++ b/doc/forum/Multiple_group_groupwanted_settings_help/comment_1_e78d0aa6b23225c509b9e97549fe915b._comment
@@ -0,0 +1,34 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-06-12T16:17:30Z"
+ content="""
+groupwanted expressions are only used when there is only one that the
+repository could use. The repository can be in two groups and if only one
+of the groups has a groupwanted expression, it will be used, but if both
+do, neither expression will be used. 
+
+This limitation to exactly one expression also holds for standard
+preferred content expressions (which is documented).
+git-annex does not try to combine together two expressions with OR.
+Why not? Well, consider the standard preferred content expressions for the 
+client and transfer groups. The one for transfer looks at whether client
+repositories have the content:
+`not (inallgroup=client and copies=client:2) ...`
+
+If a repository is both a transfer repository and a client at the same
+time, that would make it want to have content as long as not all clients
+contain the content -- so once all other clients get the content, the
+repository would want to drop its copy. But then, it would see that
+not all client repositories now have the content (since it doesn't!) and
+want to get it. And potentially round and round.. 
+
+git-annex actually detects and prevents this kind of drop/get cycle
+(probably), but the right behavior of combining those two groupwanted
+expressions is at best undefined. So it's better to have the user write
+down an expression with what they really want to happen than try to OR
+expressions.
+
+I've updated the docs for groupwanted to note that it's only used when one 
+of a repos's groups has it.
+"""]]
diff --git a/doc/git-annex-groupwanted.mdwn b/doc/git-annex-groupwanted.mdwn
index fd51d5038..7df2c0273 100644
--- a/doc/git-annex-groupwanted.mdwn
+++ b/doc/git-annex-groupwanted.mdwn
@@ -21,6 +21,10 @@ make repositories in the group want to contain 3 copies of every file:
 		git annex wanted $repo groupwanted
 	done
 
+Note that there must be exactly one groupwanted expression configured
+amoung all the groups that a repository is in; if there's more than one,
+none of them will be used.
+
 # SEE ALSO
 
 [[git-annex]](1)
diff --git a/doc/git-annex-preferred-content.mdwn b/doc/git-annex-preferred-content.mdwn
index 54f0d4666..97e5ff643 100644
--- a/doc/git-annex-preferred-content.mdwn
+++ b/doc/git-annex-preferred-content.mdwn
@@ -172,12 +172,13 @@ elsewhere to allow removing it).
 * `groupwanted`
 
   The "groupwanted" keyword can be used to refer to a preferred content
-  expression that is associated with a group. This is like the "standard"
-  keyword, but you can configure the preferred content expressions
-  using `git annex groupwanted`.
+  expression that is associated with a group, as long as there is exactly
+  one such expression amoung the groups a repository is in. This is like
+  the "standard" keyword, but you can configure the preferred content
+  expressions using `git annex groupwanted`.
 
-  Note that when writing a groupwanted preferred content expression,
-  you can use all of the keywords listed above, including "standard".
+  When writing a groupwanted preferred content expression,
+  you can use all the keywords documented here, including "standard".
   (But not "groupwanted".)
 
   For example, to make a variant of the standard client preferred content

response
diff --git a/doc/forum/safely_dropping_git-annex_history/comment_14_979bbccf31dbee1707d403957bde3493._comment b/doc/forum/safely_dropping_git-annex_history/comment_14_979bbccf31dbee1707d403957bde3493._comment
new file mode 100644
index 000000000..8c0d512d2
--- /dev/null
+++ b/doc/forum/safely_dropping_git-annex_history/comment_14_979bbccf31dbee1707d403957bde3493._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 14"""
+ date="2018-06-12T16:11:27Z"
+ content="""
+@Rizwan yes, the things documented here still work. But, they drop old
+history about the past locations of files, they do not drop *all* information
+about the current locations of files. git-annex is keeping track of which
+files you have stored in the local repository and on the remote, and that
+information necessisarily takes up space.
+
+You're using git-annex with small files -- around 12
+kilobytes average file size. It's designed for mostly much larger files,
+so you will see a larger percentage of overhead in using it than you would
+if your files were bigger.
+"""]]

Added a comment: Are these methods still working?
diff --git a/doc/forum/safely_dropping_git-annex_history/comment_13_b7b920862fc82eaa5d7deae34b4aebc4._comment b/doc/forum/safely_dropping_git-annex_history/comment_13_b7b920862fc82eaa5d7deae34b4aebc4._comment
new file mode 100644
index 000000000..7a8f62260
--- /dev/null
+++ b/doc/forum/safely_dropping_git-annex_history/comment_13_b7b920862fc82eaa5d7deae34b4aebc4._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="Rizwan"
+ avatar="http://cdn.libravatar.org/avatar/15c58acba4fc6f75abd783d5836e0254"
+ subject="Are these methods still working?"
+ date="2018-06-12T14:58:03Z"
+ content="""
+I am new to git annex. I like the concept of partial checkout, given that I have multiple storages. 
+
+My repo is small right now, 2000+ files collectively amounting to 24 MB in size. However, as soon as I add and unlock the repo, the size of the Annex folder becomes 50+ MB. Is this normal? I try to sync with google drive and the size goes to 90+ Mb post syncing. 
+
+I feel like I must be doing something wrong. Does the size of gorw upto 4 times the original data. I tried the methods listed in above comments, none worked. Seeing that last comment is 4+ years old, I wanted to know if these methods are still viable?
+"""]]

Added a comment: Termux Error
diff --git a/doc/tips/install_on_Android_in_Termux/comment_12_a7bbf9e61e2e993eca9a9222de899920._comment b/doc/tips/install_on_Android_in_Termux/comment_12_a7bbf9e61e2e993eca9a9222de899920._comment
new file mode 100644
index 000000000..474b3baab
--- /dev/null
+++ b/doc/tips/install_on_Android_in_Termux/comment_12_a7bbf9e61e2e993eca9a9222de899920._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="madapeedikakkaran@2c5c8bb4520ebf2526afb49c8dcbcb60fb295973"
+ nickname="madapeedikakkaran"
+ avatar="http://cdn.libravatar.org/avatar/15c58acba4fc6f75abd783d5836e0254"
+ subject="Termux Error"
+ date="2018-06-12T11:54:02Z"
+ content="""
+I followed the instruction to install git-annex on my Lenovo Vibe K4 running Android 6
+
+At first I got the error `/bin/sh/ is missing`, which I worked around with termux-fix-shebang on runshell. Now I am getting this error. 
+
+`proot info: vpid 1: terminated with signal 11`
+
+Please advice. 
+"""]]

diff --git a/doc/forum/Multiple_group_groupwanted_settings_help.mdwn b/doc/forum/Multiple_group_groupwanted_settings_help.mdwn
new file mode 100644
index 000000000..49781d91f
--- /dev/null
+++ b/doc/forum/Multiple_group_groupwanted_settings_help.mdwn
@@ -0,0 +1,34 @@
+Help me figure this out. I can't get it to work.
+
+Here's my hypothetical repo setup: a1 a2 b1 b2
+
+All of these are connected to each other.
+
+Settings are as follows:
+
+    group a1 = a 1
+    group a2 = a 2
+    group b1 = b 1
+    group b2 = b 2
+
+    wanted a1 = groupwanted
+    wanted a2 = groupwanted
+    wanted b1 = groupwanted
+    wanted b2 = groupwanted
+
+    groupwanted 1 = not copies=1:1
+    groupwanted 2 = not copies=2:1
+    groupwanted a = not copies=a:1
+    groupwanted b = not copies=b:1
+
+    numcopies default = 2
+    config annex.synccontent = true
+
+The behavior I want is: Group "a" repos will only have 1 copy of a file, and group "b" repos will only have 1 copy of a file.
+Similarly, group "1" repos will only have 1 copy of a file, and group "2" repos will only have 1 copy of a file.
+
+So, for example, a file could be stored in a1 and b2, or a2 and b1, but NOT a1 and a2 (both are group "a") or a1 and b1 (both are group "1").
+
+But it doesn't work. When I add a file to a1, it is not copied to b2 on sync. The documentation says repos can have multiple groups, but I'm not sure how the groupwanted settings of multiple groups work together.
+
+Is this even possible?

add GETINFO to external protocol (for ronnypfa)
External special remotes can now add info to `git annex info $remote`, by
replying to the GETINFO message.
Had to generalize some helpers to allow consuming multiple messages from
the remote.
The code added to Remote/* here is AGPL licensed, thus changed the license
of the files.
This commit was sponsored by Jake Vosloo on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index 6657f52a5..f70200913 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,8 @@ git-annex (6.20180530) UNRELEASED; urgency=medium
     way can be used.
   * Fix problems accessing repositories over http when annex.tune.*
     is configured.
+  * External special remotes can now add info to `git annex info $remote`,
+    by replying to the GETINFO message.
 
  -- Joey Hess <id@joeyh.name>  Wed, 30 May 2018 11:49:08 -0400
 
diff --git a/COPYRIGHT b/COPYRIGHT
index 1c689dac0..cf08323b9 100644
--- a/COPYRIGHT
+++ b/COPYRIGHT
@@ -10,7 +10,7 @@ Copyright: © 2012-2017 Joey Hess <id@joeyh.name>
            © 2014 Sören Brunk
 License: AGPL-3+
 
-Files: Remote/Git.hs Remote/Helper/Ssh.hs Remote/Adb.hs
+Files: Remote/Git.hs Remote/Helper/Ssh.hs Remote/Adb.hs Remote/External.hs Remote/Extermal/Types.hs
 Copyright: © 2011-2018 Joey Hess <id@joeyh.name>
 License: AGPL-3+
 
diff --git a/Remote/External.hs b/Remote/External.hs
index bbbf173a1..dada0d9d8 100644
--- a/Remote/External.hs
+++ b/Remote/External.hs
@@ -1,8 +1,8 @@
 {- External special remote interface.
  -
- - Copyright 2013-2016 Joey Hess <id@joeyh.name>
+ - Copyright 2013-2018 Joey Hess <id@joeyh.name>
  -
- - Licensed under the GNU GPL version 3 or higher.
+ - Licensed under the GNU AGPL version 3 or higher.
  -}
 
 module Remote.External (remote) where
@@ -61,6 +61,7 @@ gen r u c gc
 			readonlyRemoveKey
 			(checkKeyUrl r)
 			Nothing
+			(externalInfo externaltype)
 			Nothing
 			Nothing
 			exportUnsupported
@@ -94,12 +95,13 @@ gen r u c gc
 			(removeKeyM external)
 			(checkPresentM external)
 			(Just (whereisKeyM external))
+			(getInfoM external)
 			(Just (claimUrlM external))
 			(Just (checkUrlM external))
 			exportactions
 			cheapexportsupported
   where
-	mk cst avail tostore toretrieve toremove tocheckkey towhereis toclaimurl tocheckurl exportactions cheapexportsupported = do
+	mk cst avail tostore toretrieve toremove tocheckkey towhereis togetinfo toclaimurl tocheckurl exportactions cheapexportsupported = do
 		let rmt = Remote
 			{ uuid = u
 			, cost = cst
@@ -125,7 +127,7 @@ gen r u c gc
 				{ exportSupported = cheapexportsupported }
 			, mkUnavailable = gen r u c $
 				gc { remoteAnnexExternalType = Just "!dne!" }
-			, getInfo = return [("externaltype", externaltype)]
+			, getInfo = togetinfo
 			, claimUrl = toclaimurl
 			, checkUrl = tocheckurl
 			}
@@ -151,7 +153,7 @@ externalSetup _ mu _ c gc = do
 		_ -> do
 			external <- newExternal externaltype u c' gc
 			handleRequest external INITREMOTE Nothing $ \resp -> case resp of
-				INITREMOTE_SUCCESS -> Just noop
+				INITREMOTE_SUCCESS -> result ()
 				INITREMOTE_FAILURE errmsg -> Just $ giveup errmsg
 				_ -> Nothing
 			withExternalState external $
@@ -171,21 +173,20 @@ checkExportSupported' :: External -> Annex Bool
 checkExportSupported' external = go `catchNonAsync` (const (return False))
   where
 	go = handleRequest external EXPORTSUPPORTED Nothing $ \resp -> case resp of
-		EXPORTSUPPORTED_SUCCESS -> Just $ return True
-		EXPORTSUPPORTED_FAILURE -> Just $ return False
-		UNSUPPORTED_REQUEST -> Just $ return False
+		EXPORTSUPPORTED_SUCCESS -> result True
+		EXPORTSUPPORTED_FAILURE -> result False
+		UNSUPPORTED_REQUEST -> result False
 		_ -> Nothing
 
 storeKeyM :: External -> Storer
 storeKeyM external = fileStorer $ \k f p ->
 	handleRequestKey external (\sk -> TRANSFER Upload sk f) k (Just p) $ \resp ->
 		case resp of
-			TRANSFER_SUCCESS Upload k' | k == k' ->
-				Just $ return True
+			TRANSFER_SUCCESS Upload k' | k == k' -> result True
 			TRANSFER_FAILURE Upload k' errmsg | k == k' ->
 				Just $ do
 					warning errmsg
-					return False
+					return (Result False)
 			_ -> Nothing
 
 retrieveKeyFileM :: External -> Retriever
@@ -193,7 +194,7 @@ retrieveKeyFileM external = fileRetriever $ \d k p ->
 	handleRequestKey external (\sk -> TRANSFER Download sk d) k (Just p) $ \resp ->
 		case resp of
 			TRANSFER_SUCCESS Download k'
-				| k == k' -> Just $ return ()
+				| k == k' -> result ()
 			TRANSFER_FAILURE Download k' errmsg
 				| k == k' -> Just $ giveup errmsg
 			_ -> Nothing
@@ -203,11 +204,11 @@ removeKeyM external k = safely $
 	handleRequestKey external REMOVE k Nothing $ \resp ->
 		case resp of
 			REMOVE_SUCCESS k'
-				| k == k' -> Just $ return True
+				| k == k' -> result True
 			REMOVE_FAILURE k' errmsg
 				| k == k' -> Just $ do
 					warning errmsg
-					return False
+					return (Result False)
 			_ -> Nothing
 
 checkPresentM :: External -> CheckPresent
@@ -216,32 +217,31 @@ checkPresentM external k = either giveup id <$> go
 	go = handleRequestKey external CHECKPRESENT k Nothing $ \resp ->
 		case resp of
 			CHECKPRESENT_SUCCESS k'
-				| k' == k -> Just $ return $ Right True
+				| k' == k -> result $ Right True
 			CHECKPRESENT_FAILURE k'
-				| k' == k -> Just $ return $ Right False
+				| k' == k -> result $ Right False
 			CHECKPRESENT_UNKNOWN k' errmsg
-				| k' == k -> Just $ return $ Left errmsg
+				| k' == k -> result $ Left errmsg
 			_ -> Nothing
 
 whereisKeyM :: External -> Key -> Annex [String]
 whereisKeyM external k = handleRequestKey external WHEREIS k Nothing $ \resp -> case resp of
-	WHEREIS_SUCCESS s -> Just $ return [s]
-	WHEREIS_FAILURE -> Just $ return []
-	UNSUPPORTED_REQUEST -> Just $ return []
+	WHEREIS_SUCCESS s -> result [s]
+	WHEREIS_FAILURE -> result []
+	UNSUPPORTED_REQUEST -> result []
 	_ -> Nothing
 
 storeExportM :: External -> FilePath -> Key -> ExportLocation -> MeterUpdate -> Annex Bool
 storeExportM external f k loc p = safely $
 	handleRequestExport external loc req k (Just p) $ \resp -> case resp of
-		TRANSFER_SUCCESS Upload k' | k == k' ->
-			Just $ return True
+		TRANSFER_SUCCESS Upload k' | k == k' -> result True
 		TRANSFER_FAILURE Upload k' errmsg | k == k' ->
 			Just $ do
 				warning errmsg
-				return False
+				return (Result False)
 		UNSUPPORTED_REQUEST -> Just $ do
 			warning "TRANSFEREXPORT not implemented by external special remote"
-			return False
+			return (Result False)
 		_ -> Nothing
   where
 	req sk = TRANSFEREXPORT Upload sk f
@@ -250,14 +250,14 @@ retrieveExportM :: External -> Key -> ExportLocation -> FilePath -> MeterUpdate
 retrieveExportM external k loc d p = safely $
 	handleRequestExport external loc req k (Just p) $ \resp -> case resp of
 		TRANSFER_SUCCESS Download k'
-			| k == k' -> Just $ return True
+			| k == k' -> result True
 		TRANSFER_FAILURE Download k' errmsg
 			| k == k' -> Just $ do
 				warning errmsg
-				return False
+				return (Result False)
 		UNSUPPORTED_REQUEST -> Just $ do
 			warning "TRANSFEREXPORT not implemented by external special remote"
-			return False
+			return (Result False)
 		_ -> Nothing
   where
 	req sk = TRANSFEREXPORT Download sk d
@@ -267,12 +267,12 @@ checkPresentExportM external k loc = either giveup id <$> go
   where
 	go = handleRequestExport external loc CHECKPRESENTEXPORT k Nothing $ \resp -> case resp of
 		CHECKPRESENT_SUCCESS k'
-			| k' == k -> Just $ return $ Right True
+			| k' == k -> result $ Right True
 		CHECKPRESENT_FAILURE k'
-			| k' == k -> Just $ return $ Right False

(Diff truncated)
Added a comment
diff --git a/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_1_2e89bcc3fbd069e5d9eb12c1c19993ad._comment b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_1_2e89bcc3fbd069e5d9eb12c1c19993ad._comment
new file mode 100644
index 000000000..95c038836
--- /dev/null
+++ b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory/comment_1_2e89bcc3fbd069e5d9eb12c1c19993ad._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="yves.noirjean@3f9b06d19a920fbf5c82340c362e5971b00d4af2"
+ nickname="yves.noirjean"
+ avatar="http://cdn.libravatar.org/avatar/2f1ad9d443c037337d91f29781560344"
+ subject="comment 1"
+ date="2018-06-08T15:17:16Z"
+ content="""
+I wanted to try it with a repository using v5 indirect mode. This tells me:
+
+      Filesystem allows writing to files whose write bit is not set.
+    git-annex: This repository seems to be on a crippled filesystem, you must use direct mode.
+
+Is that the case with NFS4 in general or might the disk not be mounted properly?
+"""]]

diff --git a/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory.mdwn b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory.mdwn
index 493c535e0..94953b60e 100644
--- a/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory.mdwn
+++ b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory.mdwn
@@ -18,6 +18,7 @@ I do this on an NFS4 disk. The same on the local EXT4 disk works.
 I have noticed that annex creates a link for the file on EXT4 but not on NFS4. This seems suspicious.
 
 Version information:
+
     git version
     git version 2.16.2
 

diff --git a/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory.mdwn b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory.mdwn
new file mode 100644
index 000000000..493c535e0
--- /dev/null
+++ b/doc/forum/Out_of_memory_on_NFS4_with_files_bigger_than_available_memory.mdwn
@@ -0,0 +1,33 @@
+I do the following:
+
+    # create repo
+    git init repo
+    git -C ./repo annex init --version=6
+    git -C ./repo config annex.thin true
+    # add big file
+    cd repo
+    cp /tmp/bigfile . # 12GB file on system with 8GB RAM
+    git annex add bigfile
+    # commit
+    git commit -m 'msg'
+
+On the commit, I get "fatal: Out of memory, realloc failed". I found out that "git add" leads to the same error with big files, so I use "git annex add".
+
+I do this on an NFS4 disk. The same on the local EXT4 disk works.
+
+I have noticed that annex creates a link for the file on EXT4 but not on NFS4. This seems suspicious.
+
+Version information:
+    git version
+    git version 2.16.2
+
+    git annex version
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.7.1 persistent-sqlite-2.6.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
+    local repository version: 6
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+

initial but submission
diff --git a/doc/bugs/non-git-annex_symlinks_not_perserved_when_exporting_tree_via_rsync.mdwn b/doc/bugs/non-git-annex_symlinks_not_perserved_when_exporting_tree_via_rsync.mdwn
new file mode 100644
index 000000000..793b2daea
--- /dev/null
+++ b/doc/bugs/non-git-annex_symlinks_not_perserved_when_exporting_tree_via_rsync.mdwn
@@ -0,0 +1,15 @@
+### Please describe the problem.
+
+non-git-annex symlinks are not perserved when exporting tree via rsync. instead of symlinks on the remote destination, there are files that that contain text that describe the path of the local symlink.
+
+### What steps will reproduce the problem?
+
+add an ssh-based rsync remote with exporttree=yes and run `git annex export master --to test-rsync-remote`
+
+### What version of git-annex are you using? On what operating system?
+
+6.20180529 on Trisquel 8 GNU/Linux
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes, Git Annex has allowed us to transfer Gigabytes of data without a need for 7 GB of swap space and a git push command that takes hours to complete. :)

removal of the rest of remoteGitConfig
In keyUrls, the GitConfig is used only by annexLocations
to support configured Differences. Since such configurations affect all
clones of a repository, the local repo's GitConfig must have the same
information as the remote's GitConfig would have. So, used getGitConfig
to get the local GitConfig, which is cached and so available cheaply.
That actually fixed a bug noone had ever noticed: keyUrls is
used for remotes accessed over http. The full git config of such a
remote is normally not available, so the remoteGitConfig that keyUrls
used would not have the necessary information in it.
In copyFromRemoteCheap', it uses gitAnnexLocation,
which does need the GitConfig of the remote repo itself in order to
check if it's crippled, supports symlinks, etc. So, made the
State include that GitConfig, cached. The use of gitAnnexLocation is
within a (not $ Git.repoIsUrl repo) guard, so it's local, and so
its git config will always be read and available.
(Note that gitAnnexLocation in turn calls annexLocations, so the
Differences config it uses in this case comes from the remote repo's
GitConfig and not from the local repo's GitConfig. As explained above
this is ok since they must have the same value.)
Not very happy with this mess of different GitConfigs not type-safe and
some read only sometimes etc. Very hairy. Think I got it this change
right. Test suite passes..
This commit was sponsored by Ethan Aubin.
diff --git a/CHANGELOG b/CHANGELOG
index fa2523360..6657f52a5 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -3,8 +3,10 @@ git-annex (6.20180530) UNRELEASED; urgency=medium
   * Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change. 
   * version: Show operating system and repository version list 
     when run outside a git repo too.
-  * Fixed annex-checkuuid implementation, so that remotes configured that
+  * Fix annex-checkuuid implementation, so that remotes configured that
     way can be used.
+  * Fix problems accessing repositories over http when annex.tune.*
+    is configured.
 
  -- Joey Hess <id@joeyh.name>  Wed, 30 May 2018 11:49:08 -0400
 
diff --git a/Remote/GCrypt.hs b/Remote/GCrypt.hs
index b0594108e..37b708579 100644
--- a/Remote/GCrypt.hs
+++ b/Remote/GCrypt.hs
@@ -124,7 +124,7 @@ gen' r u c gc = do
 		, config = c
 		, localpath = localpathCalc r
 		, getRepo = return r
-		, gitconfig = gc { remoteGitConfig = extractGitConfig r }
+		, gitconfig = gc
 		, readonly = Git.repoIsHttp r
 		, availability = availabilityCalc r
 		, remotetype = remote
diff --git a/Remote/Git.hs b/Remote/Git.hs
index 61659e733..da177bac5 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -176,7 +176,7 @@ gen r u c gc
 			, config = c
 			, localpath = localpathCalc r
 			, getRepo = getRepoFromState st
-			, gitconfig = gc { remoteGitConfig = extractGitConfig r }
+			, gitconfig = gc
 			, readonly = Git.repoIsHttp r
 			, availability = availabilityCalc r
 			, remotetype = remote
@@ -340,8 +340,9 @@ inAnnex' repo rmt (State connpool duc _) key
   where
 	checkhttp = do
 		showChecking repo
+		gc <- Annex.getGitConfig
 		ifM (Url.withUrlOptions $ \uo -> liftIO $
-			anyM (\u -> Url.checkBoth u (keySize key) uo) (keyUrls repo rmt key))
+			anyM (\u -> Url.checkBoth u (keySize key) uo) (keyUrls gc repo rmt key))
 				( return True
 				, giveup "not found"
 				)
@@ -355,22 +356,21 @@ inAnnex' repo rmt (State connpool duc _) key
 		, cantCheck repo
 		)
 
-keyUrls :: Git.Repo -> Remote -> Key -> [String]
-keyUrls repo r key = map tourl locs'
+keyUrls :: GitConfig -> Git.Repo -> Remote -> Key -> [String]
+keyUrls gc repo r key = map tourl locs'
   where
 	tourl l = Git.repoLocation repo ++ "/" ++ l
 	-- If the remote is known to not be bare, try the hash locations
 	-- used for non-bare repos first, as an optimisation.
 	locs
-		| remoteAnnexBare remoteconfig == Just False = reverse (annexLocations cfg key)
-		| otherwise = annexLocations cfg key
+		| remoteAnnexBare remoteconfig == Just False = reverse (annexLocations gc key)
+		| otherwise = annexLocations gc key
 #ifndef mingw32_HOST_OS
 	locs' = locs
 #else
 	locs' = map (replace "\\" "/") locs
 #endif
 	remoteconfig = gitconfig r
-	cfg = remoteGitConfig remoteconfig
 
 dropKey :: Remote -> State -> Key -> Annex Bool
 dropKey r st key = do
@@ -471,8 +471,9 @@ copyFromRemote' forcersync r st key file dest meterupdate = do
 
 copyFromRemote'' :: Git.Repo -> Bool -> Remote -> State -> Key -> AssociatedFile -> FilePath -> MeterUpdate -> Annex (Bool, Verification)
 copyFromRemote'' repo forcersync r (State connpool _ _) key file dest meterupdate
-	| Git.repoIsHttp repo = unVerified $
-		Annex.Content.downloadUrl key meterupdate (keyUrls repo r key) dest
+	| Git.repoIsHttp repo = unVerified $ do
+		gc <- Annex.getGitConfig
+		Annex.Content.downloadUrl key meterupdate (keyUrls gc repo r key) dest
 	| not $ Git.repoIsUrl repo = guardUsable repo (unVerified (return False)) $ do
 		params <- Ssh.rsyncParams r Download
 		u <- getUUID
@@ -567,10 +568,10 @@ copyFromRemoteCheap r st key af file = do
 copyFromRemoteCheap' :: Git.Repo -> Remote -> State -> Key -> AssociatedFile -> FilePath -> Annex Bool
 #ifndef mingw32_HOST_OS
 copyFromRemoteCheap' repo r st key af file
-	| not $ Git.repoIsUrl repo = guardUsable repo (return False) $ liftIO $ do
-		loc <- gitAnnexLocation key repo $
-			remoteGitConfig $ gitconfig r
-		ifM (doesFileExist loc)
+	| not $ Git.repoIsUrl repo = guardUsable repo (return False) $ do
+		gc <- getGitConfigFromState st
+		loc <- liftIO $ gitAnnexLocation key repo gc
+		liftIO $ ifM (doesFileExist loc)
 			( do
 				absloc <- absPath loc
 				catchBoolIO $ do
@@ -782,10 +783,14 @@ mkCopier remotewanthardlink rsyncparams = do
  - This returns False when the repository UUID is not as expected. -}
 type DeferredUUIDCheck = Annex Bool
 
-data State = State Ssh.P2PSshConnectionPool DeferredUUIDCheck (Annex Git.Repo)
+data State = State Ssh.P2PSshConnectionPool DeferredUUIDCheck (Annex (Git.Repo, GitConfig))
 
 getRepoFromState :: State -> Annex Git.Repo
-getRepoFromState (State _ _ a) = a
+getRepoFromState (State _ _ a) = fst <$> a
+
+{- The config of the remote git repository, cached for speed. -}
+getGitConfigFromState :: State -> Annex GitConfig
+getGitConfigFromState (State _ _ a) = snd <$> a
 
 mkState :: Git.Repo -> UUID -> RemoteGitConfig -> Annex State
 mkState r u gc = do
@@ -794,21 +799,23 @@ mkState r u gc = do
 	return $ State pool duc getrepo
   where
 	go
-		| remoteAnnexCheckUUID gc = return (return True, return r)
+		| remoteAnnexCheckUUID gc = return
+			(return True, return (r, extractGitConfig r))
 		| otherwise = do
 			rv <- liftIO newEmptyMVar
 			let getrepo = ifM (liftIO $ isEmptyMVar rv)
 				( do
 					r' <- tryGitConfigRead False r
-					void $ liftIO $ tryPutMVar rv r'
-					return r'
+					let t = (r', extractGitConfig r')
+					void $ liftIO $ tryPutMVar rv t
+					return t
 				, liftIO $ readMVar rv
 				)
 
 			cv <- liftIO newEmptyMVar
 			let duc = ifM (liftIO $ isEmptyMVar cv)
 				( do
-					r' <- getrepo
+					r' <- fst <$> getrepo
 					u' <- getRepoUUID r'
 					let ok = u' == u
 					void $ liftIO $ tryPutMVar cv ok
diff --git a/Remote/P2P.hs b/Remote/P2P.hs
index 79ce589a3..6ea029da8 100644
--- a/Remote/P2P.hs
+++ b/Remote/P2P.hs
@@ -65,7 +65,7 @@ chainGen addr r u c gc = do
 		, config = c
 		, localpath = Nothing
 		, getRepo = return r
-		, gitconfig = gc { remoteGitConfig = extractGitConfig r }
+		, gitconfig = gc
 		, readonly = False
 		, availability = GloballyAvailable
 		, remotetype = remote
diff --git a/Types/GitConfig.hs b/Types/GitConfig.hs
index 5984f1605..508d02f4d 100644
--- a/Types/GitConfig.hs
+++ b/Types/GitConfig.hs
@@ -19,6 +19,7 @@ import Common
 import qualified Git
 import qualified Git.Config
 import qualified Git.Construct
+import Git.Types
 import Git.ConfigTypes
 import Utility.DataUnits
 import Config.Cost
@@ -195,7 +196,12 @@ mergeGitConfig gitconfig repoglobals = gitconfig
 
 {- Per-remote git-annex settings. Each setting corresponds to a git-config
  - key such as <remote>.annex-foo, or if that is not set, a default from
- - annex.foo -}
+ - annex.foo.
+ -
+ - Note that this is from the perspective of the local repository,
+ - it is not influenced in any way by the contents of the remote
+ - repository's git config.
+ -}
 data RemoteGitConfig = RemoteGitConfig
 	{ remoteAnnexCost :: DynamicConfig (Maybe Cost)
 	, remoteAnnexIgnore :: DynamicConfig Bool
@@ -235,11 +241,11 @@ data RemoteGitConfig = RemoteGitConfig
 	, remoteAnnexDdarRepo :: Maybe String
 	, remoteAnnexHookType :: Maybe String
 	, remoteAnnexExternalType :: Maybe String
-	{- A regular git remote's git repository config. -}
-	, remoteGitConfig :: GitConfig
 	}
 
-extractRemoteGitConfig :: Git.Repo -> String -> STM RemoteGitConfig

(Diff truncated)
remove use of remoteGitConfig
Unfortunately one more use remains..
This should be just as fast as the other method. The remote's Git.Repo
has already had its config read, so Annex.new's call to Git.Config.read
is a noop.
Thid commit was sponsored by andrea rota.
diff --git a/Command/Sync.hs b/Command/Sync.hs
index 97a65452a..0fb3bdc3f 100644
--- a/Command/Sync.hs
+++ b/Command/Sync.hs
@@ -454,25 +454,27 @@ pushRemote o remote (Just branch, _) = stopUnless (pure (pushOption o) <&&> need
 				showLongNote "(non-fast-forward problems can be solved by setting receive.denyNonFastforwards to false in the remote's git config)"
 				return ok
   where
+	gc = Remote.gitconfig remote
 	needpush
 		| remoteAnnexReadOnly gc = return False
 		| not (remoteAnnexPush gc) = return False
 		| otherwise = anyM (newer remote) [syncBranch branch, Annex.Branch.name]
 	-- Do updateInstead emulation for remotes on eg removable drives
-	-- formatted FAT, where the post-update hook won't run.
-	postpushupdate repo
-		| annexCrippledFileSystem (remoteGitConfig (Remote.gitconfig remote)) =
-			case Git.repoWorkTree repo of
-				Nothing -> return True
-				Just wt -> ifM (Remote.Git.onLocal repo remote needUpdateInsteadEmulation)
-					( liftIO $ do
-						p <- readProgramFile
-						boolSystem' p [Param "post-receive"]
-							(\cp -> cp { cwd = Just wt })
-					, return True
-					)
-		| otherwise = return True
-	gc = Remote.gitconfig remote
+	-- formatted FAT, where the post-receive hook won't run.
+	postpushupdate repo = case Git.repoWorkTree repo of
+		Nothing -> return True
+		Just wt -> ifM needemulation
+			( liftIO $ do
+				p <- readProgramFile
+				boolSystem' p [Param "post-receive"]
+					(\cp -> cp { cwd = Just wt })
+			, return True
+			)
+	  where
+		needemulation = Remote.Git.onLocal repo remote $
+			(annexCrippledFileSystem <$> Annex.getGitConfig)
+				<&&>
+			needUpdateInsteadEmulation			
 
 {- Pushes a regular branch like master to a remote. Also pushes the git-annex
  - branch.
diff --git a/doc/todo/need_to_remove_remoteGitConfig_for_checkuuid_support.mdwn b/doc/todo/need_to_remove_remoteGitConfig_for_checkuuid_support.mdwn
index 9a8a82160..81a1f9ba6 100644
--- a/doc/todo/need_to_remove_remoteGitConfig_for_checkuuid_support.mdwn
+++ b/doc/todo/need_to_remove_remoteGitConfig_for_checkuuid_support.mdwn
@@ -1,12 +1,10 @@
 annex-checkuuid=false prevents the git config of a remote from being read.
 So, the remoteGitConfig will be an empty config when that's set.
 
-Only a few things use remoteGitConfig. Annex.Ssh uses it, but is not
-impacted by the problem. 
-
-And `git annex sync` looks at it to determine
-if the remote is a FAT-formatted drive, and does updateInstead emulation.
-So, that's broken for remotes with annex-checkuuid=false
+I've mostly removed uses of remoteGitConfig, but there are two in
+Remote.Git, which are needed for annexDifferences.
+So, `annex.tune.*` config the remote won't be honored when
+annex-checkuuid=false.
 
 The best thing would be to remove remoteGitConfig, to avoid such problems
 in the future. --[[Joey]]

close
diff --git a/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed..mdwn b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed..mdwn
index 726226d66..bf2972257 100644
--- a/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed..mdwn
+++ b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed..mdwn
@@ -33,3 +33,5 @@ git annex find --copies=trusted:1
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 Yes! Thanks so much for your work!
+
+> Closing per comments. [[done]] --[[Joey]]

fix incorrect comment
diff --git a/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_2_298b1a8418380d932464962ca00fb2f3._comment b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_2_298b1a8418380d932464962ca00fb2f3._comment
index e00bbe651..625ed48de 100644
--- a/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_2_298b1a8418380d932464962ca00fb2f3._comment
+++ b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_2_298b1a8418380d932464962ca00fb2f3._comment
@@ -3,7 +3,10 @@
  subject="""comment 2"""
  date="2018-06-04T17:40:23Z"
  content="""
-Actually, Remote.gitconfig is not a problem; it contains
-the local repos's remote config settings, not the remote repo's
-own git config.
+I've changed Remote.repo to Remote.getRepo so IO can be done there.
+
+The remaining problem is the remoteGitConfig part
+of RemoteGitConfig. That is dependent on the git config of the remote being
+read when a Remote is constructed, and when checkuuid=false, that won't be
+done. Only a few things use it, but my first try at removing it failed.
 """]]

fix annex-checkuuid
Fixed annex-checkuuid implementation, so that remotes configured that way
can be used. This was 100% broken from the first commit of it, oops.
This commit was sponsored by Øyvind Andersen Holm.
diff --git a/CHANGELOG b/CHANGELOG
index 2462900d5..fa2523360 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -3,6 +3,8 @@ git-annex (6.20180530) UNRELEASED; urgency=medium
   * Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change. 
   * version: Show operating system and repository version list 
     when run outside a git repo too.
+  * Fixed annex-checkuuid implementation, so that remotes configured that
+    way can be used.
 
  -- Joey Hess <id@joeyh.name>  Wed, 30 May 2018 11:49:08 -0400
 
diff --git a/Remote/Git.hs b/Remote/Git.hs
index d44bde0e8..61659e733 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -175,7 +175,7 @@ gen r u c gc
 				else Just $ repairRemote r
 			, config = c
 			, localpath = localpathCalc r
-			, getRepo = return r
+			, getRepo = getRepoFromState st
 			, gitconfig = gc { remoteGitConfig = extractGitConfig r }
 			, readonly = Git.repoIsHttp r
 			, availability = availabilityCalc r
@@ -333,7 +333,7 @@ inAnnex rmt st key = do
 	inAnnex' repo rmt st key
 
 inAnnex' :: Git.Repo -> Remote -> State -> Key -> Annex Bool
-inAnnex' repo rmt (State connpool duc) key
+inAnnex' repo rmt (State connpool duc _) key
 	| Git.repoIsHttp repo = checkhttp
 	| Git.repoIsUrl repo = checkremote
 	| otherwise = checklocal
@@ -378,7 +378,7 @@ dropKey r st key = do
 	dropKey' repo r st key
 
 dropKey' :: Git.Repo -> Remote -> State -> Key -> Annex Bool
-dropKey' repo r (State connpool duc) key
+dropKey' repo r (State connpool duc _) key
 	| not $ Git.repoIsUrl repo = ifM duc
 		( guardUsable repo (return False) $
 			commitOnCleanup repo r $ onLocalFast repo r $ do
@@ -402,7 +402,7 @@ lockKey r st key callback = do
 	lockKey' repo r st key callback
 
 lockKey' :: Git.Repo -> Remote -> State -> Key -> (VerifiedCopy -> Annex r) -> Annex r
-lockKey' repo r (State connpool duc) key callback
+lockKey' repo r (State connpool duc _) key callback
 	| not $ Git.repoIsUrl repo = ifM duc
 		( guardUsable repo failedlock $ do
 			inorigrepo <- Annex.makeRunner
@@ -470,7 +470,7 @@ copyFromRemote' forcersync r st key file dest meterupdate = do
 	copyFromRemote'' repo forcersync r st key file dest meterupdate
 
 copyFromRemote'' :: Git.Repo -> Bool -> Remote -> State -> Key -> AssociatedFile -> FilePath -> MeterUpdate -> Annex (Bool, Verification)
-copyFromRemote'' repo forcersync r (State connpool _) key file dest meterupdate
+copyFromRemote'' repo forcersync r (State connpool _ _) key file dest meterupdate
 	| Git.repoIsHttp repo = unVerified $
 		Annex.Content.downloadUrl key meterupdate (keyUrls repo r key) dest
 	| not $ Git.repoIsUrl repo = guardUsable repo (unVerified (return False)) $ do
@@ -595,7 +595,7 @@ copyToRemote r st key file meterupdate = do
 	copyToRemote' repo r st key file meterupdate
 
 copyToRemote' :: Git.Repo -> Remote -> State -> Key -> AssociatedFile -> MeterUpdate -> Annex Bool
-copyToRemote' repo r (State connpool duc) key file meterupdate
+copyToRemote' repo r (State connpool duc _) key file meterupdate
 	| not $ Git.repoIsUrl repo = ifM duc
 		( guardUsable repo (return False) $ commitOnCleanup repo r $
 			copylocal =<< Annex.Content.prepSendAnnex key
@@ -775,13 +775,6 @@ mkCopier remotewanthardlink rsyncparams = do
 		, return copier
 		)
 
-data State = State Ssh.P2PSshConnectionPool DeferredUUIDCheck
-
-mkState :: Git.Repo -> UUID -> RemoteGitConfig -> Annex State
-mkState r u gc = State
-	<$> Ssh.mkP2PSshConnectionPool
-	<*> mkDeferredUUIDCheck r u gc
-
 {- Normally the UUID of a local repository is checked at startup,
  - but annex-checkuuid config can prevent that. To avoid getting
  - confused, a deferred check is done just before the repository
@@ -789,19 +782,40 @@ mkState r u gc = State
  - This returns False when the repository UUID is not as expected. -}
 type DeferredUUIDCheck = Annex Bool
 
-mkDeferredUUIDCheck :: Git.Repo -> UUID -> RemoteGitConfig -> Annex DeferredUUIDCheck
-mkDeferredUUIDCheck r u gc
-	| remoteAnnexCheckUUID gc = return (return True)
-	| otherwise = do
-		v <- liftIO newEmptyMVar
-		return $ ifM (liftIO $ isEmptyMVar v)
-			( do
-				r' <- tryGitConfigRead False r
-				u' <- getRepoUUID r'
-				let ok = u' == u
-				void $ liftIO $ tryPutMVar v ok
-				unless ok $
-					warning $ Git.repoDescribe r ++ " is not the expected repository. The remote's annex-checkuuid configuration prevented noticing the change until now."
-				return ok
-			, liftIO $ readMVar v
-			)
+data State = State Ssh.P2PSshConnectionPool DeferredUUIDCheck (Annex Git.Repo)
+
+getRepoFromState :: State -> Annex Git.Repo
+getRepoFromState (State _ _ a) = a
+
+mkState :: Git.Repo -> UUID -> RemoteGitConfig -> Annex State
+mkState r u gc = do
+	pool <- Ssh.mkP2PSshConnectionPool
+	(duc, getrepo) <- go
+	return $ State pool duc getrepo
+  where
+	go
+		| remoteAnnexCheckUUID gc = return (return True, return r)
+		| otherwise = do
+			rv <- liftIO newEmptyMVar
+			let getrepo = ifM (liftIO $ isEmptyMVar rv)
+				( do
+					r' <- tryGitConfigRead False r
+					void $ liftIO $ tryPutMVar rv r'
+					return r'
+				, liftIO $ readMVar rv
+				)
+
+			cv <- liftIO newEmptyMVar
+			let duc = ifM (liftIO $ isEmptyMVar cv)
+				( do
+					r' <- getrepo
+					u' <- getRepoUUID r'
+					let ok = u' == u
+					void $ liftIO $ tryPutMVar cv ok
+					unless ok $
+						warning $ Git.repoDescribe r ++ " is not the expected repository. The remote's annex-checkuuid configuration prevented noticing the change until now."
+					return ok
+				, liftIO $ readMVar cv
+				)
+
+			return (duc, getrepo)
diff --git a/doc/bugs/annex-checkuuid_renders_remotes_inaccessible.mdwn b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible.mdwn
index 6e634ac19..812d8e00f 100644
--- a/doc/bugs/annex-checkuuid_renders_remotes_inaccessible.mdwn
+++ b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible.mdwn
@@ -38,3 +38,5 @@ Setting `remote.<name>.annex-checkuuid` to `false` renders remote `<name>` inacc
     dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.2 feed-1.0.0.0 ghc-8.4.2 http-client-0.5.12.1 persistent-sqlite-2.8.1.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
     key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
     remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_3_4e68ba8e7bc1e3d935093bf45c5afb10._comment b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_3_4e68ba8e7bc1e3d935093bf45c5afb10._comment
new file mode 100644
index 000000000..74868844e
--- /dev/null
+++ b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_3_4e68ba8e7bc1e3d935093bf45c5afb10._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2018-06-04T20:43:13Z"
+ content="""
+Since I have been working on this for 4 hours now and want to
+end with something usable today, I went ahead with the fix.
+
+Opened a todo to keep track of the remaining problems with the remote's git
+config not being read:
+[[todo/need_to_remove_remoteGitConfig_for_checkuuid_support]]
+"""]]
diff --git a/doc/todo/need_to_remove_remoteGitConfig_for_checkuuid_support.mdwn b/doc/todo/need_to_remove_remoteGitConfig_for_checkuuid_support.mdwn
new file mode 100644
index 000000000..9a8a82160
--- /dev/null
+++ b/doc/todo/need_to_remove_remoteGitConfig_for_checkuuid_support.mdwn
@@ -0,0 +1,12 @@
+annex-checkuuid=false prevents the git config of a remote from being read.
+So, the remoteGitConfig will be an empty config when that's set.
+
+Only a few things use remoteGitConfig. Annex.Ssh uses it, but is not
+impacted by the problem. 
+
+And `git annex sync` looks at it to determine
+if the remote is a FAT-formatted drive, and does updateInstead emulation.
+So, that's broken for remotes with annex-checkuuid=false
+
+The best thing would be to remove remoteGitConfig, to avoid such problems
+in the future. --[[Joey]]

Added a comment
diff --git a/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed./comment_2_73b2d20a793a8a17cf0a3d593b3b4ed4._comment b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed./comment_2_73b2d20a793a8a17cf0a3d593b3b4ed4._comment
new file mode 100644
index 000000000..c9b6f38a7
--- /dev/null
+++ b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed./comment_2_73b2d20a793a8a17cf0a3d593b3b4ed4._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="ypid"
+ avatar="http://cdn.libravatar.org/avatar/4286841cddc77176758dde874afe9a3a"
+ subject="comment 2"
+ date="2018-06-04T19:42:30Z"
+ content="""
+Works for me, thanks. I also think this is not a real issue, I just noticed the edge case.
+"""]]

change Remote.repo to Remote.getRepo
This is groundwork for letting a repo be instantiated the first time
it's actually used, instead of at startup.
The only behavior change is that some old special cases for xmpp remotes
were removed. Where before git-annex silently did nothing with those
no-longer supported remotes, it may now fail in some way.
The additional IO action should have no performance impact as long as
it's simply return.
This commit was sponsored by Boyd Stephen Smith Jr. on Patreon
diff --git a/Assistant/DaemonStatus.hs b/Assistant/DaemonStatus.hs
index 2cb5574ab..2f1df5725 100644
--- a/Assistant/DaemonStatus.hs
+++ b/Assistant/DaemonStatus.hs
@@ -55,8 +55,7 @@ calcSyncRemotes = do
 	let good r = Remote.uuid r `elem` alive
 	let syncable = filter good rs
 	contentremotes <- filterM (not <$$> liftIO . getDynamicConfig . remoteAnnexIgnore . Remote.gitconfig) $
-		filter (\r -> Remote.uuid r /= NoUUID) $
-		filter (not . Remote.isXMPPRemote) syncable
+		filter (\r -> Remote.uuid r /= NoUUID) syncable
 	let (exportremotes, dataremotes) = partition (exportTree . Remote.config) contentremotes
 
 	return $ \dstatus -> dstatus
diff --git a/Assistant/Pairing/MakeRemote.hs b/Assistant/Pairing/MakeRemote.hs
index a97bb31f0..22baffe67 100644
--- a/Assistant/Pairing/MakeRemote.hs
+++ b/Assistant/Pairing/MakeRemote.hs
@@ -47,7 +47,8 @@ finishedLocalPairing msg keypair = do
 			("git-annex-shell -c configlist " ++ T.unpack (sshDirectory sshdata))
 			Nothing
 	r <- liftAnnex $ addRemote $ makeSshRemote sshdata
-	liftAnnex $ setRemoteCost (Remote.repo r) semiExpensiveRemoteCost
+	repo <- liftAnnex $ Remote.getRepo r
+	liftAnnex $ setRemoteCost repo semiExpensiveRemoteCost
 	syncRemote r
 
 {- Mostly a straightforward conversion.  Except:
diff --git a/Assistant/Sync.hs b/Assistant/Sync.hs
index 5f08ec81c..508b86efa 100644
--- a/Assistant/Sync.hs
+++ b/Assistant/Sync.hs
@@ -64,26 +64,25 @@ reconnectRemotes rs = void $ do
 		mapM_ signal $ filter (`notElem` failedrs) rs'
 	recordExportCommit
   where
-	gitremotes = filter (notspecialremote . Remote.repo) rs
-	(_xmppremotes, nonxmppremotes) = partition Remote.isXMPPRemote rs
+	gitremotes = liftAnnex $ 
+		filterM (notspecialremote <$$> Remote.getRepo) rs
 	notspecialremote r
 		| Git.repoIsUrl r = True
 		| Git.repoIsLocal r = True
 		| Git.repoIsLocalUnknown r = True
 		| otherwise = False
 	sync currentbranch@(Just _, _) = do
-		(failedpull, diverged) <- manualPull currentbranch gitremotes
+		(failedpull, diverged) <- manualPull currentbranch =<< gitremotes
 		now <- liftIO getCurrentTime
-		failedpush <- pushToRemotes' now gitremotes
+		failedpush <- pushToRemotes' now =<< gitremotes
 		return (nub $ failedpull ++ failedpush, diverged)
 	{- No local branch exists yet, but we can try pulling. -}
-	sync (Nothing, _) = manualPull (Nothing, Nothing) gitremotes
+	sync (Nothing, _) = manualPull (Nothing, Nothing) =<< gitremotes
 	go = do
 		(failed, diverged) <- sync
 			=<< liftAnnex (join Command.Sync.getCurrBranch)
 		addScanRemotes diverged =<<
-			filterM (not <$$> liftIO . getDynamicConfig . remoteAnnexIgnore . Remote.gitconfig)
-				nonxmppremotes
+			filterM (not <$$> liftIO . getDynamicConfig . remoteAnnexIgnore . Remote.gitconfig) rs
 		return failed
 	signal r = liftIO . mapM_ (flip tryPutMVar ())
 		=<< fromMaybe [] . M.lookup (Remote.uuid r) . connectRemoteNotifiers
@@ -130,8 +129,7 @@ pushToRemotes' now remotes = do
 			<$> gitRepo
 			<*> join Command.Sync.getCurrBranch
 			<*> getUUID
-	let (_xmppremotes, normalremotes) = partition Remote.isXMPPRemote remotes
-	ret <- go True branch g u normalremotes
+	ret <- go True branch g u remotes
 	return ret
   where
 	go _ (Nothing, _) _ _ _ = return [] -- no branch, so nothing to do
@@ -174,7 +172,8 @@ parallelPush g rs a = do
   where
 	topush r = (,)
 		<$> pure r
-		<*> sshOptionsTo (Remote.repo r) (Remote.gitconfig r) g
+		<*> (Remote.getRepo r >>= \repo ->
+			sshOptionsTo repo (Remote.gitconfig r) g)
 
 {- Displays an alert while running an action that syncs with some remotes,
  - and returns any remotes that it failed to sync with.
@@ -187,7 +186,7 @@ syncAction rs a
 	| otherwise = do
 		i <- addAlert $ syncAlert visibleremotes
 		failed <- a rs
-		let failed' = filter (not . Git.repoIsLocalUnknown . Remote.repo) failed
+		failed' <- filterM (not . Git.repoIsLocalUnknown <$$> liftAnnex . Remote.getRepo) failed
 		let succeeded = filter (`notElem` failed) visibleremotes
 		if null succeeded && null failed'
 			then removeAlert i
@@ -195,8 +194,7 @@ syncAction rs a
 				syncResultAlert succeeded failed'
 		return failed
   where
-	visibleremotes = filter (not . Remote.readonly) $
-		filter (not . Remote.isXMPPRemote) rs
+	visibleremotes = filter (not . Remote.readonly) rs
 
 {- Manually pull from remotes and merge their branches. Returns any
  - remotes that it failed to pull from, and a Bool indicating
@@ -206,17 +204,18 @@ syncAction rs a
 manualPull :: Command.Sync.CurrBranch -> [Remote] -> Assistant ([Remote], Bool)
 manualPull currentbranch remotes = do
 	g <- liftAnnex gitRepo
-	let (_xmppremotes, normalremotes) = partition Remote.isXMPPRemote remotes
-	failed <- forM normalremotes $ \r -> if wantpull $ Remote.gitconfig r
+	failed <- forM remotes $ \r -> if wantpull $ Remote.gitconfig r
 		then do
-			g' <- liftAnnex $ sshOptionsTo (Remote.repo r) (Remote.gitconfig r) g
+			g' <- liftAnnex $ do
+				repo <- Remote.getRepo r
+				sshOptionsTo repo (Remote.gitconfig r) g
 			ifM (liftIO $ Git.Command.runBool [Param "fetch", Param $ Remote.name r] g')
 				( return Nothing
 				, return $ Just r
 				)
 		else return Nothing
 	haddiverged <- liftAnnex Annex.Branch.forceUpdate
-	forM_ normalremotes $ \r ->
+	forM_ remotes $ \r ->
 		liftAnnex $ Command.Sync.mergeRemote r
 			currentbranch Command.Sync.mergeConfig def
 	when haddiverged $
@@ -263,10 +262,10 @@ changeSyncable (Just r) False = do
 
 changeSyncFlag :: Remote -> Bool -> Annex ()
 changeSyncFlag r enabled = do
+	repo <- Remote.getRepo r
+	let key = Config.remoteConfig repo "sync"
 	Config.setConfig key (boolConfig enabled)
 	void Remote.remoteListRefresh
-  where
-	key = Config.remoteConfig (Remote.repo r) "sync"
 
 updateExportTreeFromLogAll :: Assistant ()
 updateExportTreeFromLogAll = do
diff --git a/Assistant/Threads/Cronner.hs b/Assistant/Threads/Cronner.hs
index 145a76e7b..3e2153138 100644
--- a/Assistant/Threads/Cronner.hs
+++ b/Assistant/Threads/Cronner.hs
@@ -210,11 +210,11 @@ runActivity' urlrenderer (ScheduledRemoteFsck u s d) = dispatch =<< liftAnnex (r
 			 - Annex monad. -}
 			go rmt =<< liftAnnex (mkfscker (annexFsckParams d))
 	go rmt annexfscker = do
+		repo <- liftAnnex $ Remote.getRepo rmt
 		fsckresults <- showFscking urlrenderer (Just rmt) $ tryNonAsync $ do
 			void annexfscker
-			let r = Remote.repo rmt
-			if Git.repoIsLocal r && not (Git.repoIsLocalUnknown r)
-				then Just <$> Git.Fsck.findBroken True r
+			if Git.repoIsLocal repo && not (Git.repoIsLocalUnknown repo)
+				then Just <$> Git.Fsck.findBroken True repo
 				else pure Nothing
 		maybe noop (void . repairWhenNecessary urlrenderer u (Just rmt)) fsckresults
 
diff --git a/Assistant/Threads/MountWatcher.hs b/Assistant/Threads/MountWatcher.hs
index bd8d0e614..c5d075f86 100644
--- a/Assistant/Threads/MountWatcher.hs
+++ b/Assistant/Threads/MountWatcher.hs
@@ -144,7 +144,8 @@ handleMounts urlrenderer wasmounted nowmounted =
 handleMount :: UrlRenderer -> FilePath -> Assistant ()
 handleMount urlrenderer dir = do
 	debug ["detected mount of", dir]
-	rs <- filter (Git.repoIsLocal . Remote.repo) <$> remotesUnder dir
+	rs <- filterM (Git.repoIsLocal <$$> liftAnnex . Remote.getRepo)
+		=<< remotesUnder dir
 	mapM_ (fsckNudge urlrenderer . Just) rs
 	reconnectRemotes rs
 
diff --git a/Assistant/Threads/ProblemFixer.hs b/Assistant/Threads/ProblemFixer.hs
index 86ee027f7..19f7ccca2 100644
--- a/Assistant/Threads/ProblemFixer.hs
+++ b/Assistant/Threads/ProblemFixer.hs
@@ -49,20 +49,23 @@ handleProblem urlrenderer repoproblem = do
 		liftIO $ afterFix repoproblem
 
 handleRemoteProblem :: UrlRenderer -> Remote -> Assistant Bool
-handleRemoteProblem urlrenderer rmt
-	| Git.repoIsLocal r && not (Git.repoIsLocalUnknown r) =
+handleRemoteProblem urlrenderer rmt = do
+	repo <- liftAnnex $ Remote.getRepo rmt
+	handleRemoteProblem' repo urlrenderer rmt
+
+handleRemoteProblem' :: Git.Repo -> UrlRenderer -> Remote -> Assistant Bool
+handleRemoteProblem' repo urlrenderer rmt
+	| Git.repoIsLocal repo && not (Git.repoIsLocalUnknown repo) =
 		ifM (liftIO $ checkAvailable True rmt)
 			( do
-				fixedlocks <- repairStaleGitLocks r
+				fixedlocks <- repairStaleGitLocks repo
 				fsckresults <- showFscking urlrenderer (Just rmt) $ tryNonAsync $
-					Git.Fsck.findBroken True r
+					Git.Fsck.findBroken True repo
 				repaired <- repairWhenNecessary urlrenderer (Remote.uuid rmt) (Just rmt) fsckresults
 				return $ fixedlocks || repaired
 			, return False

(Diff truncated)
analysis
diff --git a/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_1_451498fab01ae745eb33d42da8682ad3._comment b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_1_451498fab01ae745eb33d42da8682ad3._comment
new file mode 100644
index 000000000..b88e5f7d8
--- /dev/null
+++ b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible/comment_1_451498fab01ae745eb33d42da8682ad3._comment
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-06-04T16:44:39Z"
+ content="""
+I think checkuuid=false has always had this problem; it's never been
+actually usable.
+
+Problem is that checkuuid=false prevents reading the git config, so
+the Repo is LocalUnknown, and such repositories are assumed to not be
+currently available to use.
+
+To get from LocalUnknown to Local, it would have to read the repo's 
+config to determine if the repo is bare or not. But reading the repo's
+config every time is exactly what checkuuid=false is intended to prevent.
+
+Only path I can see is to make the DeferredUUIDCheck that's done
+with checkuuid=false return the Local Repo. That could be done in
+Remote.Git. But, other modules use Remote.repo to look at the Repo too,
+including Command.Sync, and Assistant.TransferSlots. Remote.repo can't
+be updated after it's constructed without some mess of updating the remote
+list, which does not seem like a good idea. Seems that Remote.repo would
+need to be converted to an IO action. 
+
+(So would Remote.gitconfig, which incidentally doesn't contain the config
+of the remote when checkuuid=false, which could also be considered a bug
+with checkuuid=false.)
+"""]]

avoid unncessary version output differences in different contexts
Show operating system and repository version list when run outside
a git repo too.
Also made it only display the local repository version when in a git-annex
repo. Before it showed "unknown" when run in a git repo that was not
git-annex initialized. That seemed like confusing behavior.
This commit was sponsored by Jochen Bartl on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index 4d6b2b8d5..2462900d5 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,6 +1,8 @@
 git-annex (6.20180530) UNRELEASED; urgency=medium
 
   * Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change. 
+  * version: Show operating system and repository version list 
+    when run outside a git repo too.
 
  -- Joey Hess <id@joeyh.name>  Wed, 30 May 2018 11:49:08 -0400
 
diff --git a/Command/Version.hs b/Command/Version.hs
index ef3ef39d7..a7a5d503e 100644
--- a/Command/Version.hs
+++ b/Command/Version.hs
@@ -48,16 +48,9 @@ seekNoRepo o
 
 showVersion :: Annex ()
 showVersion = do
-	v <- getVersion
-	liftIO $ do
-		showPackageVersion
-		vinfo "local repository version" $ fromMaybe "unknown" v
-		vinfo "supported repository versions" $
-			unwords supportedVersions
-		vinfo "upgrade supported from repository versions" $
-			unwords upgradableVersions
-		vinfo "operating system" $
-			unwords [os, arch]
+	liftIO showPackageVersion
+	maybe noop (liftIO . vinfo "local repository version")
+		=<< getVersion
 
 showPackageVersion :: IO ()
 showPackageVersion = do
@@ -67,6 +60,11 @@ showPackageVersion = do
 	vinfo "key/value backends" $ unwords $
 		map (formatKeyVariety . B.backendVariety) Backend.list
 	vinfo "remote types" $ unwords $ map R.typename Remote.remoteTypes
+	vinfo "operating system" $ unwords [os, arch]
+	vinfo "supported repository versions" $
+		unwords supportedVersions
+	vinfo "upgrade supported from repository versions" $
+		unwords upgradableVersions
 
 showRawVersion :: IO ()
 showRawVersion = do
diff --git a/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
index c3300690e..b4d84b6e5 100644
--- a/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
+++ b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
@@ -45,3 +45,5 @@ It might make sense to also always output "supported repository versions" and "u
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 I am a long time happy user of git-annex. Thank you for your never ending effort!
+
+> [[done]] --[[Joey]]
diff --git a/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not/comment_1_17ab67ebfd21e0844973461bad1d40f9._comment b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not/comment_1_17ab67ebfd21e0844973461bad1d40f9._comment
new file mode 100644
index 000000000..e618fbaf4
--- /dev/null
+++ b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not/comment_1_17ab67ebfd21e0844973461bad1d40f9._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-06-04T16:16:39Z"
+ content="""
+Good catch.
+"""]]

followup
diff --git a/Limit.hs b/Limit.hs
index 483336eb9..5d00e2e68 100644
--- a/Limit.hs
+++ b/Limit.hs
@@ -163,6 +163,8 @@ addCopies = addLimit . limitCopies
 
 limitCopies :: MkLimit Annex
 limitCopies want = case splitc ':' want of
+	-- Note that in case of a group having the same name as a trust
+	-- level, it's parsed as a trust level, not as a group.
 	[v, n] -> case parsetrustspec v of
 		Just checker -> go n $ checktrust checker
 		Nothing -> go n $ checkgroup v
diff --git a/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed./comment_1_b4ba41b6e0e611d90ddda92bbe991266._comment b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed./comment_1_b4ba41b6e0e611d90ddda92bbe991266._comment
new file mode 100644
index 000000000..4bd924792
--- /dev/null
+++ b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed./comment_1_b4ba41b6e0e611d90ddda92bbe991266._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-06-04T16:06:57Z"
+ content="""
+Thanks for pointing out this ambiguity, which I don't remember having
+ever considered.
+
+The code for this does look for the name of a trust level first,
+and only a group name if it's not a trust level. So, adding a group
+with a colliding name will never change the behavior of such a command
+or preferred content expression.
+
+The only problem then would be that you couldn't match
+on a group by that name. But if you run into that problem,
+you can simply rename your group.
+
+So I don't think that git-annex needs a sanity check, really.
+
+(I have added a comment in the code to make clear that the order of
+parsing this does matter.)
+"""]]

update
diff --git a/doc/thanks/list b/doc/thanks/list
index 2032312ae..ff6056dad 100644
--- a/doc/thanks/list
+++ b/doc/thanks/list
@@ -97,3 +97,7 @@ John Pellman,
 Duncan Holm, 
 Eric Drechsel, 
 Eric Prestemon, 
+Diederik de Haas, 
+Nick Piper, 
+Ryan Newton, 
+Brett Eisenberg, 

Added a comment: Consequences of this error message
diff --git a/doc/forum/git-annex_add_out_of_memory_error/comment_12_d7d3785b65b0608424b25056c9e3a139._comment b/doc/forum/git-annex_add_out_of_memory_error/comment_12_d7d3785b65b0608424b25056c9e3a139._comment
new file mode 100644
index 000000000..0f5299be7
--- /dev/null
+++ b/doc/forum/git-annex_add_out_of_memory_error/comment_12_d7d3785b65b0608424b25056c9e3a139._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="davicastro"
+ avatar="http://cdn.libravatar.org/avatar/4e708663cf4d5b9e8cfea74caf4307fc"
+ subject="Consequences of this error message"
+ date="2018-06-04T14:16:23Z"
+ content="""
+Hi, can someone comment what is the consequences when the \"fatal: Out of memory, getdelim failed\" occurs? It happened to me in some files when adding a bunch but I don't see any side effects. Git fsck and Git annex fsck are OK. git annex unused makes sense. Could this error cause a file to not be added to the repository? (I'm not using NFS file system, git annex version: 6.20180409-gfb0780266, git version: 2.14.1. It might be due to actual out of memory, I'm not sure, I'm just concerned about the side effects).
+"""]]

Added a comment: the remote im working on
diff --git a/doc/todo/borg_special_remote/comment_1_f8543e05bea3a88769d9c54fdfce3723._comment b/doc/todo/borg_special_remote/comment_1_f8543e05bea3a88769d9c54fdfce3723._comment
new file mode 100644
index 000000000..0a8caf6d9
--- /dev/null
+++ b/doc/todo/borg_special_remote/comment_1_f8543e05bea3a88769d9c54fdfce3723._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="RonnyPfannschmidt"
+ avatar="http://cdn.libravatar.org/avatar/c5379a3fe2188b7571858c49f9db63c6"
+ subject="the remote im working on"
+ date="2018-06-04T07:51:57Z"
+ content="""
+Hi Joey,
+
+i am currently working on a remote to use borg as a tree import source and a content souce
+
+the work is started in https://github.com/RonnyPfannschmidt/git-annex-borg
+
+note that borg does **not** do delta storage - it does content informed dynamic chunk sizes (which helps deduplication)
+
+freestanding borg will not be a good remote for putting things out,
+so i will be pulling things out mostly (but i hope to hit a point where its viable to generate a borg archive from the tree of expected contents thats viable for putting things in)
+
+-- Ronny
+
+"""]]

diff --git a/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed..mdwn b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed..mdwn
new file mode 100644
index 000000000..726226d66
--- /dev/null
+++ b/doc/bugs/Missing_sanity_check_of_group_names._Special_group_names_should_not_be_allowed..mdwn
@@ -0,0 +1,35 @@
+### Please describe the problem.
+
+As documented in [[git-annex-matching-options]], `--copies` accepts `trustlevel` or `groupname` in the following format:
+
+* `--copies=trustlevel:number`
+* `--copies=groupname:number`
+
+This is ambiguous in the unlikely case where a user might come up with the idea to create a group called `trusted`, `semitrusted` or `untrusted` (`remote.<name>.annex-trustlevel`). It should thereby not be allowed to use such special groups.
+
+### What steps will reproduce the problem?
+
+
+[[!format sh """
+## In an annex repo:
+
+git annex group here trusted
+
+## What does this command do now?
+## For reference, it interprets it as `--copies=trustlevel:number`
+git annex find --copies=trusted:1
+
+"""]]
+
+### What version of git-annex are you using? On what operating system?
+
+	git-annex version: 6.20180509
+	build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+	dependency versions: aws-0.14.1 bloomfilter-2.0.1.0 cryptonite-0.20 DAV-1.3.1 feed-0.3.11.1 ghc-8.0.1 http-client-0.4.31.1 persistent-sqlite-2.6 torrent-10000.0.0 uuid-1.3.12 yesod-1.4.3
+	key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+	remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
+	operating system: linux x86_64
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes! Thanks so much for your work!

Fix typo in bug report
diff --git a/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
index 8a5e380d3..c3300690e 100644
--- a/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
+++ b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
@@ -1,6 +1,6 @@
 ### Please describe the problem.
 
-When looking over the [[bug reports|bugs]] here some of them are missing the "Operating system" info in the `git annex version` output. The reason for this is that the `git annex version` output depends on whether it was run inside a annex repo or not.
+When looking over the [[bug reports|bugs]] here some of them are missing the "Operating system" info in the `git annex version` output. The reason for this is that the `git annex version` output depends on whether it was run inside an annex repo or not.
 
 ### What steps will reproduce the problem?
 

Create bug report: Operating system should be contained in `git annex version` output
diff --git a/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
new file mode 100644
index 000000000..8a5e380d3
--- /dev/null
+++ b/doc/bugs/Operating_system_should_be_contained_in___96__git_annex_version__96___output_regardless_if_in_annex_repo_or_not.mdwn
@@ -0,0 +1,47 @@
+### Please describe the problem.
+
+When looking over the [[bug reports|bugs]] here some of them are missing the "Operating system" info in the `git annex version` output. The reason for this is that the `git annex version` output depends on whether it was run inside a annex repo or not.
+
+### What steps will reproduce the problem?
+
+Run `git annex version` outside of an annex repo:
+
+    git-annex version: 6.20180509
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.14.1 bloomfilter-2.0.1.0 cryptonite-0.20 DAV-1.3.1 feed-0.3.11.1 ghc-8.0.1 http-client-0.4.31.1 persistent-sqlite-2.6 torrent-10000.0.0 uuid-1.3.12 yesod-1.4.3
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
+
+
+Run `git annex version` inside an annex repo:
+
+    git-annex version: 6.20180509
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.14.1 bloomfilter-2.0.1.0 cryptonite-0.20 DAV-1.3.1 feed-0.3.11.1 ghc-8.0.1 http-client-0.4.31.1 persistent-sqlite-2.6 torrent-10000.0.0 uuid-1.3.12 yesod-1.4.3
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
+    local repository version: 5
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+
+
+### What version of git-annex are you using? On what operating system?
+
+    git-annex version: 6.20180509
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.14.1 bloomfilter-2.0.1.0 cryptonite-0.20 DAV-1.3.1 feed-0.3.11.1 ghc-8.0.1 http-client-0.4.31.1 persistent-sqlite-2.6 torrent-10000.0.0 uuid-1.3.12 yesod-1.4.3
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+
+
+### Please provide any additional information below.
+
+It might make sense to also always output "supported repository versions" and "upgrade supported from repository versions" because they also do not depend on the current annex repo.
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+I am a long time happy user of git-annex. Thank you for your never ending effort!

Fix spelling in doc/design/iabackup.mdwn
diff --git a/doc/design/iabackup.mdwn b/doc/design/iabackup.mdwn
index 8133c2332..0f52ae3b7 100644
--- a/doc/design/iabackup.mdwn
+++ b/doc/design/iabackup.mdwn
@@ -61,7 +61,7 @@ the filenames and urls that belong in it.
 
 A script can do this using the `git annex fromkey` and `git annex
 registerurl` commands. Time to make such a repository with 100k files
-is in the 10 minute range (faster on SSD or randisk).
+is in the 10 minute range (faster on SSD or ramdisk).
 
 ## adding a client
 

diff --git a/doc/bugs/annex-checkuuid_renders_remotes_inaccessible.mdwn b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible.mdwn
new file mode 100644
index 000000000..6e634ac19
--- /dev/null
+++ b/doc/bugs/annex-checkuuid_renders_remotes_inaccessible.mdwn
@@ -0,0 +1,40 @@
+### Please describe the problem.
+
+Setting `remote.<name>.annex-checkuuid` to `false` renders remote `<name>` inaccessible for git-annex, i.e., when using commands such as `get`.
+
+### What steps will reproduce the problem?
+
+    $ cat git-annex-checkuuid.sh
+    pushd $(mktemp -d)
+
+    git init origin
+    pushd origin
+    git annex init origin
+    touch blob
+    git annex add blob
+    git annex sync
+
+    popd
+    git clone origin clone
+    pushd clone
+    git annex init clone
+    git config remote.origin.annex-checkuuid false
+    git annex get blob
+
+    $ bash git-annex-checkuuid.sh
+    …
+    get blob
+      Unable to access these remotes: origin
+
+      Try making some of these repositories available:
+        d27a9c2d-af76-4b01-8335-bc2d72a1d28a -- [origin]
+    failed
+    git-annex: get: 1 failed
+
+### What version of git-annex are you using? On what operating system?
+
+    git-annex version: 6.20180529-g33834140e                                                                                                                                                      
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.2 feed-1.0.0.0 ghc-8.4.2 http-client-0.5.12.1 persistent-sqlite-2.8.1.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external

Add Guix to the installation instructions, as there is now package available!
diff --git a/doc/install.mdwn b/doc/install.mdwn
index 6255ee4b3..f9f9c9afb 100644
--- a/doc/install.mdwn
+++ b/doc/install.mdwn
@@ -13,6 +13,7 @@ detailed instructions             | quick install
 &nbsp;&nbsp;[[ArchLinux]]         | `pacman -S git-annex`
 &nbsp;&nbsp;[[NixOS]]             | `nix-env -i git-annex`
 &nbsp;&nbsp;[[Gentoo]]            | `emerge git-annex`
+&nbsp;&nbsp;[[Guix]]              | `guix package -i git-annex`
 &nbsp;&nbsp;[[Void]]              | `xbps-install git-annex`
 &nbsp;&nbsp;[[ScientificLinux5]]  |
 &nbsp;&nbsp;[[openSUSE]]          | `zypper in git-annex`

Add install page for Guix
diff --git a/doc/install/Guix.mdwn b/doc/install/Guix.mdwn
new file mode 100644
index 000000000..fe7f404f0
--- /dev/null
+++ b/doc/install/Guix.mdwn
@@ -0,0 +1,7 @@
+git-annex can be installed with the [GNU Guix](https://www.gnu.org/software/guix) package manager.
+
+    guix package -i git-annex
+
+Currently (as of June 2018), some features are not enabled in the package.
+
+    -Android -Assistant -Pairing -S3 -Webapp -WebDAV

Added a comment
diff --git a/doc/forum/done_bugs_are_missing/comment_3_6eb74e1bcf6b062b61b25e59b0b1ddc6._comment b/doc/forum/done_bugs_are_missing/comment_3_6eb74e1bcf6b062b61b25e59b0b1ddc6._comment
new file mode 100644
index 000000000..2558227bb
--- /dev/null
+++ b/doc/forum/done_bugs_are_missing/comment_3_6eb74e1bcf6b062b61b25e59b0b1ddc6._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="comment 3"
+ date="2018-05-31T17:43:21Z"
+ content="""
+Thanks!
+"""]]

Added a comment
diff --git a/doc/forum/git_status_typechange_in_v5_direct_false/comment_1_eb4e2b1fd7615e2b6936b5df01d68d1f._comment b/doc/forum/git_status_typechange_in_v5_direct_false/comment_1_eb4e2b1fd7615e2b6936b5df01d68d1f._comment
new file mode 100644
index 000000000..2e71853af
--- /dev/null
+++ b/doc/forum/git_status_typechange_in_v5_direct_false/comment_1_eb4e2b1fd7615e2b6936b5df01d68d1f._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="comment 1"
+ date="2018-05-31T17:39:10Z"
+ content="""
+This all seems very OK and expected to me.
+
+When you do `git-annex unlock photographers/jboxman/2004/06/16/20040616-004528.jpg` you are changing out the symlink for the actual file, so git has recognized the thing with this name is different. If you do `git-annex add photographers/jboxman/2004/06/16/20040616-004528.jpg` git-annex will recognize that you have not modified any content and will replace the file with the symlink, the file will no longer appear in git status and the file won't have any commits associated with it.
+
+So if you unlock everything, launch digiKam and generate your sidecars, then we see both jpgs and xmps in `git status`. If you then run `git annex add .` all your jpgs will turn back into symlinks and all your xmp files will appear in `git status` per your `largefiles` rule staged for committing.
+
+I haven't tested with your exact setup so I would recommend testing the workflow on a small folder to start. 
+
+
+"""]]

response
diff --git a/doc/forum/done_bugs_are_missing/comment_2_1fdcfb4ed807e424fe73f134c4ad9526._comment b/doc/forum/done_bugs_are_missing/comment_2_1fdcfb4ed807e424fe73f134c4ad9526._comment
new file mode 100644
index 000000000..880b012f7
--- /dev/null
+++ b/doc/forum/done_bugs_are_missing/comment_2_1fdcfb4ed807e424fe73f134c4ad9526._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2018-05-31T15:50:05Z"
+ content="""
+The page was simply only listing the most recently closed 10
+bugs, and not all the others. I've changed it to list them all.
+
+(I do archive old closed bugs, generally about a year after the last
+activity on the bug, to keep the size of the wiki in check.)
+"""]]

list all (non-archived) done bugs, not only most recent 10
diff --git a/doc/bugs/done.mdwn b/doc/bugs/done.mdwn
index 1575c8d89..62b5fc9e9 100644
--- a/doc/bugs/done.mdwn
+++ b/doc/bugs/done.mdwn
@@ -1,4 +1,4 @@
 recently fixed [[bugs]]
 
-[[!inline pages="./* and link(./done) and !*/Discussion" sort=mtime show=10
+[[!inline pages="./* and link(./done) and !*/Discussion" sort=mtime show=0
 archive=yes feedlimit=10]]
diff --git a/doc/todo/done.mdwn b/doc/todo/done.mdwn
index bb3784ed4..2ab037cfb 100644
--- a/doc/todo/done.mdwn
+++ b/doc/todo/done.mdwn
@@ -1,4 +1,4 @@
 recently fixed [[todo]] items.
 
-[[!inline pages="./* and link(./done) and !*/Discussion" sort=mtime show=10
+[[!inline pages="./* and link(./done) and !*/Discussion" sort=mtime show=0
 archive=yes feedlimit=10]]

Added a comment: bare remotes, workflow
diff --git a/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_6_69781b2f00afe6150bc5997335b8af90._comment b/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_6_69781b2f00afe6150bc5997335b8af90._comment
new file mode 100644
index 000000000..88d00cd33
--- /dev/null
+++ b/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_6_69781b2f00afe6150bc5997335b8af90._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="bare remotes, workflow"
+ date="2018-05-31T12:50:55Z"
+ content="""
+Aaah.
+
+So although git-annex does support storing files in bare repositories it treats them in much the same way as git in general. Namely, they are meant to be added as remotes from other full repositories. Assistant is responsible for monitoring a working directory for changes and automatically committing and syncing those changes. Bare repositories don't actually need assistant monitoring them because they don't have a working directory where users are adding and removing files from.
+
+In your case, you want assistant monitoring `~/Desktop/annex` which it now is. If you then add `/Volumes/SeagateBlack/annex` as a remote to `~/Desktop/annex` then assistant, by watching just `~/Desktop/annex` will sync with all remotes whenever you add remove files in `~/Desktop/annex`.  I imagine you already have your USB drive correctly setup as a remote. Just run `git annex info` from your `~/Desktop/annex` directory to see what remotes it knows about.
+
+Once USB is a remote of `~/Desktop/annex`, then if you launch the webapp from your `~/Desktop/annex` directory,`git-annex webapp` you should actually see both `~/Desktop/annex` and `/Volumes/SeagateBlack/annex` listed. The webapp automatically lists all remotes your repo knows about.
+
+In terms of workflow, I am not actually sure that using a bare repo is what you want on your USB drive. I always setup normal full git repositories on USB drives (not bare repos). That way I can use Finder to actually browse out to the files on the USB drive, add files when I am on the go, copy files etc… If you use a bare repo, then the files won't actually be listed in a usable form. But, if you don't actually want to ever look at the files on your USB drive (from Finder) then using a bare remote on a USB drive has the advantage that you don't need assistant monitoring it, and is probably marginally faster in some extreme cases since there is no working tree to update.
+
+"""]]

Added a comment: Fixed a first problem and then ran into bigger problems
diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_db4b772371ee49c743605c728e23c91a._comment b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_db4b772371ee49c743605c728e23c91a._comment
new file mode 100644
index 000000000..36fccdd80
--- /dev/null
+++ b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_db4b772371ee49c743605c728e23c91a._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="marek@33e8ba4fbc201af14a2badcc0656024401f5c916"
+ nickname="marek"
+ avatar="http://cdn.libravatar.org/avatar/65a60e8f5183feeeef8cef815bf73e61"
+ subject="Fixed a first problem and then ran into bigger problems"
+ date="2018-05-31T09:29:30Z"
+ content="""
+By using 
+    git show git-annex:remote.log
+
+one can extract the cipher from the old remote and set it with
+
+    git annex enableremote $newremote cipher=$CIPHER
+
+However, then I ran into a new problem. The git-annex-remote-hubic special remote created files with leading slashes (/GPGHMACSHA1--00021a41c3143f1e52f16af93625d1072e861543).
+Rclone, and I don`t even mean git-annex-remote-rclone, cannot access these files. It interprets the slash as directory separator which leads to those files not being found.
+
+I guess I will just reupload to fix the slash error in the first place. Or find a Hubic client that can deal with slashes and rename the objects remotely.
+"""]]

removed
diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_655a911c6729e71bc1338a0434dc6216._comment b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_655a911c6729e71bc1338a0434dc6216._comment
deleted file mode 100644
index f63f23364..000000000
--- a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_655a911c6729e71bc1338a0434dc6216._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="marek@33e8ba4fbc201af14a2badcc0656024401f5c916"
- nickname="marek"
- avatar="http://cdn.libravatar.org/avatar/65a60e8f5183feeeef8cef815bf73e61"
- subject="Quick fix"
- date="2018-05-31T09:22:57Z"
- content="""
-So I managed to change the intermediate cipher by looking it up with
-
-"""]]

Added a comment: Quick fix
diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_655a911c6729e71bc1338a0434dc6216._comment b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_655a911c6729e71bc1338a0434dc6216._comment
new file mode 100644
index 000000000..f63f23364
--- /dev/null
+++ b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_2_655a911c6729e71bc1338a0434dc6216._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="marek@33e8ba4fbc201af14a2badcc0656024401f5c916"
+ nickname="marek"
+ avatar="http://cdn.libravatar.org/avatar/65a60e8f5183feeeef8cef815bf73e61"
+ subject="Quick fix"
+ date="2018-05-31T09:22:57Z"
+ content="""
+So I managed to change the intermediate cipher by looking it up with
+
+"""]]

diff --git a/doc/forum/git_status_typechange_in_v5_direct_false.mdwn b/doc/forum/git_status_typechange_in_v5_direct_false.mdwn
index 66bb79b3e..f326850d3 100644
--- a/doc/forum/git_status_typechange_in_v5_direct_false.mdwn
+++ b/doc/forum/git_status_typechange_in_v5_direct_false.mdwn
@@ -6,28 +6,28 @@ Is it possible to suppress typechange in some way?
 
 Thanks.
 
-On branch master
-Changes not staged for commit:
-  (use "git add <file>..." to update what will be committed)
-  (use "git checkout -- <file>..." to discard changes in working directory)
+    On branch master
+    Changes not staged for commit:
+    (use "git add <file>..." to update what will be committed)
+    (use "git checkout -- <file>..." to discard changes in working directory)
 
         typechange: photographers/jboxman/2004/06/16/20040616-004528.jpg
         .......
         typechange: photographers/jboxman/2004/06/17/20040617-010127.jpg
 
-[annex]
-        uuid = 56e0d203-7898-xxxx-xxxx-1b04cec6597c
-        version = 5
-        largefiles = largerthan=20kb and not (include=*.xmp or include=*.thor or include=bin/*)
-        direct = false
-
-git-annex version: 6.20171124
-build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents TorrentParser MagicMime Feeds Quvi
-dependency versions: aws-0.17.1 bloomfilter-2.0.1.0 cryptonite-0.24 DAV-1.3.1 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.7.1 persistent-sqlite-2.6.3.1torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
-key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384ESHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
-remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
-local repository version: 5
-supported repository versions: 3 5 6
-upgrade supported from repository versions: 0 1 2 3 4 5
-operating system: darwin x86_64
+    [annex]
+    uuid = 56e0d203-7898-xxxx-xxxx-1b04cec6597c
+    version = 5
+    largefiles = largerthan=20kb and not (include=*.xmp or include=*.thor or include=bin/*)
+    direct = false
+
+    git-annex version: 6.20171124
+    build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents TorrentParser MagicMime Feeds Quvi
+    dependency versions: aws-0.17.1 bloomfilter-2.0.1.0 cryptonite-0.24 DAV-1.3.1 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.7.1 persistent-sqlite-2.6.3.1torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384ESHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+    local repository version: 5
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: darwin x86_64
 

diff --git a/doc/forum/git_status_typechange_in_v5_direct_false.mdwn b/doc/forum/git_status_typechange_in_v5_direct_false.mdwn
new file mode 100644
index 000000000..66bb79b3e
--- /dev/null
+++ b/doc/forum/git_status_typechange_in_v5_direct_false.mdwn
@@ -0,0 +1,33 @@
+Should I see a typechange from `git status` when using direct mode = false?
+
+I'm using digiKam to manage photos in a git-annex repository. digiKam will only write XMP sidecar files for real files, not symlinks, so I need to unlock every file in order to sync metadata to XMP files.
+
+Is it possible to suppress typechange in some way?
+
+Thanks.
+
+On branch master
+Changes not staged for commit:
+  (use "git add <file>..." to update what will be committed)
+  (use "git checkout -- <file>..." to discard changes in working directory)
+
+        typechange: photographers/jboxman/2004/06/16/20040616-004528.jpg
+        .......
+        typechange: photographers/jboxman/2004/06/17/20040617-010127.jpg
+
+[annex]
+        uuid = 56e0d203-7898-xxxx-xxxx-1b04cec6597c
+        version = 5
+        largefiles = largerthan=20kb and not (include=*.xmp or include=*.thor or include=bin/*)
+        direct = false
+
+git-annex version: 6.20171124
+build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents TorrentParser MagicMime Feeds Quvi
+dependency versions: aws-0.17.1 bloomfilter-2.0.1.0 cryptonite-0.24 DAV-1.3.1 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.7.1 persistent-sqlite-2.6.3.1torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384ESHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+local repository version: 5
+supported repository versions: 3 5 6
+upgrade supported from repository versions: 0 1 2 3 4 5
+operating system: darwin x86_64
+

Added a comment
diff --git a/doc/forum/done_bugs_are_missing/comment_1_400c15824ec84bd67583102a023edcf8._comment b/doc/forum/done_bugs_are_missing/comment_1_400c15824ec84bd67583102a023edcf8._comment
new file mode 100644
index 000000000..5c65bfa60
--- /dev/null
+++ b/doc/forum/done_bugs_are_missing/comment_1_400c15824ec84bd67583102a023edcf8._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="comment 1"
+ date="2018-05-31T03:15:37Z"
+ content="""
+Ooops. Hit post, before I finished.
+
+…but I couldn't find the bug report. I was pleasantly surprised when I did discover you had fixed this, so quickly! But the fix was only mentioned in the release notes. I think it is great to see the entire list of bugs you have fixed on the website, it shows how much effort you put into this project and how actively maintained it is. 
+"""]]

Added a comment
diff --git a/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_5_47aa5de003b409d3d2a4c58a0f08a14a._comment b/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_5_47aa5de003b409d3d2a4c58a0f08a14a._comment
new file mode 100644
index 000000000..722d92dd3
--- /dev/null
+++ b/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_5_47aa5de003b409d3d2a4c58a0f08a14a._comment
@@ -0,0 +1,45 @@
+[[!comment format=mdwn
+ username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
+ nickname="jhnichol"
+ avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
+ subject="comment 5"
+ date="2018-05-30T17:36:30Z"
+ content="""
+I started over, making new repos with the webapp & version: 6.20180527-gc3064edac .
+I still get an error in the webapp when adding the USB repo .
+The USB repo `/Volumes/SeagateBlack/annex` was not in `~/.config/git-annex/autostart` , so I added it .
+I enabled debugging in the webapp, shutdown the webapp and assistants, and used the terminal:
+
+	JamesMac:annex jhnichol$ cd /Volumes/SeagateBlack/annex 
+	JamesMac:annex jhnichol$ git-annex assistant --autostart
+	git-annex: You cannot run this command in a bare repository.
+	JamesMac:annex jhnichol$ cd /Users/jhnichol/Desktop/annex 
+	JamesMac:annex jhnichol$ git-annex assistant --autostart
+	git-annex autostart in /Users/jhnichol/Desktop/annex
+	[2018-05-30 20:13:45.005716] call: /Applications/git-annex.app/Contents/MacOS/bundle/git-annex [\"assistant\",\"--stop\"]
+	[2018-05-30 20:13:45.064878] process done ExitSuccess
+	[2018-05-30 20:13:45.065006] call: /Applications/git-annex.app/Contents/MacOS/bundle/git-annex [\"assistant\",\"--startdelay=5s\"]
+	[2018-05-30 20:13:45.117653] logging to .git/annex/daemon.log
+	[2018-05-30 20:13:45.507796] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"core.bare=false\",\"show-ref\",\"git-annex\"]
+	[2018-05-30 20:13:45.520686] process done ExitSuccess
+	[2018-05-30 20:13:45.520806] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"core.bare=false\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
+	[2018-05-30 20:13:45.526297] process done ExitSuccess
+	[2018-05-30 20:13:45.526857] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"core.bare=false\",\"log\",\"refs/heads/git-annex..2f0d4e1b6fe058a5cc899012ef99870cd2877d2c\",\"--pretty=%H\",\"-n1\"]
+	[2018-05-30 20:13:45.537446] process done ExitSuccess
+	[2018-05-30 20:13:45.537572] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"core.bare=false\",\"log\",\"refs/heads/git-annex..31d29d92a8a48146cdf9dd4517f5a034daf4a467\",\"--pretty=%H\",\"-n1\"]
+	[2018-05-30 20:13:45.543984] process done ExitSuccess
+	[2018-05-30 20:13:45.544626] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"core.bare=false\",\"cat-file\",\"--batch\"]
+	[2018-05-30 20:13:45.546186] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"core.bare=false\",\"cat-file\",\"--batch-check=%(objectname) %(objecttype) %(objectsize)\"]
+	[2018-05-30 20:13:45.550793] read: git [\"config\",\"--null\",\"--list\"]
+	[2018-05-30 20:13:45.556288] process done ExitSuccess
+	[2018-05-30 20:13:45.557233] logging to .git/annex/daemon.log
+	[2018-05-30 20:13:45.565149] process done ExitSuccess
+	ok
+	git-annex autostart in /Volumes/SeagateBlack/annex 
+	failed
+
+so you see I got the \"You cannot run this command in a bare repository\" error from the assistant.
+Also, the assistant for the USB drive just fails with no error details.
+
+Thank you so much for your comments! I feel so close to having working software!
+"""]]

diff --git a/doc/forum/git-annex_webapp_launch_behavior_is_confusing.mdwn b/doc/forum/git-annex_webapp_launch_behavior_is_confusing.mdwn
new file mode 100644
index 000000000..d061913e8
--- /dev/null
+++ b/doc/forum/git-annex_webapp_launch_behavior_is_confusing.mdwn
@@ -0,0 +1,8 @@
+After first installing git-annex if I run `git-annex webapp` on Ubuntu 16.04.4 LTS (annex version 6.20180528-g5300386c2) then the process does not go to the background, but instead stays a foreground process. I can add files to a repo that assistant is watching and it will see them. But, if I then close the web browser and then kill the foreground process that launched the webapp, then all annex processes go away, and git-annex assistant is no longer running anywhere. IE if I then add a file to an assistant repo it won't be seen.
+
+If I reboot, then a bunch of annex processes are automatically started as daemons. Now my repos are watched again. If I launch git-annex webapp, this time the process immediately forks to the background which is different behavior.
+
+The behavior I would expect is that calling `git-annex webapp` should launch, the webapp dameon, load the page in a browser and call something like `git-annex assistant --autostart` as well.
+
+
+—[Andrew](http://git-annex.branchable.com/users/andrew/)

response
diff --git a/doc/git-annex-addurl/comment_2_9bc984fb80d77309b62a4e915e65a31a._comment b/doc/git-annex-addurl/comment_2_9bc984fb80d77309b62a4e915e65a31a._comment
new file mode 100644
index 000000000..a12fc56ca
--- /dev/null
+++ b/doc/git-annex-addurl/comment_2_9bc984fb80d77309b62a4e915e65a31a._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2018-05-30T16:30:26Z"
+ content="""
+There's not currently a way to do per-file youtube-dl options.
+The difficulty is that we don't know what youtube-dl options might be
+unsafe, and which such a feature could make eg `git annex get` use when run
+by a different user.
+
+I feel that this needs some support in youtube-dl to avoid git-annex
+needing to know about all its safe options. Especially since which options
+are available, or safe, could vary between versions of youtube-dl.
+"""]]

Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change
https://prime.haskell.org/wiki/Libraries/Proposals/SemigroupMonoid
I am not happy with the fragile pile of CPP boilerplate required to support
ghc back to 7.0, which git-annex still targets for both the android build
and the standalone build targeting old linux kernels. It makes me unlikely
to want to use Semigroup more in git-annex, because the benefit of the
abstraction is swamped by the ugliness. I actually considered ripping out
all the Semigroup instances, but some are needed to use
optparse-applicative.
The problem, I think, is they made this transaction on too fast a timeline.
(Although ironically, work on it started in 2015 or earlier!)
In particular, Debian oldstable is not out of security support, and it's
not possible to follow the simpler workarounds documented on the wiki and
have it build on oldstable (because the semigroups package in it is too
old).
I have only tested this build with ghc 8.2.2, not the newer and older
versions that branches of the CPP support. So there could be typoes, we'll
see.
This commit was sponsored by Brock Spratlen on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index 5ad219155..4d6b2b8d5 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,9 @@
+git-annex (6.20180530) UNRELEASED; urgency=medium
+
+  * Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change. 
+
+ -- Joey Hess <id@joeyh.name>  Wed, 30 May 2018 11:49:08 -0400
+
 git-annex (6.20180529) upstream; urgency=medium
 
   * Prevent haskell http-client from decompressing gzip files, so downloads
diff --git a/Command/Info.hs b/Command/Info.hs
index c9a314056..9e7e52729 100644
--- a/Command/Info.hs
+++ b/Command/Info.hs
@@ -5,7 +5,7 @@
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
-{-# LANGUAGE BangPatterns, DeriveDataTypeable #-}
+{-# LANGUAGE BangPatterns, DeriveDataTypeable, CPP #-}
 
 module Command.Info where
 
@@ -13,6 +13,10 @@ import "mtl" Control.Monad.State.Strict
 import qualified Data.Map.Strict as M
 import qualified Data.Vector as V
 import Data.Ord
+#if MIN_VERSION_base(4,9,0)
+import qualified Data.Semigroup as Sem
+#endif
+import Prelude
 
 import Command
 import qualified Git
@@ -55,15 +59,28 @@ data KeyData = KeyData
 	, unknownSizeKeys :: Integer
 	, backendsKeys :: M.Map KeyVariety Integer
 	}
+	
+appendKeyData :: KeyData -> KeyData -> KeyData
+appendKeyData a b = KeyData
+	{ countKeys = countKeys a + countKeys b
+	, sizeKeys = sizeKeys a + sizeKeys b
+	, unknownSizeKeys = unknownSizeKeys a + unknownSizeKeys b
+	, backendsKeys = backendsKeys a <> backendsKeys b
+	}
+	
+#if MIN_VERSION_base(4,9,0)
+instance Sem.Semigroup KeyData where
+	(<>) = appendKeyData
+#endif
 
 instance Monoid KeyData where
 	mempty = KeyData 0 0 0 M.empty
-	mappend a b = KeyData
-		{ countKeys = countKeys a + countKeys b
-		, sizeKeys = sizeKeys a + sizeKeys b
-		, unknownSizeKeys = unknownSizeKeys a + unknownSizeKeys b
-		, backendsKeys = backendsKeys a <> backendsKeys b
-		}
+#if MIN_VERSION_base(4,11,0)
+#elif MIN_VERSION_base(4,9,0)
+	mappend = (Sem.<>)
+#else
+	mappend = appendKeyData
+#endif
 
 data NumCopiesStats = NumCopiesStats
 	{ numCopiesVarianceMap :: M.Map Variance Integer
diff --git a/Git/Fsck.hs b/Git/Fsck.hs
index a716b56e3..3092ff2c6 100644
--- a/Git/Fsck.hs
+++ b/Git/Fsck.hs
@@ -5,7 +5,7 @@
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
-{-# LANGUAGE BangPatterns #-}
+{-# LANGUAGE BangPatterns, CPP #-}
 
 module Git.Fsck (
 	FsckResults(..),
@@ -26,6 +26,10 @@ import qualified Git.Version
 
 import qualified Data.Set as S
 import Control.Concurrent.Async
+#if MIN_VERSION_base(4,9,0)
+import qualified Data.Semigroup as Sem
+#endif
+import Prelude
 
 data FsckResults 
 	= FsckFoundMissing
@@ -44,15 +48,29 @@ type MissingObjects = S.Set Sha
 
 type Truncated = Bool
 
+appendFsckOutput :: FsckOutput -> FsckOutput -> FsckOutput
+appendFsckOutput (FsckOutput s1 t1) (FsckOutput s2 t2) =
+	FsckOutput (S.union s1 s2) (t1 || t2)
+appendFsckOutput (FsckOutput s t) _ = FsckOutput s t
+appendFsckOutput _ (FsckOutput s t) = FsckOutput s t
+appendFsckOutput NoFsckOutput NoFsckOutput = NoFsckOutput
+appendFsckOutput AllDuplicateEntriesWarning AllDuplicateEntriesWarning = AllDuplicateEntriesWarning
+appendFsckOutput AllDuplicateEntriesWarning NoFsckOutput = AllDuplicateEntriesWarning
+appendFsckOutput NoFsckOutput AllDuplicateEntriesWarning = AllDuplicateEntriesWarning
+
+#if MIN_VERSION_base(4,9,0)
+instance Sem.Semigroup FsckOutput where
+	(<>) = appendFsckOutput
+#endif
+
 instance Monoid FsckOutput where
 	mempty = NoFsckOutput
-	mappend (FsckOutput s1 t1) (FsckOutput s2 t2) = FsckOutput (S.union s1 s2) (t1 || t2)
-	mappend (FsckOutput s t) _ = FsckOutput s t
-	mappend _ (FsckOutput s t) = FsckOutput s t
-	mappend NoFsckOutput NoFsckOutput = NoFsckOutput
-	mappend AllDuplicateEntriesWarning AllDuplicateEntriesWarning = AllDuplicateEntriesWarning
-	mappend AllDuplicateEntriesWarning NoFsckOutput = AllDuplicateEntriesWarning
-	mappend NoFsckOutput AllDuplicateEntriesWarning = AllDuplicateEntriesWarning
+#if MIN_VERSION_base(4,11,0)
+#elif MIN_VERSION_base(4,9,0)
+	mappend = (Sem.<>)
+#else
+	mappend = appendFsckOutput
+#endif
 
 {- Runs fsck to find some of the broken objects in the repository.
  - May not find all broken objects, if fsck fails on bad data in some of
diff --git a/Types/DesktopNotify.hs b/Types/DesktopNotify.hs
index e6df05ab1..ce7e4c4a3 100644
--- a/Types/DesktopNotify.hs
+++ b/Types/DesktopNotify.hs
@@ -5,9 +5,14 @@
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
+{-# LANGUAGE CPP #-}
+
 module Types.DesktopNotify where
 
 import Data.Monoid
+#if MIN_VERSION_base(4,9,0)
+import qualified Data.Semigroup as Sem
+#endif
 import Prelude
 
 data DesktopNotify = DesktopNotify
@@ -16,10 +21,23 @@ data DesktopNotify = DesktopNotify
 	}
 	deriving (Show)
 
+appendDesktopNotify :: DesktopNotify -> DesktopNotify -> DesktopNotify
+appendDesktopNotify (DesktopNotify s1 f1) (DesktopNotify s2 f2) =
+	DesktopNotify (s1 || s2) (f1 || f2)
+
+#if MIN_VERSION_base(4,9,0)
+instance Sem.Semigroup DesktopNotify where
+	(<>) = appendDesktopNotify
+#endif
+
 instance Monoid DesktopNotify where
 	mempty = DesktopNotify False False
-	mappend (DesktopNotify s1 f1) (DesktopNotify s2 f2) =
-		DesktopNotify (s1 || s2) (f1 || f2)
+#if MIN_VERSION_base(4,11,0)
+#elif MIN_VERSION_base(4,9,0)
+	mappend = (Sem.<>)
+#else
+	mappend = appendDesktopNotify
+#endif
 
 mkNotifyStart :: DesktopNotify
 mkNotifyStart = DesktopNotify True False
diff --git a/Types/Difference.hs b/Types/Difference.hs
index 4abc75c44..0f7100c0c 100644
--- a/Types/Difference.hs
+++ b/Types/Difference.hs
@@ -5,6 +5,8 @@
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
+{-# LANGUAGE CPP #-}
+
 module Types.Difference (
 	Difference(..),
 	Differences(..),
@@ -23,8 +25,11 @@ import qualified Git.Config
 
 import Data.Maybe
 import Data.Monoid
-import Prelude
 import qualified Data.Set as S
+#if MIN_VERSION_base(4,9,0)
+import qualified Data.Semigroup as Sem

(Diff truncated)
Added a comment: Using youtube-dl commandline options with git-annex-addurl
diff --git a/doc/git-annex-addurl/comment_1_ce9c660be160a22c28aeb6de8b3b5818._comment b/doc/git-annex-addurl/comment_1_ce9c660be160a22c28aeb6de8b3b5818._comment
new file mode 100644
index 000000000..cff3bca56
--- /dev/null
+++ b/doc/git-annex-addurl/comment_1_ce9c660be160a22c28aeb6de8b3b5818._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joseph.rawson.works@85a210ab8c0e37a0b2d6bb235738b20e23e8878f"
+ nickname="joseph.rawson.works"
+ avatar="http://cdn.libravatar.org/avatar/6b473d5484b68803e8c47eeff9197397"
+ subject="Using youtube-dl commandline options with git-annex-addurl"
+ date="2018-05-30T15:29:16Z"
+ content="""
+I have been trying to figure out how to use addurl to get this video.
+I have this in my mscourtstuff annex as a large binary, but I would really like to 
+use the web as a remote for this.
+
+Hughes v Hosemann 2010-CA-01949-SCT-43112001.mp4
+youtube-dl --referer 'http://judicial.mc.edu/case.php?id=24206'  http://player.vimeo.com/video/43112001
+
+
+"""]]

response
diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment
new file mode 100644
index 000000000..e3a63b3f9
--- /dev/null
+++ b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-05-30T15:21:57Z"
+ content="""
+Yes, there is an intermediate encryption key.
+
+What may work is to use `git annex enableremote` on the original remote
+with `externaltype=rclone`, which will make it use the other program.
+
+Whether this will work depends on implementation details of both special
+remotes. If they use the same names/urls/whatever to access the data stored
+in them, it should work.
+"""]]

Revert "response"
This reverts commit 96373a8555d55f728bfa074d0e2a2e2ae975898c.
Contained unrelated change.
diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment
deleted file mode 100644
index e3a63b3f9..000000000
--- a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment
+++ /dev/null
@@ -1,14 +0,0 @@
-[[!comment format=mdwn
- username="joey"
- subject="""comment 1"""
- date="2018-05-30T15:21:57Z"
- content="""
-Yes, there is an intermediate encryption key.
-
-What may work is to use `git annex enableremote` on the original remote
-with `externaltype=rclone`, which will make it use the other program.
-
-Whether this will work depends on implementation details of both special
-remotes. If they use the same names/urls/whatever to access the data stored
-in them, it should work.
-"""]]
diff --git a/stack.yaml b/stack.yaml
index f7bffe8b4..64e1081f9 100644
--- a/stack.yaml
+++ b/stack.yaml
@@ -16,16 +16,9 @@ flags:
 packages:
 - '.'
 extra-deps:
-- IfElse-0.85
-- aws-0.20
+- aws-0.17.1
 - bloomfilter-2.0.1.0
-- esqueleto-2.5.3
-- sandi-0.4.2
 - torrent-10000.1.1
-- persistent-2.7.3.1
-- conduit-1.3.0.2
-- resourcet-1.2.1
-allow-newer: true
 explicit-setup-deps:
   git-annex: true
-resolver: lts-11.11
+resolver: lts-9.9

response
diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment
new file mode 100644
index 000000000..e3a63b3f9
--- /dev/null
+++ b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another/comment_1_6e8e3350ae26b6c59daa56b7e86cc46a._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-05-30T15:21:57Z"
+ content="""
+Yes, there is an intermediate encryption key.
+
+What may work is to use `git annex enableremote` on the original remote
+with `externaltype=rclone`, which will make it use the other program.
+
+Whether this will work depends on implementation details of both special
+remotes. If they use the same names/urls/whatever to access the data stored
+in them, it should work.
+"""]]
diff --git a/stack.yaml b/stack.yaml
index 64e1081f9..f7bffe8b4 100644
--- a/stack.yaml
+++ b/stack.yaml
@@ -16,9 +16,16 @@ flags:
 packages:
 - '.'
 extra-deps:
-- aws-0.17.1
+- IfElse-0.85
+- aws-0.20
 - bloomfilter-2.0.1.0
+- esqueleto-2.5.3
+- sandi-0.4.2
 - torrent-10000.1.1
+- persistent-2.7.3.1
+- conduit-1.3.0.2
+- resourcet-1.2.1
+allow-newer: true
 explicit-setup-deps:
   git-annex: true
-resolver: lts-9.9
+resolver: lts-11.11

Added a comment
diff --git a/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_4_704ef9c0ed78a00030f7d4d151ecbe5b._comment b/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_4_704ef9c0ed78a00030f7d4d151ecbe5b._comment
new file mode 100644
index 000000000..5e4ff1019
--- /dev/null
+++ b/doc/bugs/OSX_Assistant_will_not_automatically_drop/comment_4_704ef9c0ed78a00030f7d4d151ecbe5b._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="comment 4"
+ date="2018-05-30T15:24:47Z"
+ content="""
+So [bare repositories](https://git-annex.branchable.com/bare_repositories/) should work fine with Assistant.
+
+You said `it is bare and will not allow me to start an assistant on it`. Do you get an error message from the webapp?
+
+What happens if you manually add the path to your bare USB drive repo to the `~/.config/git-annex/autostart` file, then run `git-annex assistant --autostart`? Then is assistant running on it?
+"""]]

diff --git a/doc/forum/done_bugs_are_missing.mdwn b/doc/forum/done_bugs_are_missing.mdwn
new file mode 100644
index 000000000..1121f431d
--- /dev/null
+++ b/doc/forum/done_bugs_are_missing.mdwn
@@ -0,0 +1,5 @@
+Joey, did you mean to delete all of the [done bugs](http://git-annex.branchable.com/bugs/done/) during your [mass bug closures](http://git-annex.branchable.com/devblog/day_498__unexpected_release_prep/)?
+
+I usually like to check done bugs on the website to see what has been fixed in a release. But, it seems many of the fixed bugs are now missing. I had been looking to see if you fixed the webapp not launching on OS-X, but I couldn't find the bug report.
+
+—[Andrew](http://git-annex.branchable.com/users/andrew/)

Added a comment
diff --git a/doc/forum/Dealing_with_crippled_Android_file_system/comment_5_a210b77d8567643667bbe92b40b5bef0._comment b/doc/forum/Dealing_with_crippled_Android_file_system/comment_5_a210b77d8567643667bbe92b40b5bef0._comment
new file mode 100644
index 000000000..33c86afb5
--- /dev/null
+++ b/doc/forum/Dealing_with_crippled_Android_file_system/comment_5_a210b77d8567643667bbe92b40b5bef0._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="comment 5"
+ date="2018-05-30T14:43:37Z"
+ content="""
+Aaaah. OK, that makes sense.
+
+So, when you call [termux-setup-storage](https://wiki.termux.com/wiki/Internal_and_external_storage) in Termux it creates a bunch of symlinks to the standard Android directories, calling [setupStorageSymlinks](https://github.com/termux/termux-app/blob/master/app/src/main/java/com/termux/app/TermuxInstaller.java). But yeah, the new folders sitting in $HOME are just symlinks to Android's FAT32 view of these folders. I was thinking maybe they were something else…
+
+So after reading [Diving into SDCardFS: How Google’s FUSE Replacement Will Reduce I/O Overhead](https://www.xda-developers.com/diving-into-sdcardfs-how-googles-fuse-replacement-will-reduce-io-overhead/) it appears that FAT32 is very much the baked-in standard. The article also has some nice information on the naming conventions in Android of various storage types. Some Google-ing does imply that people have changed the underlying filesystem of the root OS, with rooting, but the solutions seems to now, at least, be [device specific](https://stackoverflow.com/a/29727734/8671834).
+
+I can't find any elegant, filesystem level solution for de-duplication of files that would be visible as media to all Android apps in `external storage`.
+
+Maybe it is possible that with Android's new [Adoptable Storage](https://source.android.com/devices/storage/adoptable) one could create symlinks, but I haven't tried this and I am not optimistic. Adoptable storage is not widely supported, but essentially it fully encrypts your device and re-formats your “external storage” as ext4 or f2fs. But I would imagine that any public facing views of external storage would probably still be FAT32 so their wouldn't be any way to create symlinks.
+
+My other thought would be to create an app specific solution to your issue. For example you are trying to make photos visible to Gallery apps and you have a lot of duplicate photos present in multiple folders. One solution would be to use albums, which I am pretty sure already support having the same photo in multiple albums (at least in Google Photos), while only taking up storage for a single photo. So, you could copy a single copy of each of your photos to `~/storage/pictures`, then using Android APIs [add the photos into multiple albums](https://stackoverflow.com/a/11983767/8671834). You could probably automate this process with scripts or perhaps a Termux addon? Or, perhaps there is some other album supporting Android Gallery app that uses a simpler album format, like a JSON file or something.
+
+"""]]

diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another.mdwn b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another.mdwn
index e12aef5ac..ac06b9990 100644
--- a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another.mdwn
+++ b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another.mdwn
@@ -31,6 +31,7 @@ However, when I fsck for a file that is on hubic or on hubic2 I only get a posit
  
     git-annex-remote-hubic[1] <-- CHECKPRESENT GPGHMACSHA1--64e155e3010f69e909e629c6d9fdacae5c0c3bef
 respectively:
+
     git-annex-remote-rclone[1] <-- CHECKPRESENT GPGHMACSHA1--5cf9edd1a413c561775f8920fc7b21410be6c5a4
 
 There is one thing I noticed though:

diff --git a/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another.mdwn b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another.mdwn
new file mode 100644
index 000000000..e12aef5ac
--- /dev/null
+++ b/doc/forum/Migrate_from_one_connection_mechanism_for_special_remote_to_another.mdwn
@@ -0,0 +1,43 @@
+Hi there,
+
+I want to migrate from git-annex-remote-hubic to git-annex-remote-rclone to access the same hubic remote, same container, same data. So ideally I just want to switch to rclone but reuse the old content (since uploading to hubic is so sloooooow). 
+
+My old remote is called hubic, my new one hubic2 in my git annex repo.
+Both are reported as similar by git annex info -F hubic/hubic2 :
+
+    remote: hubic
+    description: [hubic]
+    uuid: 396e481c-5fc9-4933-b6a4-b1398fb7c61b
+    trust: trusted
+    cost: 225.0
+    type: external
+    externaltype: hubic
+    encryption: pubkey (to gpg keys: XXX)
+    chunking: none
+
+    remote: hubic2
+    description: [hubic2]
+    uuid: adaea02d-920c-48fa-811d-c27e51bda85f
+    trust: semitrusted
+    cost: 250.0
+    type: external
+    externaltype: rclone
+    encryption: pubkey (to gpg keys: XXX)
+    chunking: none
+
+Yes, the GPG key is the same. I only have one configured on this machine.
+
+However, when I fsck for a file that is on hubic or on hubic2 I only get a positive result for hubic. The chatlogs of both fsck reveal that the key that fsck is looking for is different for the different remotes.
+ 
+    git-annex-remote-hubic[1] <-- CHECKPRESENT GPGHMACSHA1--64e155e3010f69e909e629c6d9fdacae5c0c3bef
+respectively:
+    git-annex-remote-rclone[1] <-- CHECKPRESENT GPGHMACSHA1--5cf9edd1a413c561775f8920fc7b21410be6c5a4
+
+There is one thing I noticed though:
+Although in the remote-hubic case I have not chunking configured, I see with *rclone lsl* that I have some files in chunks up there.
+However, the file I am using for this test is only 2MB and it is not chunked on the remote.
+
+What am I missing, what am I doing wrong? While writing this I thought of the following: Is there an intermediate key used for the actual encryption or for the encryption of the filenames? If yes how can I access it or make it available to the new remote?
+
+Thanks,
+Marek

diff --git a/doc/bugs/GHC_8.4.3_build_failure_in_Types__47__DesktopNotify.hs.mdwn b/doc/bugs/GHC_8.4.3_build_failure_in_Types__47__DesktopNotify.hs.mdwn
new file mode 100644
index 000000000..e9f0d113f
--- /dev/null
+++ b/doc/bugs/GHC_8.4.3_build_failure_in_Types__47__DesktopNotify.hs.mdwn
@@ -0,0 +1,136 @@
+### Please describe the problem.
+The build fails with GHC 8.4.3
+
+### What steps will reproduce the problem?
+Be sure to use esqueleto HEAD to avoid getting stuck on dependencies.
+
+### What version of git-annex are you using? On what operating system?
+git-annex-6.20180529 on macOS
+
+### Please provide any additional information below.
+
+[[!format sh """
+==> cabal install --jobs=8 --max-backjumps=100000 --prefix=/usr/local/Cellar/git-annex/6.20180529 --flags=s3 webapp
+clang: warning: -Wl,-headerpad_max_install_names: 'linker' input unused
+clang: warning: argument unused during compilation: '-L/usr/local/opt/gettext/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/libffi/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/icu4c/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/openssl/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/readline/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/sqlite/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/lib'
+clang: warning: argument unused during compilation: '-L/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries'
+Resolving dependencies...
+Notice: installing into a sandbox located at
+/private/tmp/git-annex-20180529-26197-14ebtme/git-annex-6.20180529/.cabal-sandbox
+Configuring git-annex-6.20180529...
+Building git-annex-6.20180529...
+Failed to install git-annex-6.20180529
+Build log ( /private/tmp/git-annex-20180529-26197-14ebtme/git-annex-6.20180529/.cabal-sandbox/logs/ghc-8.4.3/git-annex-6.20180529-9O50etRsVNH1q2HUlIF2Gu.log ):
+cabal: Entering directory '.'
+[ 1 of 34] Compiling Utility.Applicative ( Utility/Applicative.hs, dist/dist-sandbox-669b35af/setup/Utility/Applicative.o )
+[ 2 of 34] Compiling Utility.Data     ( Utility/Data.hs, dist/dist-sandbox-669b35af/setup/Utility/Data.o )
+[ 3 of 34] Compiling Utility.Exception ( Utility/Exception.hs, dist/dist-sandbox-669b35af/setup/Utility/Exception.o )
+[ 4 of 34] Compiling Utility.Env.Basic ( Utility/Env/Basic.hs, dist/dist-sandbox-669b35af/setup/Utility/Env/Basic.o )
+[ 5 of 34] Compiling Build.Mans       ( Build/Mans.hs, dist/dist-sandbox-669b35af/setup/Build/Mans.o )
+[ 6 of 34] Compiling Utility.FileSize ( Utility/FileSize.hs, dist/dist-sandbox-669b35af/setup/Utility/FileSize.o )
+[ 7 of 34] Compiling Utility.Misc     ( Utility/Misc.hs, dist/dist-sandbox-669b35af/setup/Utility/Misc.o )
+[ 8 of 34] Compiling Utility.Monad    ( Utility/Monad.hs, dist/dist-sandbox-669b35af/setup/Utility/Monad.o )
+[ 9 of 34] Compiling Utility.PartialPrelude ( Utility/PartialPrelude.hs, dist/dist-sandbox-669b35af/setup/Utility/PartialPrelude.o )
+[10 of 34] Compiling Utility.Process.Shim ( Utility/Process/Shim.hs, dist/dist-sandbox-669b35af/setup/Utility/Process/Shim.o )
+[11 of 34] Compiling Utility.Process  ( Utility/Process.hs, dist/dist-sandbox-669b35af/setup/Utility/Process.o )
+[12 of 34] Compiling Utility.Network  ( Utility/Network.hs, dist/dist-sandbox-669b35af/setup/Utility/Network.o )
+[13 of 34] Compiling Utility.Split    ( Utility/Split.hs, dist/dist-sandbox-669b35af/setup/Utility/Split.o )
+[14 of 34] Compiling Utility.SafeCommand ( Utility/SafeCommand.hs, dist/dist-sandbox-669b35af/setup/Utility/SafeCommand.o )
+[15 of 34] Compiling Utility.ExternalSHA ( Utility/ExternalSHA.hs, dist/dist-sandbox-669b35af/setup/Utility/ExternalSHA.o )
+[16 of 34] Compiling Utility.FileSystemEncoding ( Utility/FileSystemEncoding.hs, dist/dist-sandbox-669b35af/setup/Utility/FileSystemEncoding.o )
+[17 of 34] Compiling Utility.SystemDirectory ( Utility/SystemDirectory.hs, dist/dist-sandbox-669b35af/setup/Utility/SystemDirectory.o )
+[18 of 34] Compiling Utility.Tmp      ( Utility/Tmp.hs, dist/dist-sandbox-669b35af/setup/Utility/Tmp.o )
+[19 of 34] Compiling Utility.Directory ( Utility/Directory.hs, dist/dist-sandbox-669b35af/setup/Utility/Directory.o )
+[20 of 34] Compiling Build.Version    ( Build/Version.hs, dist/dist-sandbox-669b35af/setup/Build/Version.o )
+[21 of 34] Compiling Utility.UserInfo ( Utility/UserInfo.hs, dist/dist-sandbox-669b35af/setup/Utility/UserInfo.o )
+[22 of 34] Compiling Utility.Path     ( Utility/Path.hs, dist/dist-sandbox-669b35af/setup/Utility/Path.o )
+[23 of 34] Compiling Common           ( Common.hs, dist/dist-sandbox-669b35af/setup/Common.o )
+[24 of 34] Compiling Utility.DottedVersion ( Utility/DottedVersion.hs, dist/dist-sandbox-669b35af/setup/Utility/DottedVersion.o )
+[25 of 34] Compiling Git.Version      ( Git/Version.hs, dist/dist-sandbox-669b35af/setup/Git/Version.o )
+[26 of 34] Compiling Build.TestConfig ( Build/TestConfig.hs, dist/dist-sandbox-669b35af/setup/Build/TestConfig.o )
+[27 of 34] Compiling Build.Configure  ( Build/Configure.hs, dist/dist-sandbox-669b35af/setup/Build/Configure.o )
+[28 of 34] Compiling Utility.OSX      ( Utility/OSX.hs, dist/dist-sandbox-669b35af/setup/Utility/OSX.o )
+[29 of 34] Compiling Utility.FreeDesktop ( Utility/FreeDesktop.hs, dist/dist-sandbox-669b35af/setup/Utility/FreeDesktop.o )
+[30 of 34] Compiling Config.Files     ( Config/Files.hs, dist/dist-sandbox-669b35af/setup/Config/Files.o )
+[31 of 34] Compiling Assistant.Install.Menu ( Assistant/Install/Menu.hs, dist/dist-sandbox-669b35af/setup/Assistant/Install/Menu.o )
+[32 of 34] Compiling Assistant.Install.AutoStart ( Assistant/Install/AutoStart.hs, dist/dist-sandbox-669b35af/setup/Assistant/Install/AutoStart.o )
+[33 of 34] Compiling Build.DesktopFile ( Build/DesktopFile.hs, dist/dist-sandbox-669b35af/setup/Build/DesktopFile.o )
+[34 of 34] Compiling Main             ( dist/dist-sandbox-669b35af/setup/setup.hs, dist/dist-sandbox-669b35af/setup/Main.o )
+Linking ./dist/dist-sandbox-669b35af/setup/setup ...
+  checking version...fatal: Not a git repository (or any of the parent directories): .git
+ 6.20180529
+  checking UPGRADE_LOCATION... not available
+  checking git... yes
+  checking git version... 2.10.1 (Apple Git-78)
+  checking cp -a... yes
+  checking cp -p... yes
+  checking cp --preserve=timestamps... no
+  checking cp --reflink=auto... no
+  checking xargs -0... yes
+  checking rsync... yes
+  checking curl... yes
+  checking bup... no
+  checking nice... yes
+  checking ionice... no
+  checking nocache... no
+  checking gpg... not available
+  checking lsof... lsof
+  checking git-remote-gcrypt... not available
+  checking ssh connection caching... yes
+  checking sha1... not available
+  checking sha256... not available
+  checking sha512... not available
+  checking sha224... not available
+  checking sha384... not available
+Configuring git-annex-6.20180529...
+clang: warning: -Wl,-headerpad_max_install_names: 'linker' input unused
+clang: warning: argument unused during compilation: '-L/usr/local/opt/gettext/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/libffi/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/icu4c/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/openssl/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/readline/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/opt/sqlite/lib'
+clang: warning: argument unused during compilation: '-L/usr/local/lib'
+clang: warning: argument unused during compilation: '-L/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries'
+Preprocessing executable 'git-annex' for git-annex-6.20180529..
+Building executable 'git-annex' for git-annex-6.20180529..
+[  1 of 593] Compiling Assistant.Types.BranchChange ( Assistant/Types/BranchChange.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Assistant/Types/BranchChange.o )
+[  2 of 593] Compiling Assistant.Types.ThreadName ( Assistant/Types/ThreadName.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Assistant/Types/ThreadName.o )
+[  3 of 593] Compiling Assistant.Types.TransferSlots ( Assistant/Types/TransferSlots.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Assistant/Types/TransferSlots.o )
+[  4 of 593] Compiling BuildFlags       ( BuildFlags.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/BuildFlags.o )
+[  5 of 593] Compiling BuildInfo        ( BuildInfo.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/BuildInfo.o )
+[  6 of 593] Compiling Build.BundledPrograms ( Build/BundledPrograms.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Build/BundledPrograms.o )
+[  7 of 593] Compiling Config.Cost      ( Config/Cost.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Config/Cost.o )
+[  8 of 593] Compiling Logs.Line        ( Logs/Line.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Logs/Line.o )
+[  9 of 593] Compiling Types.Availability ( Types/Availability.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Types/Availability.o )
+[ 10 of 593] Compiling Types.BranchState ( Types/BranchState.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Types/BranchState.o )
+[ 11 of 593] Compiling Types.Concurrency ( Types/Concurrency.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Types/Concurrency.o )
+[ 12 of 593] Compiling Types.Creds      ( Types/Creds.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Types/Creds.o )
+[ 13 of 593] Compiling Assistant.Types.CredPairCache ( Assistant/Types/CredPairCache.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Assistant/Types/CredPairCache.o )
+[ 14 of 593] Compiling Types.DesktopNotify ( Types/DesktopNotify.hs, dist/dist-sandbox-669b35af/build/git-annex/git-annex-tmp/Types/DesktopNotify.o )
+
+Types/DesktopNotify.hs:19:10: error:
+    • No instance for (Semigroup DesktopNotify)
+        arising from the superclasses of an instance declaration
+    • In the instance declaration for ‘Monoid DesktopNotify’
+   |
+19 | instance Monoid DesktopNotify where
+   |          ^^^^^^^^^^^^^^^^^^^^
+cabal: Leaving directory '.'
+cabal: Error: some packages failed to install:
+git-annex-6.20180529-9O50etRsVNH1q2HUlIF2Gu failed during the building phase.
+The exception was:
+ExitFailure 1
+"""]]
+
+Full log https://gist.github.com/ilovezfs/e3af135ed0362e0253a786c71f2a914f
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes! Thanks :)

add news item for git-annex 6.20180529
diff --git a/doc/news/version_6.20180529.mdwn b/doc/news/version_6.20180529.mdwn
new file mode 100644
index 000000000..c129a3eda
--- /dev/null
+++ b/doc/news/version_6.20180529.mdwn
@@ -0,0 +1,27 @@
+git-annex 6.20180529 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Prevent haskell http-client from decompressing gzip files, so downloads
+     of such files works the same as it used to with wget and curl.
+   * Workaround for bug in an old version of cryptonite that broke https
+     downloads, by using curl for downloads when git-annex is built with it.
+   * view, vadd: Fix crash when a git submodule has a name starting with a dot.
+   * Don't allow entering a view with staged or unstaged changes.
+   * move: --force was accidentially enabling two unrelated behaviors
+     since 6.20180427. The older behavior, which has never been well
+     documented and seems almost entirely useless, has been removed.
+   * copy: --force no longer does anything.
+   * migrate: Fix bug in migration between eg SHA256 and SHA256E,
+     that caused the extension to be included in SHA256 keys,
+     and omitted from SHA256E keys.
+     (Bug introduced in version 6.20170214)
+   * migrate: Check for above bug when migrating from SHA256 to SHA256
+     (and same for SHA1 to SHA1 etc), and remove the extension that should
+     not be in the SHA256 key.
+   * fsck: Detect and warn when keys need an upgrade, either to fix up
+     from the above migrate bug, or to add missing size information
+     (a long ago transition), or because of a few other past key related
+     bugs.
+   * git-annex-shell: GIT\_ANNEX\_SHELL\_APPENDONLY makes it allow writes,
+     but not deletion of annexed content. Note that securing pushes to
+     the git repository is left up to the user.
+   * setpresentkey: Added --batch support."""]]
\ No newline at end of file

comment
diff --git a/doc/todo/lockdown_hooks.mdwn b/doc/todo/lockdown_hooks.mdwn
index c190ccd53..0ec8b4624 100644
--- a/doc/todo/lockdown_hooks.mdwn
+++ b/doc/todo/lockdown_hooks.mdwn
@@ -44,3 +44,7 @@ as only read access is needed.
 Probably running a shell script is not too much overhead in many cases,
 if it was too slow, there could be a variant that is run once and 
 fed the names of files to operate on via stdin.
+
+> These hooks may be too specific to this purpose, while a more generalized
+> hook could also support things like [[storing_xattrs|support_for_storing_xattrs]]
+> --[[Joey]]
diff --git a/doc/todo/support_for_storing_xattrs/comment_2_c10f87259aef110510014ba16d0792ad._comment b/doc/todo/support_for_storing_xattrs/comment_2_c10f87259aef110510014ba16d0792ad._comment
new file mode 100644
index 000000000..f35944910
--- /dev/null
+++ b/doc/todo/support_for_storing_xattrs/comment_2_c10f87259aef110510014ba16d0792ad._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2018-05-29T16:51:48Z"
+ content="""
+I also would rather see this as a hook than built into git-annex. It's
+something that git-annex's metadata can be leveraged for, perhaps,
+but different use cases call for different amounts of data.
+
+I suppose the thing to provide a hook into is whenever git-annex adds an
+object content to .git/annex/objects. That will allow the hook
+to store the current xattrs after `git annex add`, and to restore them
+after eg `git annex get`.
+
+But, suppose you run: `git annex add file; git annex move file --to remote`
+Then the remote won't have the updated git-annex branch yet when it stores the
+object content, and so the hook run on it won't be able to do anything.
+
+When new git-annex branch version adds/changes xattr values,
+the local repository would need to be updated to reflect them.
+So, there would also need to be a hook that's run when git-annex
+metadata has changed.
+
+See also: [[lockdown_hooks]] which have different use cases, but seem to
+call for the same kind of hooks.
+"""]]

followup
diff --git a/doc/forum/Feature_Request__58___Specify_file_size_when_adding_files_to_URL_backend/comment_1_9d2fa7d796fd40718d0df9feeea654aa._comment b/doc/forum/Feature_Request__58___Specify_file_size_when_adding_files_to_URL_backend/comment_1_9d2fa7d796fd40718d0df9feeea654aa._comment
new file mode 100644
index 000000000..b3c3f261e
--- /dev/null
+++ b/doc/forum/Feature_Request__58___Specify_file_size_when_adding_files_to_URL_backend/comment_1_9d2fa7d796fd40718d0df9feeea654aa._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-05-29T16:41:34Z"
+ content="""
+I wonder if you've tried addurl --fast with a recent version of git-annex?
+
+Since it recently stopped shelling out to curl for every url, it keeps a
+http connection open to the server, and is limited only by network
+latency. Combined with -J10, it might be fast enough now to not be a
+serious bottleneck.
+
+The mangling of urls in keys is ad-hoc and I'd rather not document that
+as part of git-annex's interface, so building your own url keys is not
+recommended.
+
+I could see adding a size value to git-annex registerurl, and
+I think you'd also want git-annex fromkey to have that too so you can
+add the keys to the working tree.
+"""]]

Added a comment
diff --git a/doc/todo/support_for_storing_xattrs/comment_1_55c839d1b8af5150a5df29f0314e4b79._comment b/doc/todo/support_for_storing_xattrs/comment_1_55c839d1b8af5150a5df29f0314e4b79._comment
new file mode 100644
index 000000000..5708a1006
--- /dev/null
+++ b/doc/todo/support_for_storing_xattrs/comment_1_55c839d1b8af5150a5df29f0314e4b79._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="unqueued"
+ avatar="http://cdn.libravatar.org/avatar/3bcbe0c9e9825637ad7efa70f458640d"
+ subject="comment 1"
+ date="2018-05-28T14:55:34Z"
+ content="""
+I wrote a quick and dirty script to store xattrs in a git repo, because I wanted to keep my OSX labels and other metadata like resource forks in my annex. But it only works on OSX at the moment.
+
+https://github.com/unqueued/git-xattr
+
+I think maybe metadata storage would be best implemented as a hook.
+"""]]

Added a comment
diff --git a/doc/forum/Feature_request__58___Multiple_concurrent_transfers/comment_1_aa666349e9b524fd5394a259ad622c3b._comment b/doc/forum/Feature_request__58___Multiple_concurrent_transfers/comment_1_aa666349e9b524fd5394a259ad622c3b._comment
new file mode 100644
index 000000000..4ff1504ff
--- /dev/null
+++ b/doc/forum/Feature_request__58___Multiple_concurrent_transfers/comment_1_aa666349e9b524fd5394a259ad622c3b._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="unqueued"
+ avatar="http://cdn.libravatar.org/avatar/3bcbe0c9e9825637ad7efa70f458640d"
+ subject="comment 1"
+ date="2018-05-28T14:46:04Z"
+ content="""
+I have been toying with \"external\" transfer handlers. Sometimes when I am dealing with very large files, it makes more sense to just copy the keys outside of git-annex, and then after they are copied, run fsck. Perhaps at some point there could be a way to have a transfer \"shim\" of some kind, for handling transfers.
+"""]]

diff --git a/doc/forum/Feature_Request__58___Specify_file_size_when_adding_files_to_URL_backend.mdwn b/doc/forum/Feature_Request__58___Specify_file_size_when_adding_files_to_URL_backend.mdwn
new file mode 100644
index 000000000..bc7be2775
--- /dev/null
+++ b/doc/forum/Feature_Request__58___Specify_file_size_when_adding_files_to_URL_backend.mdwn
@@ -0,0 +1,10 @@
+It seems that the only way to add a key with the URL backend and with an explicit file size is to use addurl --fast, which requires a a connection to be made to the remote server. However, it becomes impractical when I have tens of thousands of URLs, and I end up hammering the server, when I know ahead of time exactly what the file sizes are. I know it is possible to create URL keys with a size attribute. It would be great to be able to have something like addurl --size=342345 --raw, or be able to pass a size to registerurl.
+
+I have figured out how to construct a SHA256 key outside of git-annex, but I've looked at the source and I admit I'm a bit over my head with manually constructing a URL key.
+
+
+I've been doing a lot of experimenting with [pre-seeding](https://git-annex.branchable.com/forum/__34__Preseeding__34___a_special_remote/), which I would like to share. I think it could be a really cool application of git-annex.
+
+So far, I've been able to generate a distfiles repo automatically for Gentoo portage, which contains BLAKE2B512 signatures for all of their distfiles. I think there could potentially be some really cool applications to keeping track of large sets of files, and I want to eventually try to make it so that portage could fetch distfiles from other users over ipfs.
+
+But, there are many interesting collections of files that are in open directories, and all I have is the filenames and sizes, and being able to quickly generate a repo with knowledge of the exact size would make it really easy to divvy up downloading and mirroring with git. I have a script that converts an FTP server to a git-annex repo with a WORM backend, but I still think that using URL keys would be a better representation of where the resource is.

Added a comment: append-only and gitolite
diff --git a/doc/todo/append-only_mode/comment_1_eaa7ae80c3758bccd23c4e0a8d1eefc4._comment b/doc/todo/append-only_mode/comment_1_eaa7ae80c3758bccd23c4e0a8d1eefc4._comment
new file mode 100644
index 000000000..fb904da84
--- /dev/null
+++ b/doc/todo/append-only_mode/comment_1_eaa7ae80c3758bccd23c4e0a8d1eefc4._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="https://christian.amsuess.com/chrysn"
+ nickname="chrysn"
+ avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc"
+ subject="append-only and gitolite"
+ date="2018-05-28T11:47:13Z"
+ content="""
+Thanks, that's a nice feature.
+
+How could this be integrated with gitolite? Should the `if ( can_write($repo) )` branch be amended by a check for the \"+\" flag ([short documentation](http://gitolite.com/gitolite/conf-2/) says that \"RW\" is roughly \"create or fast-forward, no deletes\" while \"RW+\" includes deletes and non-fast-forwards)?
+
+(This might change the user experience on repositories run with \"RW\" permissions currently, but IMO patching a possibly unwanted access that can be easily granted if so desired.)
+"""]]

setpresentkey: Added --batch support (for ronnypfa)
This commit was sponsored by Peter on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index d52cf6f0b..9d78cbd0b 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -24,6 +24,7 @@ git-annex (6.20180510) UNRELEASED; urgency=medium
   * git-annex-shell: GIT_ANNEX_SHELL_APPENDONLY makes it allow writes,
     but not deletion of annexed content. Note that securing pushes to
     the git repository is left up to the user.
+  * setpresentkey: Added --batch support.
 
  -- Joey Hess <id@joeyh.name>  Mon, 14 May 2018 13:42:41 -0400
 
diff --git a/Command/SetPresentKey.hs b/Command/SetPresentKey.hs
index 6e7075e8c..a954c884d 100644
--- a/Command/SetPresentKey.hs
+++ b/Command/SetPresentKey.hs
@@ -16,19 +16,40 @@ cmd = noCommit $
 	command "setpresentkey" SectionPlumbing
 		"change records of where key is present"
 		(paramPair paramKey (paramPair paramUUID "[1|0]"))
-		(withParams seek)
+		(seek <$$> optParser)
 
-seek :: CmdParams -> CommandSeek
-seek = withWords start
+data SetPresentKeyOptions = SetPresentKeyOptions
+	{ params :: CmdParams
+	, batchOption :: BatchMode
+	}
 
-start :: [String] -> CommandStart
-start (ks:us:vs:[]) = do
+optParser :: CmdParamsDesc -> Parser SetPresentKeyOptions
+optParser desc = SetPresentKeyOptions
+	<$> cmdParams desc
+	<*> parseBatchOption
+
+seek :: SetPresentKeyOptions -> CommandSeek
+seek o = case batchOption o of
+	Batch -> batchInput
+		(parseKeyStatus . words)
+		(batchCommandAction . start)
+	NoBatch -> either giveup (commandAction . start)
+		(parseKeyStatus $ params o)
+
+data KeyStatus = KeyStatus Key UUID LogStatus
+
+parseKeyStatus :: [String] -> Either String KeyStatus
+parseKeyStatus (ks:us:vs:[]) = do
+	k <- maybe (Left "bad key") Right (file2key ks)
+	let u = toUUID us
+	s <- maybe (Left "bad value") Right (parseStatus vs)
+	return $ KeyStatus k u s
+parseKeyStatus _ = Left "Bad input. Expected: key uuid value"
+
+start :: KeyStatus -> CommandStart
+start (KeyStatus k u s) = do
 	showStartKey "setpresentkey" k (mkActionItem k)
-	next $ perform k (toUUID us) s
-  where
-	k = fromMaybe (giveup "bad key") (file2key ks)
-	s = fromMaybe (giveup "bad value") (parseStatus vs)
-start _ = giveup "Wrong number of parameters"
+	next $ perform k u s
 
 perform :: Key -> UUID -> LogStatus -> CommandPerform
 perform k u s = next $ do
diff --git a/doc/git-annex-setpresentkey.mdwn b/doc/git-annex-setpresentkey.mdwn
index 9838b819d..44368ea05 100644
--- a/doc/git-annex-setpresentkey.mdwn
+++ b/doc/git-annex-setpresentkey.mdwn
@@ -14,6 +14,13 @@ the specified key's content is present in a remote with the specified uuid.
 Use 1 to indicate the key is present, or 0 to indicate the key is
 not present.
 
+# OPTIONS
+
+* `--batch`
+
+  Enables batch mode, in which lines are read from stdin.
+  The line format is "key uuid [1|0]"
+
 # SEE ALSO
 
 [[git-annex]](1)

idea
diff --git a/doc/todo/borg_special_remote.mdwn b/doc/todo/borg_special_remote.mdwn
new file mode 100644
index 000000000..0d4e41728
--- /dev/null
+++ b/doc/todo/borg_special_remote.mdwn
@@ -0,0 +1,15 @@
+borg backup is pretty cool, and could be a great special remote backend.
+In particular it does delta compression and stuff.
+
+There seem to be two ways it could work. Probably there are borg commands
+that allow storing a given blob in it, and retrieving a given blob. And
+that could be used for a traditional special remote.
+
+But also, if a whole git-annex repository has been backed up with borg,
+then git-annex could look inside such a backup, and see if
+.git/annex/object/ contains an object. It could then mark it as
+present in the borg special remote. This way you'd use borg to take
+backups, and git-annex would then be aware of what was backed up in borg,
+and could do things like count that as a copy.
+
+--[[Joey]]

GIT_ANNEX_SHELL_APPENDONLY
Makes it allow writes, but not deletion of annexed content. Note that
securing pushes to the git repository is left up to the user.
This commit was sponsored by Jack Hill on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index 519afb193..d52cf6f0b 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -21,6 +21,9 @@ git-annex (6.20180510) UNRELEASED; urgency=medium
     from the above migrate bug, or to add missing size information
     (a long ago transition), or because of a few other past key related
     bugs.
+  * git-annex-shell: GIT_ANNEX_SHELL_APPENDONLY makes it allow writes,
+    but not deletion of annexed content. Note that securing pushes to
+    the git repository is left up to the user.
 
  -- Joey Hess <id@joeyh.name>  Mon, 14 May 2018 13:42:41 -0400
 
diff --git a/CmdLine/GitAnnexShell.hs b/CmdLine/GitAnnexShell.hs
index 3dc31e602..c18aa1f59 100644
--- a/CmdLine/GitAnnexShell.hs
+++ b/CmdLine/GitAnnexShell.hs
@@ -17,6 +17,7 @@ import Annex.UUID
 import CmdLine.GitAnnexShell.Checks
 import CmdLine.GitAnnexShell.Fields
 import Remote.GCrypt (getGCryptUUID)
+import P2P.Protocol (ServerMode(..))
 
 import qualified Command.ConfigList
 import qualified Command.InAnnex
@@ -30,39 +31,44 @@ import qualified Command.NotifyChanges
 import qualified Command.GCryptSetup
 import qualified Command.P2PStdIO
 
-cmds_readonly :: [Command]
-cmds_readonly =
-	[ Command.ConfigList.cmd
-	, gitAnnexShellCheck Command.InAnnex.cmd
-	, gitAnnexShellCheck Command.LockContent.cmd
-	, gitAnnexShellCheck Command.SendKey.cmd
-	, gitAnnexShellCheck Command.TransferInfo.cmd
-	, gitAnnexShellCheck Command.NotifyChanges.cmd
-	]
-
-cmds_notreadonly :: [Command]
-cmds_notreadonly =
-	[ gitAnnexShellCheck Command.RecvKey.cmd
-	, gitAnnexShellCheck Command.DropKey.cmd
-	, gitAnnexShellCheck Command.Commit.cmd
-	, Command.GCryptSetup.cmd
-	]
+import qualified Data.Map as M
 
--- Commands that can operate readonly or not; they use checkNotReadOnly.
-cmds_readonly_capable :: [Command]
-cmds_readonly_capable =
-	[ gitAnnexShellCheck Command.P2PStdIO.cmd
+cmdsMap :: M.Map ServerMode [Command]
+cmdsMap = M.fromList $ map mk
+	[ (ServeReadOnly, readonlycmds)
+	, (ServeAppendOnly, appendcmds)
+	, (ServeReadWrite, allcmds)
 	]
-
-cmds_readonly_safe :: [Command]
-cmds_readonly_safe = cmds_readonly ++ cmds_readonly_capable
-
-cmds :: [Command]
-cmds = map (adddirparam . noMessages)
-	(cmds_readonly ++ cmds_notreadonly ++ cmds_readonly_capable)
   where
+	readonlycmds = 
+		[ Command.ConfigList.cmd
+		, gitAnnexShellCheck Command.InAnnex.cmd
+		, gitAnnexShellCheck Command.LockContent.cmd
+		, gitAnnexShellCheck Command.SendKey.cmd
+		, gitAnnexShellCheck Command.TransferInfo.cmd
+		, gitAnnexShellCheck Command.NotifyChanges.cmd
+		-- p2pstdio checks the enviroment variables to
+		-- determine the security policy to use
+		, gitAnnexShellCheck Command.P2PStdIO.cmd
+		]
+	appendcmds = readonlycmds ++
+		[ gitAnnexShellCheck Command.RecvKey.cmd
+		, gitAnnexShellCheck Command.Commit.cmd
+		]
+	allcmds =
+		[ gitAnnexShellCheck Command.DropKey.cmd
+		, Command.GCryptSetup.cmd
+		]
+
+	mk (s, l) = (s, map (adddirparam . noMessages) l)
 	adddirparam c = c { cmdparamdesc = "DIRECTORY " ++ cmdparamdesc c }
 
+cmdsFor :: ServerMode -> [Command]
+cmdsFor = fromMaybe [] . flip M.lookup cmdsMap
+
+cmdsList :: [Command]
+cmdsList = concat $ M.elems cmdsMap
+
 globalOptions :: [GlobalOption]
 globalOptions = 
 	globalSetter checkUUID (strOption
@@ -101,17 +107,19 @@ run c@(cmd:_)
 	| otherwise = external c
 
 builtins :: [String]
-builtins = map cmdname cmds
+builtins = map cmdname cmdsList
 
 builtin :: String -> String -> [String] -> IO ()
 builtin cmd dir params = do
-	unless (cmd `elem` map cmdname cmds_readonly_safe)
+	unless (cmd `elem` map cmdname (cmdsFor ServeReadOnly))
 		checkNotReadOnly
+	unless (cmd `elem` map cmdname (cmdsFor ServeAppendOnly))
+		checkNotAppendOnly
 	checkDirectory $ Just dir
 	let (params', fieldparams, opts) = partitionParams params
 	    rsyncopts = ("RsyncOptions", unwords opts)
 	    fields = rsyncopts : filter checkField (parseFields fieldparams)
-	dispatch False (cmd : params') cmds globalOptions fields mkrepo
+	dispatch False (cmd : params') cmdsList globalOptions fields mkrepo
 		"git-annex-shell"
 		"Restricted login shell for git-annex only SSH access"
   where
@@ -161,6 +169,6 @@ checkField (field, val)
 	| otherwise = False
 
 failure :: IO ()
-failure = giveup $ "bad parameters\n\n" ++ usage h cmds
+failure = giveup $ "bad parameters\n\n" ++ usage h cmdsList
   where
 	h = "git-annex-shell [-c] command [parameters ...] [option ...]"
diff --git a/CmdLine/GitAnnexShell/Checks.hs b/CmdLine/GitAnnexShell/Checks.hs
index 3409884c0..bcef88ce2 100644
--- a/CmdLine/GitAnnexShell/Checks.hs
+++ b/CmdLine/GitAnnexShell/Checks.hs
@@ -26,6 +26,12 @@ readOnlyEnv = "GIT_ANNEX_SHELL_READONLY"
 checkNotReadOnly :: IO ()
 checkNotReadOnly = checkEnv readOnlyEnv
 
+appendOnlyEnv :: String
+appendOnlyEnv = "GIT_ANNEX_SHELL_APPENDONLY"
+
+checkNotAppendOnly :: IO ()
+checkNotAppendOnly = checkEnv appendOnlyEnv
+
 checkEnv :: String -> IO ()
 checkEnv var = checkEnvSet var >>= \case
 	False -> noop
diff --git a/Command/P2PStdIO.hs b/Command/P2PStdIO.hs
index 38a3eb0cf..2bb28a310 100644
--- a/Command/P2PStdIO.hs
+++ b/Command/P2PStdIO.hs
@@ -26,10 +26,13 @@ seek _ = giveup "missing UUID parameter"
 
 start :: UUID -> CommandStart
 start theiruuid = do
-	servermode <- liftIO $ 
-		Checks.checkEnvSet Checks.readOnlyEnv >>= return . \case
-			True -> P2P.ServeReadOnly
-			False -> P2P.ServeReadWrite
+	servermode <- liftIO $ do
+		ro <- Checks.checkEnvSet Checks.readOnlyEnv
+		ao <- Checks.checkEnvSet Checks.appendOnlyEnv
+		return $ case (ro, ao) of
+			(True, _) -> P2P.ServeReadOnly
+			(False, True) -> P2P.ServeAppendOnly
+			(False, False) -> P2P.ServeReadWrite
 	myuuid <- getUUID
 	conn <- stdioP2PConnection <$> Annex.gitRepo
 	let server = do
diff --git a/P2P/Protocol.hs b/P2P/Protocol.hs
index 944c81995..49a3d5bf6 100644
--- a/P2P/Protocol.hs
+++ b/P2P/Protocol.hs
@@ -411,13 +411,21 @@ serveAuth myuuid = serverLoop handler
 				return ServerContinue
 	handler _ = return ServerUnexpected
 
-data ServerMode = ServeReadOnly | ServeReadWrite
+data ServerMode
+	= ServeReadOnly
+	-- ^ Allow reading, but not writing.
+	| ServeAppendOnly
+	-- ^ Allow reading, and storing new objects, but not deleting objects.
+	| ServeReadWrite
+	-- ^ Full read and write access.
+	deriving (Eq, Ord)
 
 -- | Serve the protocol, with a peer that has authenticated.
 serveAuthed :: ServerMode -> UUID -> Proto ()
 serveAuthed servermode myuuid = void $ serverLoop handler
   where
 	readonlyerror = net $ sendMessage (ERROR "this repository is read-only; write access denied")
+	appendonlyerror = net $ sendMessage (ERROR "this repository is append-only; removal denied")
 	handler (VERSION theirversion) = do
 		let v = min theirversion maxProtocolVersion
 		net $ setProtocolVersion v
@@ -439,22 +447,15 @@ serveAuthed servermode myuuid = void $ serverLoop handler
 		ServeReadWrite -> do
 			sendSuccess =<< local (removeContent key)
 			return ServerContinue

(Diff truncated)