diff --git a/doc/devblog/day_613__end-to-end_ByteString_milestone.mdwn b/doc/devblog/day_613__end-to-end_ByteString_milestone.mdwn
new file mode 100644
index 000000000..1b774c019
--- /dev/null
+++ b/doc/devblog/day_613__end-to-end_ByteString_milestone.mdwn
@@ -0,0 +1,17 @@
+The `bs` branch has reached a milestone: `git-annex find` and `git-annex
+get` (when all files are present) process ByteStrings end-to-end with 
+no String conversion. That sped it up by around 30% on top of the previous
+optimisations. 
+
+To get here, I spent a couple of days creating the
+[filepath-bytestring](https://joeyh.name/blog/entry/announcing_the_filepath-bytestring_haskell_library/)
+library, which git-annex will depend on. Lots more git-annex internals
+were switched to ByteString, especially everything having to do with
+statting files.
+
+Other commands, like `git-annex whereis`, still do some String
+conversions. Optimisation never ends.
+
+But the bs branch is ready to merge as-is, and the diff is 10 thousand lines,
+so not a branch I want to maintain for long. Planning to merge it after
+the next release.

diff --git a/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_3_529c38fc63540b32c51ae75529e9005e._comment b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_3_529c38fc63540b32c51ae75529e9005e._comment
new file mode 100644
index 000000000..d2a79d3cf
--- /dev/null
+++ b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_3_529c38fc63540b32c51ae75529e9005e._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="annex-verify"
+ date="2019-12-11T18:13:48Z"
+ content="""
+\"it is a feature of a external remote, not sure why it should be in the config\" -- because the user might not trust an external remote's implementation of this feature.  Besides bugs, there might be [[security exploits|security/CVE-2018-10857_and_CVE-2018-10859]] if external remotes could single-handedly disable verification.
+"""]]

diff --git a/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_2_fa9e2c0a83bad2e0c2da3016b9bf2100._comment b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_2_fa9e2c0a83bad2e0c2da3016b9bf2100._comment
new file mode 100644
index 000000000..ca032f7a0
--- /dev/null
+++ b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_2_fa9e2c0a83bad2e0c2da3016b9bf2100._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2019-12-10T22:01:12Z"
+ content="""
+> An external remote could also do its own checksum checking and then set `remote..annex-verify=false`
+
+that is an interesting idea, thanks!  Not sure if that makes it easy for mass consumption though since it is a feature of a external remote, not sure why it should be in the config.  Ideally it should be a property of a remote.
+
+Joey, what do you think in regard of built-in remotes?
+"""]]

diff --git a/doc/tips/android_sync_with_adb/comment_1_305dbaac1c9c526c4b467314960e3bcd._comment b/doc/tips/android_sync_with_adb/comment_1_305dbaac1c9c526c4b467314960e3bcd._comment
new file mode 100644
index 000000000..8542376aa
--- /dev/null
+++ b/doc/tips/android_sync_with_adb/comment_1_305dbaac1c9c526c4b467314960e3bcd._comment
@@ -0,0 +1,27 @@
+[[!comment format=mdwn
+ username="MichaelC"
+ avatar="http://cdn.libravatar.org/avatar/248593885d551a3912e488c4bc9d311c"
+ subject="Does android adb special remote support MTP?"
+ date="2019-12-10T21:23:31Z"
+ content="""
+I'm attempting to init this special remote with my Pixel 3a android phone (which connect via MTP) but keep hitting the following error.
+
+In Debian, this will mount to /run , specifically in my case
+
+/run/user/1000/gvfs/mtp\\:host\=Google_Pixel_3a_94MBY/Internal\ shared\ storage/DCIM
+
+git annex initremote android type=adb androiddirectory=/run/user/1000/gvfs/mtp\\:host\=Google_Pixel_3a_94MBY/Internal\ shared\ storage/DCIM  encryption=none exporttree=yes importtree=yes
+initremote android 
+git-annex: adb: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
+failed
+git-annex: initremote: 1 failed
+
+I've tried enclosing the androiddirectory in single/double quotes (based on the hope that the colon before host was the issue)
+I've also attempted to substitute a symbolic link to the directory instead - but no luck.
+
+(*) I can cd  into this directroy via xterm and browse etc.
+
+Thanks,
+M
+
+"""]]

diff --git a/doc/tips/android_sync_with_adb/comment_1_c504fe0c08bc820ec033172b164d3ccc._comment b/doc/tips/android_sync_with_adb/comment_1_c504fe0c08bc820ec033172b164d3ccc._comment
deleted file mode 100644
index 87c7fb778..000000000
--- a/doc/tips/android_sync_with_adb/comment_1_c504fe0c08bc820ec033172b164d3ccc._comment
+++ /dev/null
@@ -1,31 +0,0 @@
-[[!comment format=mdwn
- username="michael.clifford.com@7ca464bc0ba25fd5f2922deb8f531668727a66fb"
- nickname="michael.clifford.com"
- avatar="http://cdn.libravatar.org/avatar/248593885d551a3912e488c4bc9d311c"
- subject="android special remote via mtp supported?"
- date="2019-12-10T21:01:52Z"
- content="""
-I've tried to use this special remote with my android Pixel 3a phone. It connects via mtp.  
-In Debian, this mounts to the /run folder.
-Specifically in my case:  /run/user/1000/gvfs/mtp:host=Google_Pixel_3a_94MBY0DHLL/Internal shared storage/DCIM
-
-I've tried as follows but I keep getting below error.
-
-
-git annex initremote android type=adb androiddirectory=/run/user/1000/gvfs/mtp\:host\=Google_Pixel_3a_94MBY0DHLL/Internal\ shared\ storage/DCIM  encryption=none exporttree=yes importtree=yes
-initremote android 
-git-annex: adb: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
-failed
-git-annex: initremote: 1 failed
-
-I've attempted to surround the androiddirectory is both single and double quotes, or substituting it with a symbolic link, but can't get past this.
-
-Thanks
-M
-
-
-
-
-
-
-"""]]

diff --git a/doc/tips/android_sync_with_adb/comment_1_c504fe0c08bc820ec033172b164d3ccc._comment b/doc/tips/android_sync_with_adb/comment_1_c504fe0c08bc820ec033172b164d3ccc._comment
new file mode 100644
index 000000000..87c7fb778
--- /dev/null
+++ b/doc/tips/android_sync_with_adb/comment_1_c504fe0c08bc820ec033172b164d3ccc._comment
@@ -0,0 +1,31 @@
+[[!comment format=mdwn
+ username="michael.clifford.com@7ca464bc0ba25fd5f2922deb8f531668727a66fb"
+ nickname="michael.clifford.com"
+ avatar="http://cdn.libravatar.org/avatar/248593885d551a3912e488c4bc9d311c"
+ subject="android special remote via mtp supported?"
+ date="2019-12-10T21:01:52Z"
+ content="""
+I've tried to use this special remote with my android Pixel 3a phone. It connects via mtp.  
+In Debian, this mounts to the /run folder.
+Specifically in my case:  /run/user/1000/gvfs/mtp:host=Google_Pixel_3a_94MBY0DHLL/Internal shared storage/DCIM
+
+I've tried as follows but I keep getting below error.
+
+
+git annex initremote android type=adb androiddirectory=/run/user/1000/gvfs/mtp\:host\=Google_Pixel_3a_94MBY0DHLL/Internal\ shared\ storage/DCIM  encryption=none exporttree=yes importtree=yes
+initremote android 
+git-annex: adb: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
+failed
+git-annex: initremote: 1 failed
+
+I've attempted to surround the androiddirectory is both single and double quotes, or substituting it with a symbolic link, but can't get past this.
+
+Thanks
+M
+
+
+
+
+
+
+"""]]

diff --git a/doc/bugs/on_some_remotes_failing_to_detect_annex_spits_out_message_to_stderr_and_empty_lines_to_stderr__44___ignores_--json-error-messages.mdwn b/doc/bugs/on_some_remotes_failing_to_detect_annex_spits_out_message_to_stderr_and_empty_lines_to_stderr__44___ignores_--json-error-messages.mdwn
new file mode 100644
index 000000000..4c4fca407
--- /dev/null
+++ b/doc/bugs/on_some_remotes_failing_to_detect_annex_spits_out_message_to_stderr_and_empty_lines_to_stderr__44___ignores_--json-error-messages.mdwn
@@ -0,0 +1,56 @@
+### Please describe the problem.
+
+
+### What steps will reproduce the problem?
+
+
+### What version of git-annex are you using? On what operating system?
+
+
+### Please provide any additional information below.
+
+[[!format sh """
+lena:/tmp
+$> git clone http://kumo.ovgu.de/~mih/myHP/bd2/4e4aa-7aea-11e6-9d5d-002590f97d84/
+Cloning into '4e4aa-7aea-11e6-9d5d-002590f97d84'...
+
+$> cd 4e4aa-7aea-11e6-9d5d-002590f97d84 
+
+$> git annex init                                                                
+init  (merging origin/git-annex into git-annex...)
+(recording state in git...)
+(scanning for unlocked files...)
+
+  Failed to get annex.uuid configuration of repository origin
+
+  Instead, got: "core.repositoryformatversion\n0\NULcore.filemode\ntrue\NULcore.bare\ntrue\NUL"
+
+  This is unexpected; please check the network transport!
+(Auto enabling special remote datalad-archives...)
+(Auto enabling special remote inm7-storage...)
+
+  Cannot run git-annex-remote-ria -- It is not installed in PATH (/usr/lib/git-annex.linux/bin:/usr/lib/git-core:/home/yoh/picts/mris/heudiconv-master/venvs/dev3/bin:/home/yoh/gocode/bin:/home/yoh/gocode/bin:/home/yoh/bin:/home/yoh/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/usr/local/sbin:/usr/lib/git-annex.linux/extra)
+ok
+(recording state in git...)
+
+"""]]
+
+so two issues in above:
+- not clear why it dumps git config if fetched instead of just announcing that remote has no git-annex and set to ignore
+- empty lines between all the messages go to stdout:
+
+[[!format sh """
+$> git annex init 2>/dev/null
+init  (scanning for unlocked files...)
+
+
+
+(Auto enabling special remote inm7-storage...)
+
+ok
+"""]]
+
+PS ignore "ignores --json-error-messages" part of the subject -- was detected in 7.20190819+git2-g908476a9b-1~ndall+1 during `get` but upgrade to 7.20191114+git43-ge29663773-1~ndall+1 resolved it
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

diff --git a/doc/internals/key_format/comment_1_4ec126bffafc81fae04e183874ffce39._comment b/doc/internals/key_format/comment_1_4ec126bffafc81fae04e183874ffce39._comment
new file mode 100644
index 000000000..8a3e8f7bf
--- /dev/null
+++ b/doc/internals/key_format/comment_1_4ec126bffafc81fae04e183874ffce39._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="https://christian.amsuess.com/chrysn"
+ nickname="chrysn"
+ avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc"
+ subject="Key character set"
+ date="2019-12-10T10:27:58Z"
+ content="""
+Are there limitations on the character set git-annex guarantees?
+
+It appears from experiments that git-annex only uses ASCII characters in there, given both a file 'test.extü' (in UTF-8 encoding) 'test.ext\xff' produced extension-free key names in the SHA256E hash – but it'd be good to have that confirmed.
+"""]]

diff --git a/doc/todo/optimize_by_converting_String_to_ByteString.mdwn b/doc/todo/optimize_by_converting_String_to_ByteString.mdwn
index 13d29603f..7ac7efe38 100644
--- a/doc/todo/optimize_by_converting_String_to_ByteString.mdwn
+++ b/doc/todo/optimize_by_converting_String_to_ByteString.mdwn
@@ -3,19 +3,26 @@ Converting to ByteString, and RawFilePath, should speed it up
 significantly, according to [[/profiling]].
 
 I've made a test branch, `bs`, to see what kind of performance improvement
-to expect. Most commands don't built yet in that branch, but `git annex
-find` does. Speedups range from 28-66%. The files fly by much more
-snappily.
+to expect. 
 
-As well as adding back all the code that was disabled to get it to build,
-the `bs` branch has quite a lot of things still needing work, including:
+Benchmarking `git-annex find`, speedups range from 28-66%. The files fly by
+much more snappily. Other commands likely also speed up, but do more work
+than find so the improvement is not as large.
+
+The `bs` branch is in a mergeable state now, but still needs work:
 
 * Eliminate all the fromRawFilePath, toRawFilePath, encodeBS,
   decodeBS conversions. Or at least most of them. There are likely
   quite a few places where a value is converted back and forth several times.
 
-  It would be good to instrument them with Debug.Trace and find out which
-  are the hot ones that get called, and focus on those.
+  As a first step, profile and look for the hot spots. Known hot spots:
+  
+  * keyFile uses fromRawFilePath and that adds around 3% overhead in `git-annex find`. 
+    Converting it to a RawFilePath needs a version of `</>` for RawFilePaths.
+  * getJournalFileStale uses fromRawFilePath, and adds 3-5% overhead in
+    `git-annex whereis`. Converting it to RawFilePath needs a version 
+    of `</>` for RawFilePaths. It also needs a ByteString.readFile
+    for RawFilePath.
 
 * System.FilePath is not available for RawFilePath, and many of the
   conversions are to get a FilePath in order to use that library.
@@ -28,7 +35,3 @@ the `bs` branch has quite a lot of things still needing work, including:
   avoiding a conversion. Note that these are only available on unix, not
   windows, so a compatability shim will be needed.
   (I can't seem to find any library that provides one.)
-
-* Eliminate some Data.ByteString.Lazy.toStrict, which is a slow copy.
-
-* Use ByteString for parsing git config to speed up startup.

diff --git a/doc/devblog/day_612__building_again.mdwn b/doc/devblog/day_612__building_again.mdwn
new file mode 100644
index 000000000..a6e4e518a
--- /dev/null
+++ b/doc/devblog/day_612__building_again.mdwn
@@ -0,0 +1,11 @@
+I've gotten the `bs` branch to build everything again. Was not trivial,
+the diff is over 7000 lines.
+
+Had hoped this was a mechanical enough conversion it would not introduce
+many bugs, but the test suite quickly found a lot of problems. So that
+branch is not ready for merging yet.
+
+I'm considering making a library that's like
+[filepath](http://hackage.haskell.org/package/filepath) but for
+RawFilePath. That would probably speed git-annex up by another 5% or so,
+in places where it currently has to convert back to FilePath.

diff --git a/doc/thanks/list b/doc/thanks/list
index 00670e948..3a5de35e3 100644
--- a/doc/thanks/list
+++ b/doc/thanks/list
@@ -70,3 +70,4 @@ Ryan Rix,
 Svenne Krap, 
 Jelmer Vernooij, 
 Rian McGuire, 
+Cesar, 

diff --git a/doc/devblog/day_608__easier_git-lfs_setup/comment_1_b72bea21d4445d12cade9f54ecc3767d._comment b/doc/devblog/day_608__easier_git-lfs_setup/comment_1_b72bea21d4445d12cade9f54ecc3767d._comment
new file mode 100644
index 000000000..8a45019d3
--- /dev/null
+++ b/doc/devblog/day_608__easier_git-lfs_setup/comment_1_b72bea21d4445d12cade9f54ecc3767d._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="jkrenzer"
+ avatar="http://cdn.libravatar.org/avatar/49eda64e3bd130a5dfaf48f09218cfb1"
+ subject="Great work! Thank you!"
+ date="2019-12-05T12:10:47Z"
+ content="""
+Hi Joe,
+
+I have been using git-annex since several years now and am really happy with it.
+
+But the addition of git-lfs support was, in my humble opinion, an extraordinary leap forward! After gitlab ceased support in favour of lfs and with the advent of lfs in github, annex-integration with widespread used collaborative git-platforms was not very straight-forward anymore. And so it became harder to justify or promote it's use in my institute or other organizations I work for. This has change now again for the better!
+
+So thank you very much for all the hard work and regarding the daunting speed with which new developments and features (version 8 already!) arrive here: Please take good care and do not overstrain yourself. 
+
+Kind regards,
+
+Jörn
+"""]]

diff --git a/doc/git-annex-unused/comment_9_cfcc0bab810d58b7db66c2fa4e92f769._comment b/doc/git-annex-unused/comment_9_cfcc0bab810d58b7db66c2fa4e92f769._comment
new file mode 100644
index 000000000..41477d216
--- /dev/null
+++ b/doc/git-annex-unused/comment_9_cfcc0bab810d58b7db66c2fa4e92f769._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="comment 9"
+ date="2019-12-02T16:58:25Z"
+ content="""
+\"I synced to all remotes and dropped everything in 'here'\" -- [[git-annex-unused]] \"Checks the *annex*\" for the unused contents (unless `--from=repository` is used), so if you dropped everything in `here`, there's nothing to find.  But it seems from `du` results that contents wasn't actually dropped?  [[git-annex-whereis]] tells where git-annex thinks contents is.
+"""]]

diff --git a/doc/git-annex-unused/comment_8_10ee2770ffe1cde367f3d310b2670539._comment b/doc/git-annex-unused/comment_8_10ee2770ffe1cde367f3d310b2670539._comment
new file mode 100644
index 000000000..b7377c955
--- /dev/null
+++ b/doc/git-annex-unused/comment_8_10ee2770ffe1cde367f3d310b2670539._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="dropping contents of old keys after migration"
+ date="2019-12-02T16:48:47Z"
+ content="""
+\"May I delete them\" -- `git-annex-drop --force` may be safer, as it also updates [[location_tracking]].  You might also want to [[git-annex-dead]] the dropped keys to prevent [[git-annex-fsck]] from complaining about lost contents.
+
+Re: why [[git-annex-unused]] isn't finding the unused contents, try running it with `--used-refspec=+HEAD`, and make sure `annex.used-refspec` git config is not set.  Note that this will mark as unused any annexed contents not referenced from the latest tree of the HEAD branch, e.g. annexed files that were removed in some older commit.
+"""]]

diff --git a/doc/git-annex-unused/comment_7_e21b036767651c5cfdd34bbd24a31fb5._comment b/doc/git-annex-unused/comment_7_e21b036767651c5cfdd34bbd24a31fb5._comment
new file mode 100644
index 000000000..bcc17e1d5
--- /dev/null
+++ b/doc/git-annex-unused/comment_7_e21b036767651c5cfdd34bbd24a31fb5._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="comment 7"
+ date="2019-12-02T16:02:47Z"
+ content="""
+Did you commit after git-migrate?   Does the worktree have any uncommitted changes?
+"""]]

diff --git a/doc/git-annex-unused/comment_6_962097729f8b4a657bb7b9863fdffb68._comment b/doc/git-annex-unused/comment_6_962097729f8b4a657bb7b9863fdffb68._comment
new file mode 100644
index 000000000..564bbab66
--- /dev/null
+++ b/doc/git-annex-unused/comment_6_962097729f8b4a657bb7b9863fdffb68._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="atrent"
+ avatar="http://cdn.libravatar.org/avatar/6069dfebff03997460874771defa0fa4"
+ subject="P.P.S. i dropped all local copies"
+ date="2019-12-02T08:03:01Z"
+ content="""
+I forgot to tell that after migrating I synced to all remotes and dropped everything in 'here', that's why I was expecting no more objects locally.
+"""]]

diff --git a/doc/git-annex-unused/comment_5_65ab9f687f2817199eda7455d9b82677._comment b/doc/git-annex-unused/comment_5_65ab9f687f2817199eda7455d9b82677._comment
new file mode 100644
index 000000000..f68858940
--- /dev/null
+++ b/doc/git-annex-unused/comment_5_65ab9f687f2817199eda7455d9b82677._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="atrent"
+ avatar="http://cdn.libravatar.org/avatar/6069dfebff03997460874771defa0fa4"
+ subject="P.S. they are all SHA256E"
+ date="2019-12-02T07:29:06Z"
+ content="""
+the \"lost\" objects are all SHA256E (*with* the \"E\")
+"""]]

diff --git a/doc/git-annex-unused/comment_4_5b208cb45335f0d383a456865068e659._comment b/doc/git-annex-unused/comment_4_5b208cb45335f0d383a456865068e659._comment
new file mode 100644
index 000000000..a10db0d1a
--- /dev/null
+++ b/doc/git-annex-unused/comment_4_5b208cb45335f0d383a456865068e659._comment
@@ -0,0 +1,29 @@
+[[!comment format=mdwn
+ username="atrent"
+ avatar="http://cdn.libravatar.org/avatar/6069dfebff03997460874771defa0fa4"
+ subject="can't find unused objects"
+ date="2019-12-02T07:26:41Z"
+ content="""
+I recently migrated an annex to SHA256 (without \"E\") and I'm now trying to clean the repo from unused data.
+I have a strange situation: there are 62G of unused objects:
+
+    $ du -ks .git/annex/objects/
+    64334024	.git/annex/objects/
+
+but 'git annex unused' gives me only:
+
+    $ git annex unused
+    unused ...
+    Some annexed data is no longer used by any files:
+    NUMBER  KEY
+    1       SHA256E-s27--32efec98dc9e05442fc2385bb85d855a8c7824c68abd4ab5bf55a4dfe412b334.pdf
+    (To see where data was previously used, try: git log --stat --no-textconv -S'KEY')
+    To remove unwanted data: git-annex dropunused NUMBER
+    ok
+
+I've checked (through a small shell script) that none of the object is in fact referenced by any symlink...
+
+May I delete them? Shall I do some other checking/fscking/repairing?
+
+Thank you
+"""]]

diff --git a/doc/install/openSUSE/comment_1_a5aea1ef644d0402d3caf593fef2456f._comment b/doc/install/openSUSE/comment_1_a5aea1ef644d0402d3caf593fef2456f._comment
new file mode 100644
index 000000000..ee0bf3109
--- /dev/null
+++ b/doc/install/openSUSE/comment_1_a5aea1ef644d0402d3caf593fef2456f._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="nangal.vivek@08b8bc308cb03037792b7930fd839b9deec118df"
+ nickname="nangal.vivek"
+ avatar="http://cdn.libravatar.org/avatar/f8e7f170b837feb1008df116c7f9d0de"
+ subject="not able to find git-annex on openSUSE using zypper"
+ date="2019-12-01T17:04:53Z"
+ content="""
+Getting the following error on runnning `zypper in git-annex`
+
+    Loading repository data...
+    Reading installed packages...
+    Package 'git-annex' not found.
+
+I am running openSUSE for WSL with the following info
+
+    NAME=\"openSUSE Leap\"
+    VERSION=\"15.1 \"
+    ID=\"opensuse-leap\"
+    ID_LIKE=\"suse opensuse\"
+    VERSION_ID=\"15.1\"
+    PRETTY_NAME=\"openSUSE Leap 15.1\"
+    ANSI_COLOR=\"0;32\"
+    CPE_NAME=\"cpe:/o:opensuse:leap:15.1\"
+    BUG_REPORT_URL=\"https://bugs.opensuse.org\"
+    HOME_URL=\"https://www.opensuse.org/\"
+"""]]

diff --git a/doc/internals/comment_15_c3d12d14e4d044f39829c5d92f523655._comment b/doc/internals/comment_15_c3d12d14e4d044f39829c5d92f523655._comment
new file mode 100644
index 000000000..aa630ff5b
--- /dev/null
+++ b/doc/internals/comment_15_c3d12d14e4d044f39829c5d92f523655._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="atrent"
+ avatar="http://cdn.libravatar.org/avatar/6069dfebff03997460874771defa0fa4"
+ subject="migrating..."
+ date="2019-11-30T22:30:06Z"
+ content="""
+I'm git-annex-migrating (to SHA256) now, thank you for all suggestions!
+"""]]

diff --git a/doc/todo/Preserve_permissions_or_xattr___40__SELinux__41__.mdwn b/doc/todo/Preserve_permissions_or_xattr___40__SELinux__41__.mdwn
deleted file mode 100644
index 42318e447..000000000
--- a/doc/todo/Preserve_permissions_or_xattr___40__SELinux__41__.mdwn
+++ /dev/null
@@ -1,21 +0,0 @@
-Using cp -a, I'm able to preserve the permissions and SELinux context to one repo like so:
-
-    drwx------.  2 user user unconfined_u:object_r:ssh_home_t:s0       4096 Nov 30 14:38 .ssh/
-
-But when syncing:
-
-    git annex add --include-dotfiles . || fail
-    git annex sync --content --message="$(date +%F)" || fail
-
-    for remote in $(git remote)
-    do
-        URL=$(git remote get-url "$remote")
-        cd "$URL" || fail
-        git annex sync --content --message="$(date +%F)" || fail
-    done
-
-Neither the permissions nor the SELinux context are preserved:
-
-    drwxrwxr-x. 2 user user unconfined_u:object_r:user_home_t:s0  4096 Nov 30 14:38 .ssh/
-
-Note that I'm unlocking all the files, so I'm not using any symlinks. It would be nice if these could be propagated with sync. 

diff --git a/doc/internals/comment_14_3f62751c2dd041f4ead1c6580ea5eec1._comment b/doc/internals/comment_14_3f62751c2dd041f4ead1c6580ea5eec1._comment
new file mode 100644
index 000000000..a950c8f73
--- /dev/null
+++ b/doc/internals/comment_14_3f62751c2dd041f4ead1c6580ea5eec1._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="hardlinking identical files in annex may break invariants"
+ date="2019-11-30T21:36:38Z"
+ content="""
+P.S. Re: hardlinking identical files -- git-annex [[keeps track of inodes|todo/inode_based_clean_filter_for_less_surprising_git_add]] where contents is stored, so deleting a file might make that info stale.  Also, dropping one key will drop another key's contents without updating [[location_tracking]] info.  And dropping then getting files would lead to two separate copies again.   So I wouldn't recommend that.
+
+See also [[tips/local_caching_of_annexed_files]].
+"""]]

diff --git a/doc/internals/comment_13_e45b6fa035a30703618448a0f764f935._comment b/doc/internals/comment_13_e45b6fa035a30703618448a0f764f935._comment
new file mode 100644
index 000000000..69452682e
--- /dev/null
+++ b/doc/internals/comment_13_e45b6fa035a30703618448a0f764f935._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="comment 13"
+ date="2019-11-30T21:11:53Z"
+ content="""
+[[git-annex-migrate]] to a backend not ending in E (e.g. SHA256 not SHA256E), then [[git-annex-unused]] to drop the old keys.
+
+"""]]

diff --git a/doc/internals/comment_12_f0325cefa5cd53a5a897046606137cef._comment b/doc/internals/comment_12_f0325cefa5cd53a5a897046606137cef._comment
new file mode 100644
index 000000000..fe35b66fa
--- /dev/null
+++ b/doc/internals/comment_12_f0325cefa5cd53a5a897046606137cef._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="atrent"
+ avatar="http://cdn.libravatar.org/avatar/6069dfebff03997460874771defa0fa4"
+ subject="no collisions"
+ date="2019-11-30T20:37:00Z"
+ content="""
+I can confirm that these are not collisions: these identical files are the same photos with different names, shame on Dropbox syncing from my smartphone. I was actually hoping to dedupe through git-annex ;-)
+
+Some more questions/suggestions/conversation-starters:
+
+* I suppose I can dedup them with rdfind (i.e., hardlinking identical files), do you foresee any side effects?
+
+* may I change the hash function of git-annex to something not depending on filenames? (I suppose so, I'll have a look at the docs)
+
+* if I can change the hash function can I regenerate the whole annex without re-creating it? (again I'll have a look at docs)
+
+Thanks
+"""]]

diff --git a/doc/todo/Preserve_permissions_or_xattr___40__SELinux__41__.mdwn b/doc/todo/Preserve_permissions_or_xattr___40__SELinux__41__.mdwn
new file mode 100644
index 000000000..42318e447
--- /dev/null
+++ b/doc/todo/Preserve_permissions_or_xattr___40__SELinux__41__.mdwn
@@ -0,0 +1,21 @@
+Using cp -a, I'm able to preserve the permissions and SELinux context to one repo like so:
+
+    drwx------.  2 user user unconfined_u:object_r:ssh_home_t:s0       4096 Nov 30 14:38 .ssh/
+
+But when syncing:
+
+    git annex add --include-dotfiles . || fail
+    git annex sync --content --message="$(date +%F)" || fail
+
+    for remote in $(git remote)
+    do
+        URL=$(git remote get-url "$remote")
+        cd "$URL" || fail
+        git annex sync --content --message="$(date +%F)" || fail
+    done
+
+Neither the permissions nor the SELinux context are preserved:
+
+    drwxrwxr-x. 2 user user unconfined_u:object_r:user_home_t:s0  4096 Nov 30 14:38 .ssh/
+
+Note that I'm unlocking all the files, so I'm not using any symlinks. It would be nice if these could be propagated with sync. 

diff --git a/doc/backends.mdwn b/doc/backends.mdwn
index ee6742902..ee305b168 100644
--- a/doc/backends.mdwn
+++ b/doc/backends.mdwn
@@ -1,4 +1,4 @@
-When a file is annexed, a key is generated from its content and/or filesystem
+When a file is annexed, a [[key|internals/key_format]] is generated from its content and/or filesystem
 metadata. The file checked into git symlinks to the key. This key can later
 be used to retrieve the file's content (its value).
 

diff --git a/doc/internals/comment_11_9758bb3a17f63b4dcf51742ea482dbe9._comment b/doc/internals/comment_11_9758bb3a17f63b4dcf51742ea482dbe9._comment
new file mode 100644
index 000000000..3eb9d3442
--- /dev/null
+++ b/doc/internals/comment_11_9758bb3a17f63b4dcf51742ea482dbe9._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="same contents with different keys"
+ date="2019-11-30T16:51:58Z"
+ content="""
+@atrent -- some [[backends]] (like SHA256E) base the key not just on object contents, but also on part of its filename (the extension).   So the same content can exist with two different keys.  In your example, the same contents exists in one file ending with .jpg and in another ending with .56.jpeg .  (This is done to give the annexed contents the same extension as the original file had before annexing, to avoid confusing some programs).  There are also backends like WORM and URL, not based on checksums, that could lead to different keys with same contents.  There could also be same contents added under different backends (see also [[`git-annex-migrate`|git-annex-migrate]]).  Finally, there is the theoretical possibility of hash collisions.
+"""]]

diff --git a/doc/internals/comment_10_c4298babd96b2596bd4f6ad828212c92._comment b/doc/internals/comment_10_c4298babd96b2596bd4f6ad828212c92._comment
new file mode 100644
index 000000000..794735a02
--- /dev/null
+++ b/doc/internals/comment_10_c4298babd96b2596bd4f6ad828212c92._comment
@@ -0,0 +1,31 @@
+[[!comment format=mdwn
+ username="atrent"
+ avatar="http://cdn.libravatar.org/avatar/6069dfebff03997460874771defa0fa4"
+ subject="duplicate objects?"
+ date="2019-11-30T14:04:17Z"
+ content="""
+Do I understand correctly that in .git/annex/objects dir there should be no duplicates?
+Here follows a run of 'rdfind' done in the objects dir:
+
+    $ rdfind .
+    Now scanning \".\", found 12874 files.
+    Now have 12874 files in total.
+    Removed 0 files due to nonunique device and inode.
+    Total size is 75579281486 bytes or 70 GiB
+    Removed 8376 files due to unique sizes from list.4498 files left.
+    Now eliminating candidates based on first bytes:removed 68 files from list.4430 files left.
+    Now eliminating candidates based on last bytes:removed 66 files from list.4364 files left.
+    Now eliminating candidates based on sha1 checksum:removed 0 files from list.4364 files left.
+    It seems like you have 4364 files that are not unique
+    Totally, 10 GiB can be reduced.
+    Now making results file results.txt
+
+And here is an example pair of dupes (excerpt from the abovementioned 'results.txt'):
+
+    DUPTYPE_FIRST_OCCURRENCE 2073 3 86558 26 21057567 1 ./53/zv/SHA256E-s86558--e79a0891bb94fc9212ce2f28178fe84591c5fb24c07b5239d367099118e12ede.jpg/SHA256E-s86558--e79a0891bb94fc9212ce2f28178fe84591c5fb24c07b5239d367099118e12ede.jpg
+    DUPTYPE_WITHIN_SAME_TREE -2073 3 86558 26 1080608 1 ./7w/w2/SHA256E-s86558--e79a0891bb94fc9212ce2f28178fe84591c5fb24c07b5239d367099118e12ede.56.jpeg/SHA256E-s86558--e79a0891bb94fc9212ce2f28178fe84591c5fb24c07b5239d367099118e12ede.56.jpeg
+
+Any clues?
+
+Thank you
+"""]]

diff --git a/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file/comment_2_9a63b2918f5621efbfe8cdb33b23ff21._comment b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file/comment_2_9a63b2918f5621efbfe8cdb33b23ff21._comment
new file mode 100644
index 000000000..bf9004d76
--- /dev/null
+++ b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file/comment_2_9a63b2918f5621efbfe8cdb33b23ff21._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2019-11-29T18:09:45Z"
+ content="""
+one of the use-cases (will be) https://gin.g-node.org/ -- an archive of (primarily) electrophys data.  The platform is based on gogs, but uses git-annex underneath.  It \"will be\" because currently access to git-annex is provided only via ssh, but as of today it is already possible to `git clone` (tried on public, didn't try private) datasets via https, and developers are looking into exposing git-annex also via http.  To access private datasets authentication will need to be handled
+"""]]

diff --git a/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file.mdwn b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file.mdwn
index 353387703..d93c3623a 100644
--- a/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file.mdwn
+++ b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file.mdwn
@@ -141,6 +141,6 @@ IMHO, ideally 1. should be addressed properly (authentication), and for 2. annex
 git annex 7.20190819+git2-g908476a9b-1~ndall+1 and the same with bleeding edge 7.20191114+git43-ge29663773-1~ndall+1 (probably that commit is the one with my patch for stricter git versioning, so use the count of 42 ;))
 
 [[!meta author=yoh]]
-[[!tag projects/canada]]
+[[!tag projects/dandi]]
 
 

diff --git a/doc/todo/git-lfs_special_remote_simpler_setup/comment_1_4e9f8b60dd1b705d4755200dada8801c._comment b/doc/todo/git-lfs_special_remote_simpler_setup/comment_1_4e9f8b60dd1b705d4755200dada8801c._comment
new file mode 100644
index 000000000..f9af1d11c
--- /dev/null
+++ b/doc/todo/git-lfs_special_remote_simpler_setup/comment_1_4e9f8b60dd1b705d4755200dada8801c._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="reference original bug report"
+ date="2019-11-29T17:58:28Z"
+ content="""
+original bug report was https://git-annex.branchable.com/bugs/git-lfs_remote_URL_is_not_recorded__63__/ for an attempt to share some NWB data on github's LFS
+"""]]

diff --git a/doc/bugs/WSL_adjusted_braches__58___smudge_fails_with_sqlite_thread_crashed_-_locking_protocol/comment_3_2a9f0396df93d9500306b3d7039803a2._comment b/doc/bugs/WSL_adjusted_braches__58___smudge_fails_with_sqlite_thread_crashed_-_locking_protocol/comment_3_2a9f0396df93d9500306b3d7039803a2._comment
new file mode 100644
index 000000000..fb1206a7d
--- /dev/null
+++ b/doc/bugs/WSL_adjusted_braches__58___smudge_fails_with_sqlite_thread_crashed_-_locking_protocol/comment_3_2a9f0396df93d9500306b3d7039803a2._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="ply"
+ avatar="http://cdn.libravatar.org/avatar/1270501a59ed4a4042366b00295fe236"
+ subject="comment 3"
+ date="2019-11-28T11:18:50Z"
+ content="""
+Thanks Joey for investigating this! It looks like I need to wait for WSL 2 to become available in windows public release. In the meantime I've submitted a bug on [faulty behaviour of `git annex add` on DrvFs](https://git-annex.branchable.com/bugs/WSL1__58___git-annex-add_fails_in_DrvFs_filesystem/). I don't think you can fix it, as it is apparantly a WSL problem, but I think it's good to keep track of it and warn potential users
+"""]]

diff --git a/doc/bugs/WSL1__58___git-annex-add_fails_in_DrvFs_filesystem.mdwn b/doc/bugs/WSL1__58___git-annex-add_fails_in_DrvFs_filesystem.mdwn
new file mode 100644
index 000000000..cc0bc621a
--- /dev/null
+++ b/doc/bugs/WSL1__58___git-annex-add_fails_in_DrvFs_filesystem.mdwn
@@ -0,0 +1,123 @@
+### Please describe the problem.
+
+git-annex fails to add file to the repository because of permission problem (probably faulty permission handling in WSL). Interestingly, it is possible to add a file anyway, by executing `git annex add` twice. Unfortunately, files added this way are writeable, when they shouldn't.
+
+It's probably not in the scope of git-annex developing, but I think it's good to keep trace on the problem.
+
+### What steps will reproduce the problem?
+
+```
+cd /mnt/c
+git init test
+cd test
+git annex init test
+init test
+touch file
+git annex add file
+```
+
+### What version of git-annex are you using? On what operating system?
+
+Windows 10 Pro version 1909 build 18363.476 - WSL (Arch)
+
+```
+git-annex version: 7.20191114-ga95efcbc55
+build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
+dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.3 feed-1.2.0.1 ghc-8.6.5 http-client-0.6.4 persistent-sqlite-2.10.5 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
+operating system: linux x86_64
+supported repository versions: 7
+upgrade supported from repository versions: 0 1 2 3 4 5 6
+local repository version: 7
+```
+
+### Please provide any additional information below.
+
+[[!format sh """
+$ git annex init test
+init test
+  Detected a filesystem without fifo support.
+
+  Disabling ssh connection caching.
+(scanning for unlocked files...)
+ok
+(recording state in git...)
+$ touch file
+$ git annex add file --debug
+[2019-11-28 11:52:53.048398] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2019-11-28 11:52:53.0570463] process done ExitSuccess
+[2019-11-28 11:52:53.0573639] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2019-11-28 11:52:53.0656397] process done ExitFailure 1
+[2019-11-28 11:52:53.0660529] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--others","--exclude-standard","-z","--","file"]
+[2019-11-28 11:52:53.0742999] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
+[2019-11-28 11:52:53.0822627] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+[2019-11-28 11:52:53.0853736] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
+add file [2019-11-28 11:52:53.0949002] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2019-11-28 11:52:53.1027361] process done ExitSuccess
+[2019-11-28 11:52:53.1030132] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2019-11-28 11:52:53.1122577] process done ExitFailure 1
+
+[2019-11-28 11:52:53.1232169] call: cp ["--reflink=auto","--preserve=timestamps",".git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","file"]
+[2019-11-28 11:52:53.1606206] process done ExitSuccess
+
+git-annex: .git/annex/othertmp/file.0/file: rename: permission denied (Permission denied)
+failed
+[2019-11-28 11:52:53.1617248] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--modified","-z","--","file"]
+[2019-11-28 11:52:53.1693198] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","diff","--name-only","--diff-filter=T","-z","--cached","--","file"]
+[2019-11-28 11:52:53.1825925] process done ExitSuccess
+[2019-11-28 11:52:53.1835521] process done ExitSuccess
+[2019-11-28 11:52:53.1844047] process done ExitSuccess
+git-annex: add: 1 failed
+"""]]
+
+Second attempt:
+
+[[!format sh """
+$ git annex add file --debug
+[2019-11-28 11:57:56.4029726] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2019-11-28 11:57:56.4114361] process done ExitSuccess
+[2019-11-28 11:57:56.411681] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2019-11-28 11:57:56.4201317] process done ExitFailure 1
+[2019-11-28 11:57:56.420548] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--others","--exclude-standard","-z","--","file"]
+[2019-11-28 11:57:56.4316368] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
+[2019-11-28 11:57:56.4416827] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+[2019-11-28 11:57:56.4452357] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
+add file [2019-11-28 11:57:56.4545013] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2019-11-28 11:57:56.4626846] process done ExitSuccess
+[2019-11-28 11:57:56.4629866] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2019-11-28 11:57:56.4735385] process done ExitFailure 1
+
+[2019-11-28 11:57:56.4848163] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+[2019-11-28 11:57:56.488706] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
+ok
+[2019-11-28 11:57:56.4964438] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--modified","-z","--","file"]
+[2019-11-28 11:57:56.5043041] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","diff","--name-only","--diff-filter=T","-z","--cached","--","file"]
+(recording state in git...)
+[2019-11-28 11:57:56.5152453] feed: xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--"]
+[2019-11-28 11:57:56.5426207] process done ExitSuccess
+[2019-11-28 11:57:56.5438586] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","hash-object","-w","--stdin-paths","--no-filters"]
+[2019-11-28 11:57:56.5478542] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","update-index","-z","--index-info"]
+[2019-11-28 11:57:56.5713] process done ExitSuccess
+[2019-11-28 11:57:56.5716027] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2019-11-28 11:57:56.5803067] process done ExitSuccess
+[2019-11-28 11:57:56.5807703] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","write-tree"]
+[2019-11-28 11:57:56.6111405] process done ExitSuccess
+[2019-11-28 11:57:56.6115303] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","commit-tree","ffa5a12eba0b2ea9bc5b529278597615f70c901c","--no-gpg-sign","-p","refs/heads/git-annex"]
+[2019-11-28 11:57:56.6269742] process done ExitSuccess
+[2019-11-28 11:57:56.6272697] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","update-ref","refs/heads/git-annex","0ece4a3a069693ea12cb61168cfb701040c8a7a7"]
+[2019-11-28 11:57:56.6465065] process done ExitSuccess
+[2019-11-28 11:57:56.6506175] process done ExitSuccess
+[2019-11-28 11:57:56.651426] process done ExitSuccess
+[2019-11-28 11:57:56.6520969] process done ExitSuccess
+[2019-11-28 11:57:56.6527282] process done ExitSuccess
+[2019-11-28 11:57:56.6536136] process done ExitSuccess
+[2019-11-28 11:57:56.6554327] process done ExitSuccess
+$ echo "this should fail" > file
+$ cat file
+this should fail
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes! Thank you very much Joey for your hard work and digging into WSL bugs :)

diff --git a/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file/comment_1_08af564539efb1b0d85905c0aa862c43._comment b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file/comment_1_08af564539efb1b0d85905c0aa862c43._comment
new file mode 100644
index 000000000..53716fecb
--- /dev/null
+++ b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file/comment_1_08af564539efb1b0d85905c0aa862c43._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="related: shouldn't git annex try external remotes to download config?"
+ date="2019-11-28T01:22:53Z"
+ content="""
+I haven't tested, but I can see the situation where a specific repository URL could be handled by external special remote (such as datalad, downloaders of which do handle obscure setups such as this one without 403/404 but rather forwarding to login page) which would provide authenticated access to the URL.  Would annex even try that config URL via external special remotes?
+"""]]

diff --git a/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file.mdwn b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file.mdwn
new file mode 100644
index 000000000..353387703
--- /dev/null
+++ b/doc/bugs/leaks_git_config_error_message_upon_inability_to_read_downloaded___34__config__34___file.mdwn
@@ -0,0 +1,146 @@
+It is not a ground shaking issue, but probably would be best to handle it more gracefully.
+
+Initially mentioned while doing install using datalad. Account/permission is required to access this particular repo, ask Canadians for access if you don't have it yet Joey.  credentials I guess got asked for and cached by git upon initial invocation, so upon subsequent calls didn't ask for any:
+
+[[!format sh """
+$> datalad install https://git.bic.mni.mcgill.ca/bic/Coffey-mri-bids
+[INFO   ] Cloning https://git.bic.mni.mcgill.ca/bic/Coffey-mri-bids [1 other candidates] into '/tmp/Coffey-mri-bids'
+[INFO   ] fatal: bad config line 1 in file /home/yoh/.tmp/git-annex96493-5.tmp
+[INFO   ]   Remote origin not usable by git-annex; setting annex-ignore
+install(ok): /tmp/Coffey-mri-bids (dataset)
+"""]]
+
+which boiled down to that message being spited out during `git annex init` which samples the remote, but fails to download the config and gets instead a redirected html page:
+
+[[!format sh """
+$> git clone https://git.bic.mni.mcgill.ca/bic/Coffey-mri-bids
+Cloning into 'Coffey-mri-bids'...
+warning: redirecting to https://git.bic.mni.mcgill.ca/bic/Coffey-mri-bids.git/
+remote: Enumerating objects: 398, done.
+remote: Counting objects: 100% (398/398), done.
+remote: Compressing objects: 100% (282/282), done.
+remote: Total 398 (delta 53), reused 393 (delta 48)
+Receiving objects: 100% (398/398), 34.97 KiB | 795.00 KiB/s, done.
+Resolving deltas: 100% (53/53), done.
+
+
+$> git -C Coffey-mri-bids annex init --debug
+...
+[2019-11-27 19:27:01.341315979] Request {
+  host                 = "git.bic.mni.mcgill.ca"
+  port                 = 443
+  secure               = True
+  requestHeaders       = [("Accept-Encoding","identity"),("User-Agent","git-annex/7.20190819+git2-g908476a9b-1~ndall+1")]
+  path                 = "/bic/Coffey-mri-bids/config"
+  queryString          = ""
+  method               = "GET"
+  proxy                = Nothing
+  rawBody              = False
+  redirectCount        = 10
+  responseTimeout      = ResponseTimeoutDefault
+  requestVersion       = HTTP/1.1
+}
+
+[2019-11-27 19:27:01.90016181] read: git ["config","--null","--list","--file","/home/yoh/.tmp/git-annex228094-5.tmp"]
+fatal: bad config line 1 in file /home/yoh/.tmp/git-annex228094-5.tmp
+[2019-11-27 19:27:01.913302324] process done ExitFailure 128
+
+  Remote origin not usable by git-annex; setting annex-ignore
+
+$> wget -S https://git.bic.mni.mcgill.ca/bic/Coffey-mri-bids/config
+--2019-11-27 19:29:25--  https://git.bic.mni.mcgill.ca/bic/Coffey-mri-bids/config
+Resolving git.bic.mni.mcgill.ca (git.bic.mni.mcgill.ca)... 132.216.133.92
+Connecting to git.bic.mni.mcgill.ca (git.bic.mni.mcgill.ca)|132.216.133.92|:443... connected.
+HTTP request sent, awaiting response... 
+  HTTP/1.1 302 Found
+  Server: nginx
+  Date: Thu, 28 Nov 2019 00:29:26 GMT
+  Content-Type: text/html; charset=utf-8
+  Content-Length: 109
+  Connection: keep-alive
+  Cache-Control: no-cache
+  Location: https://git.bic.mni.mcgill.ca/users/sign_in
+  Set-Cookie: _gitlab_session=8a4f8d5569636004aaebfb73588a2d53; path=/; secure; HttpOnly
+  X-Request-Id: xTcSyu4H36
+  X-Runtime: 0.071681
+  Strict-Transport-Security: max-age=31536000
+  Referrer-Policy: strict-origin-when-cross-origin
+Location: https://git.bic.mni.mcgill.ca/users/sign_in [following]
+--2019-11-27 19:29:26--  https://git.bic.mni.mcgill.ca/users/sign_in
+Reusing existing connection to git.bic.mni.mcgill.ca:443.
+HTTP request sent, awaiting response... 
+  HTTP/1.1 200 OK
+  Server: nginx
+  Date: Thu, 28 Nov 2019 00:29:26 GMT
+  Content-Type: text/html; charset=utf-8
+  Transfer-Encoding: chunked
+  Connection: keep-alive
+  Vary: Accept-Encoding
+  Cache-Control: max-age=0, private, must-revalidate
+  Etag: W/"305857ff0ba591a1e4ee7fec83b5687c"
+  Referrer-Policy: strict-origin-when-cross-origin
+  Set-Cookie: _gitlab_session=8a4f8d5569636004aaebfb73588a2d53; path=/; expires=Thu, 28 Nov 2019 02:29:26 -0000; secure; HttpOnly
+  X-Content-Type-Options: nosniff
+  X-Download-Options: noopen
+  X-Frame-Options: DENY
+  X-Permitted-Cross-Domain-Policies: none
+  X-Request-Id: MHFi7Yjxe82
+  X-Runtime: 0.063359
+  X-Ua-Compatible: IE=edge
+  X-Xss-Protection: 1; mode=block
+  Strict-Transport-Security: max-age=31536000
+  Referrer-Policy: strict-origin-when-cross-origin
+Length: unspecified [text/html]
+Saving to: ‘config’
+
+config                                                       [ <=>                                                                                                                              ]  13.19K  --.-KB/s    in 0s      
+
+2019-11-27 19:29:26 (89.1 MB/s) - ‘config’ saved [13505]
+
+$> cat config 
+<!DOCTYPE html>
+<html class="devise-layout-html">
+<head prefix="og: http://ogp.me/ns#">
+<meta charset="utf-8">
+<meta content="IE=edge" http-equiv="X-UA-Compatible">
+<meta content="object" property="og:type">
+<meta content="GitLab" property="og:site_name">
+<meta content="Sign in" property="og:title">
+...
+"""]]
+
+I guess the problem is multi-faceted:
+
+1. in case of authenticated http remote, `git` caches credentials, but then `git annex` tries to download file directly (instead of somehow via git), it could not "sense" that remote to be a valid annex and/or get files from it.
+
+You can try with this simple one -- user "demo", password "demo":
+
+[[!format sh """
+$> git clone http://www.onerussian.com/tmp/secret-repo/.git
+Cloning into 'secret-repo'...
+Username for 'http://www.onerussian.com': demo
+Password for 'http://demo@www.onerussian.com': 
+
+$> git -C secret-repo annex init
+init  (merging origin/git-annex into git-annex...)
+(recording state in git...)
+
+  Remote origin not usable by git-annex; setting annex-ignore
+ok
+(recording state in git...)
+
+"""]]
+
+although remote is a proper annex, indeed `git annex` cannot use it since does not authenticate as git does.
+So even though the error message is not incorrect, I would say the situation is suboptimal
+ 
+2. if remote server instead of just returning 404 or 403 error code (as eg github seems to do in similar cases of non-authenticated access) instead redirects to some login page, annex feeds that page as a config to git, ignores the error message and just marks that remote as ignored for annex, while leaking that obscure "fatal" error message from git. 
+
+IMHO, ideally 1. should be addressed properly (authentication), and for 2. annex should spit out some more sensible message ("git failed to parse a config file fetched from the remote X.  Please inspect it at this /path/config"), so keep that file around for debugging.  As it is now I had to dig quite deep to figure out WTF is going on.
+
+git annex 7.20190819+git2-g908476a9b-1~ndall+1 and the same with bleeding edge 7.20191114+git43-ge29663773-1~ndall+1 (probably that commit is the one with my patch for stricter git versioning, so use the count of 42 ;))
+
+[[!meta author=yoh]]
+[[!tag projects/canada]]
+
+

diff --git a/doc/todo/git-annex-cat.mdwn b/doc/todo/git-annex-cat.mdwn
new file mode 100644
index 000000000..ed49cca89
--- /dev/null
+++ b/doc/todo/git-annex-cat.mdwn
@@ -0,0 +1,5 @@
+It would be useful to have a [[`git-annex-cat`|forum/Is_there_a___34__git_annex_cat-file__34___type_command__63__/]] command that outputs the contents of an annexed file without storing it in the annex.  This [[can be faster|OPT: "bundle" get + check (of checksum) in a single operation]] than `git-annex-get` followed by `cat`, even if file is already present.  It avoids some failure modes of `git-annex-get` (like running out of local space, or contending for locks).  It supports a common use case of just needing a file for some operation, without needing to remember to drop it later.  It could be used to implement a web server or FUSE filesystem that serves git-annex repo files on demand.
+
+If file is not present, or `remote.here.cost` is higher than `remote.someremote.cost` where file is present, `someremote` would get a `TRANSFER` request where the `FILE` argument is a named pipe, and a `cat` of that named pipe would be started.
+
+If file is not annexed, for uniformity `git-annex-cat file` would just call `cat file`.

diff --git a/doc/devblog/day_610-611__ByteString_optimisation_early_days/comment_2_21872150f9b2ff9cc08f94d52dbdf3a6._comment b/doc/devblog/day_610-611__ByteString_optimisation_early_days/comment_2_21872150f9b2ff9cc08f94d52dbdf3a6._comment
new file mode 100644
index 000000000..7249b8dc0
--- /dev/null
+++ b/doc/devblog/day_610-611__ByteString_optimisation_early_days/comment_2_21872150f9b2ff9cc08f94d52dbdf3a6._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="parallelization"
+ date="2019-11-27T17:30:12Z"
+ content="""
+This is great.
+
+One other potential for speedup is fixing [[issues with parallel operations|forum/people's_experience_with_parallel_git-annex_operations]].  My current fix is to use `-J1`, giving up a potential 96X speedup.  There may also be additional [[todo/parallel_possibilities]].
+"""]]

diff --git a/doc/todo/parallel_possibilities/comment_1_bad182de605b7b47d66dcfe583acd4f1._comment b/doc/todo/parallel_possibilities/comment_1_bad182de605b7b47d66dcfe583acd4f1._comment
new file mode 100644
index 000000000..36a15aef8
--- /dev/null
+++ b/doc/todo/parallel_possibilities/comment_1_bad182de605b7b47d66dcfe583acd4f1._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="parallelization"
+ date="2019-11-27T17:23:14Z"
+ content="""
+When operating on many files, maybe run N parallel commands where i'th command ignores paths for which `(hash(filename) module N) != i`.   Or, if git index has size I, i'th command ignores paths that are not legixographically between `index[(I/N)*i]` and `index[(I/N)*(i+1)]` (for index state at command start).  Extending [[git-annex-matching-options]] with `--block=i` would let this be done using `xargs`.
+"""]]

diff --git a/doc/todo/optimize_by_converting_String_to_ByteString/comment_2_9c51e1986aeb16b3138b6824be9f5a58._comment b/doc/todo/optimize_by_converting_String_to_ByteString/comment_2_9c51e1986aeb16b3138b6824be9f5a58._comment
new file mode 100644
index 000000000..d3f545d55
--- /dev/null
+++ b/doc/todo/optimize_by_converting_String_to_ByteString/comment_2_9c51e1986aeb16b3138b6824be9f5a58._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="representing paths"
+ date="2019-11-27T15:08:40Z"
+ content="""
+Thanks for working on this Joey.
+
+I don't know Haskell or git-annex architecture, so my thoughts might make no sense, but I'll post just in case.
+
+\"There are likely quite a few places where a value is converted back and forth several times\" -- as a quick/temp fix, could memoization speed this up? Or memoizing the results of some system calls?
+
+The many filenames flying around often share long prefixes.  Could that be used to speed things up?  E.g. if they could be represented as pointers into some compact storage, maybe cache performance would improve.
+
+\"git annex find... files fly by much more snappily\" -- does this mean `git-annex-find` is testing each file individually, as opposed to constructing a SQL query to an indexed db?  Maybe, simpler `git-annex-find` queries that are fully mappable to SQL queries could be special-cased?
+
+Sorry for naive comments, I'll eventually read up on Haskell and make more sense...
+
+"""]]

diff --git a/doc/git-annex-add/comment_7_46fe1a6d38eecb246f5b5659bc0c00c8._comment b/doc/git-annex-add/comment_7_46fe1a6d38eecb246f5b5659bc0c00c8._comment
deleted file mode 100644
index 96515e6e3..000000000
--- a/doc/git-annex-add/comment_7_46fe1a6d38eecb246f5b5659bc0c00c8._comment
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- username="linnearight02@915958f850452a19de84ec14a765402d1f7ecdb0"
- nickname="linnearight02"
- avatar="http://cdn.libravatar.org/avatar/9c146ceff6ab204aa75ec5a686bd6cfb"
- subject="Online Coursework Service"
- date="2019-11-26T11:11:07Z"
- content="""
-Get the best [online coursework service](https://www.allassignmenthelp.com/online-coursework-service.html) by the top Aussie writers at cheap rates. We at [AllAssignmentHelp](https://www.allassignmenthelp.com/) known to provide custom coursework services and unlimited support to the Australian students when they place an order with us. All of our writers are well-qualified and trained professional writers, thus no need to be worried about the quality of the delivered work.
-"""]]

diff --git a/doc/devblog/day_610-611__ByteString_optimisation_early_days/comment_1_1601268fd4dba4df9cc3dc84932914a6._comment b/doc/devblog/day_610-611__ByteString_optimisation_early_days/comment_1_1601268fd4dba4df9cc3dc84932914a6._comment
new file mode 100644
index 000000000..45e0c377e
--- /dev/null
+++ b/doc/devblog/day_610-611__ByteString_optimisation_early_days/comment_1_1601268fd4dba4df9cc3dc84932914a6._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="anarcat"
+ avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
+ subject="amazing!"
+ date="2019-11-26T21:07:32Z"
+ content="""
+66% performance improvements is an amazing number! i take it this will be especially good for repositories with a large number of files? if so this could make my life MUCH better! :)
+
+i wonder if this connects with the [problems gorzen identified in python 3 about POSIX paths](https://changelog.complete.org/archives/10063-the-fundamental-problem-in-python-3)... does Haskell have similar problems with non-unicode filenames?
+
+in any case, I thank you for this awesome work...
+"""]]

diff --git a/doc/devblog/day_610-611__ByteString_optimisation_early_days.mdwn b/doc/devblog/day_610-611__ByteString_optimisation_early_days.mdwn
new file mode 100644
index 000000000..c64240173
--- /dev/null
+++ b/doc/devblog/day_610-611__ByteString_optimisation_early_days.mdwn
@@ -0,0 +1,12 @@
+Two entire days spent making a branch where git-annex uses ByteString
+instead of String, especially for filepaths. I commented out all the
+commands except for find, but it still took thousands of lines of patches
+to get it to compile.
+
+The result: git-annex find is between 28% and 66% faster when using
+ByteString. The files just fly by!
+
+It's going to be a long, long road to finish this, but it's good to have a
+start, and know it will be worth it.
+[[todo/optimize_by_converting_String_to_ByteString]] is the tracking page
+for this going forward.

diff --git a/doc/todo/optimize_by_converting_String_to_ByteString.mdwn b/doc/todo/optimize_by_converting_String_to_ByteString.mdwn
new file mode 100644
index 000000000..13d29603f
--- /dev/null
+++ b/doc/todo/optimize_by_converting_String_to_ByteString.mdwn
@@ -0,0 +1,34 @@
+git-annex uses FilePath (String) extensively. That's a slow data type.
+Converting to ByteString, and RawFilePath, should speed it up
+significantly, according to [[/profiling]].
+
+I've made a test branch, `bs`, to see what kind of performance improvement
+to expect. Most commands don't built yet in that branch, but `git annex
+find` does. Speedups range from 28-66%. The files fly by much more
+snappily.
+
+As well as adding back all the code that was disabled to get it to build,
+the `bs` branch has quite a lot of things still needing work, including:
+
+* Eliminate all the fromRawFilePath, toRawFilePath, encodeBS,
+  decodeBS conversions. Or at least most of them. There are likely
+  quite a few places where a value is converted back and forth several times.
+
+  It would be good to instrument them with Debug.Trace and find out which
+  are the hot ones that get called, and focus on those.
+
+* System.FilePath is not available for RawFilePath, and many of the
+  conversions are to get a FilePath in order to use that library.
+
+  It should be entirely straightforward to make a version of System.FilePath
+  that can operate on RawFilePath, except possibly there could be some
+  complications due to Windows.
+
+* Use versions of IO actions like getFileStatus that take a RawFilePath,
+  avoiding a conversion. Note that these are only available on unix, not
+  windows, so a compatability shim will be needed.
+  (I can't seem to find any library that provides one.)
+
+* Eliminate some Data.ByteString.Lazy.toStrict, which is a slow copy.
+
+* Use ByteString for parsing git config to speed up startup.
diff --git a/doc/todo/optimize_by_converting_String_to_ByteString/comment_1_403601fa8ad6946eca8f598bdc31f2d7._comment b/doc/todo/optimize_by_converting_String_to_ByteString/comment_1_403601fa8ad6946eca8f598bdc31f2d7._comment
new file mode 100644
index 000000000..0d24a70d0
--- /dev/null
+++ b/doc/todo/optimize_by_converting_String_to_ByteString/comment_1_403601fa8ad6946eca8f598bdc31f2d7._comment
@@ -0,0 +1,44 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""profiling"""
+ date="2019-11-26T20:05:28Z"
+ content="""
+Profiling the early version of the `bs` branch. 
+
+		Tue Nov 26 16:05 2019 Time and Allocation Profiling Report  (Final)
+	
+		   git-annex +RTS -p -RTS find
+	
+		total time  =        2.75 secs   (2749 ticks @ 1000 us, 1 processor)
+		total alloc = 1,642,607,120 bytes  (excludes profiling overheads)
+	
+	COST CENTRE                      MODULE                         SRC                                                 %time %alloc
+	
+	inAnnex'.\                       Annex.Content                  Annex/Content.hs:(103,61)-(118,31)                   31.2   46.8
+	keyFile'                         Annex.Locations                Annex/Locations.hs:(567,1)-(577,30)                   5.3    6.2
+	encodeW8                         Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:(189,1)-(191,70)        3.3    4.2
+	>>=.\                            Data.Attoparsec.Internal.Types Data/Attoparsec/Internal/Types.hs:(146,9)-(147,44)    2.9    0.8
+	>>=.\.succ'                      Data.Attoparsec.Internal.Types Data/Attoparsec/Internal/Types.hs:146:13-76           2.6    0.3
+	keyFile'.esc                     Annex.Locations                Annex/Locations.hs:(573,9)-(577,30)                   2.5    5.5
+	parseLinkTarget                  Annex.Link                     Annex/Link.hs:(254,1)-(262,25)                        2.4    4.4
+	getAnnexLinkTarget'.probesymlink Annex.Link                     Annex/Link.hs:78:9-46                                 2.4    2.8
+	w82s                             Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:217:1-15                2.3    6.0
+	keyPath                          Annex.Locations                Annex/Locations.hs:(606,1)-(608,23)                   1.9    4.0
+	parseKeyVariety                  Types.Key                      Types/Key.hs:(323,1)-(371,42)                         1.8    0.0
+	getState                         Annex                          Annex.hs:(251,1)-(254,27)                             1.7    0.4
+	fileKey'.go                      Annex.Locations                Annex/Locations.hs:588:9-55                           1.4    0.8
+	fileKey'                         Annex.Locations                Annex/Locations.hs:(586,1)-(596,41)                   1.4    1.7
+	hashUpdates.\.\.\                Crypto.Hash                    Crypto/Hash.hs:85:48-99                               1.3    0.0
+	parseLinkTargetOrPointer         Annex.Link                     Annex/Link.hs:(239,1)-(243,25)                        1.2    0.1
+	withPtr                          Basement.Block.Base            Basement/Block/Base.hs:(395,1)-(404,31)               1.2    0.6
+	primitive                        Basement.Monad                 Basement/Monad.hs:72:5-18                             1.0    0.1
+	decodeBS'                        Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:151:1-31                1.0    2.8
+	mkKeySerialization               Types.Key                      Types/Key.hs:(115,1)-(117,22)                         0.7    1.1
+	w82c                             Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:211:1-28                0.6    1.1
+
+Comparing with [[/profiling]] results, the alloc is down significantly.
+And the main IO actions are getting a larger share of the runtime.
+
+There is still significantly conversion going on, encodeW8 and w82s and
+decodeBS' and w82c. Likely another 5% or so speedup if that's eliminated.
+"""]]

diff --git a/doc/git-annex-add/comment_7_46fe1a6d38eecb246f5b5659bc0c00c8._comment b/doc/git-annex-add/comment_7_46fe1a6d38eecb246f5b5659bc0c00c8._comment
new file mode 100644
index 000000000..96515e6e3
--- /dev/null
+++ b/doc/git-annex-add/comment_7_46fe1a6d38eecb246f5b5659bc0c00c8._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="linnearight02@915958f850452a19de84ec14a765402d1f7ecdb0"
+ nickname="linnearight02"
+ avatar="http://cdn.libravatar.org/avatar/9c146ceff6ab204aa75ec5a686bd6cfb"
+ subject="Online Coursework Service"
+ date="2019-11-26T11:11:07Z"
+ content="""
+Get the best [online coursework service](https://www.allassignmenthelp.com/online-coursework-service.html) by the top Aussie writers at cheap rates. We at [AllAssignmentHelp](https://www.allassignmenthelp.com/) known to provide custom coursework services and unlimited support to the Australian students when they place an order with us. All of our writers are well-qualified and trained professional writers, thus no need to be worried about the quality of the delivered work.
+"""]]

diff --git a/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_1_29e601ea3ea4f22301c6cf6eed403ba4._comment b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_1_29e601ea3ea4f22301c6cf6eed403ba4._comment
new file mode 100644
index 000000000..7b2bf7b35
--- /dev/null
+++ b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_1_29e601ea3ea4f22301c6cf6eed403ba4._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="use named pipes?"
+ date="2019-11-25T16:45:26Z"
+ content="""
+For external remotes can pass to the `TRANSFER` request, as the `FILE` parameter, a named pipe, and use `tee` to create a separate stream for checksumming.
+
+An external remote could also do its own checksum checking and then set remote.<name>.annex-verify=false.
+Could also make a “wrapper” external remote that delegates all requests to a given external remote but does checksum-checking in parallel with downloading (by creating a named pipe and passing that to the wrapped remote).
+"""]]

diff --git a/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation.mdwn b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation.mdwn
new file mode 100644
index 000000000..a17aa8f02
--- /dev/null
+++ b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation.mdwn
@@ -0,0 +1,14 @@
+In neurophysiology we encounter HUGE files (HDF5 .nwb files).
+Sizes reach hundreds of GBs per file (thus exceeding any possible file system memory cache size).  While operating in the cloud or on a fast connection it is possible to fetch the files with speeds up to 100 MBps. 
+Upon successful download such files are then loaded back by git-annex for the checksum validation, and often at slower speeds (eg <60MBps on EC2 SSD drive).
+So, ironically, it does not just double, but rather nearly triples overall time to obtain a file.
+
+I think ideally, 
+
+- (at minimum) for built-in special remotes (such as web), it would be great if git-annex was check-summing incrementally as data comes in;
+- made it possible to for external special remotes to provide desired checksum on obtained content. First git-annex should of cause inform them on type (backend) of the checksum it is interested in, and may be have some information reported by external remotes on what checksums they support.
+
+If needed example, here is http://datasets.datalad.org/allen-brain-observatory/visual-coding-2p/.git with >50GB files such as ophys_movies/ophys_experiment_576261945.h5 .
+
+[[!meta author=yoh]]
+[[!tag projects/dandi]]

diff --git a/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_3_51ea1a6c7c6c46322975cf051c191887._comment b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_3_51ea1a6c7c6c46322975cf051c191887._comment
new file mode 100644
index 000000000..9f59e5140
--- /dev/null
+++ b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_3_51ea1a6c7c6c46322975cf051c191887._comment
@@ -0,0 +1,126 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="even git mv -f seems to work correctly"
+ date="2019-11-24T17:25:32Z"
+ content="""
+Also, `git mv` seems to reuse the already-smudged object contents of the source file for the target file, so even with `git mv -f` only the checksum gets checked into git:
+
+[[!format sh \"\"\"
++ cat ./test-git-mv
+#!/bin/bash
+
+set -eu -o pipefail -x
+
+cat $0
+
+TEST_DIR=/tmp/test_dir
+mkdir -p $TEST_DIR
+chmod -R u+w $TEST_DIR
+rm -rf $TEST_DIR
+mkdir -p $TEST_DIR
+pushd $TEST_DIR
+
+git init
+git annex init
+
+git --version
+git annex version
+
+rm .git/info/attributes
+echo foo > foo
+echo bar > bar
+git annex add foo bar
+git check-attr -a foo
+git check-attr -a bar
+echo 'bar filter=annex' > .gitattributes
+git add .gitattributes
+git check-attr -a foo
+git check-attr -a bar
+
+git annex unlock bar
+git mv bar foo  || true
+git mv -f bar foo
+git commit -m add
+git log -p
+
+
++ TEST_DIR=/tmp/test_dir
++ mkdir -p /tmp/test_dir
++ chmod -R u+w /tmp/test_dir
++ rm -rf /tmp/test_dir
++ mkdir -p /tmp/test_dir
++ pushd /tmp/test_dir
+/tmp/test_dir /tmp
++ git init
+Initialized empty Git repository in /tmp/test_dir/.git/
++ git annex init
+init  (scanning for unlocked files...)
+ok
+(recording state in git...)
++ git --version
+git version 2.20.1
++ git annex version
+git-annex version: 7.20191024-g6dc2272
+build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
+dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.1.0 ghc-8.6.5 http-client-0.5.14 persistent-sqlite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
+operating system: linux x86_64
+supported repository versions: 7
+upgrade supported from repository versions: 0 1 2 3 4 5 6
+local repository version: 7
++ rm .git/info/attributes
++ echo foo
++ echo bar
++ git annex add foo bar
+add foo ok
+add bar ok
+(recording state in git...)
++ git check-attr -a foo
++ git check-attr -a bar
++ echo 'bar filter=annex'
++ git add .gitattributes
++ git check-attr -a foo
++ git check-attr -a bar
+bar: filter: annex
++ git annex unlock bar
+unlock bar ok
+(recording state in git...)
++ git mv bar foo
+fatal: destination exists, source=bar, destination=foo
++ true
++ git mv -f bar foo
++ git commit -m add
+[master (root-commit) 8610c0d] add
+ 2 files changed, 2 insertions(+)
+ create mode 100644 .gitattributes
+ create mode 100644 foo
++ git log -p
+commit 8610c0d8f327140608e71dc229f167731552d284
+Author: Ilya Shlyakhter <ilya_shl@alum.mit.edu>
+Date:   Sun Nov 24 12:24:28 2019 -0500
+
+    add
+
+diff --git a/.gitattributes b/.gitattributes
+new file mode 100644
+index 0000000..649f07e
+--- /dev/null
++++ b/.gitattributes
+@@ -0,0 +1 @@
++bar filter=annex
+diff --git a/foo b/foo
+new file mode 100644
+index 0000000..266ae50
+--- /dev/null
++++ b/foo
+@@ -0,0 +1 @@
++/annex/objects/MD5E-s4--c157a79031e1c40f85931829bc5fc552
+
+\"\"\"]]
+
+
+
+
+"""]]

diff --git a/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_2_8ddf77de6313df0157de8d24c2dc7951._comment b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_2_8ddf77de6313df0157de8d24c2dc7951._comment
new file mode 100644
index 000000000..dd8ce6fc4
--- /dev/null
+++ b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_2_8ddf77de6313df0157de8d24c2dc7951._comment
@@ -0,0 +1,46 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="moving unlocked file onto locked file isn't possible"
+ date="2019-11-24T16:36:24Z"
+ content="""
+`git mv` won't move an unlocked file onto a locked file (trace below).
+
+\"The right solution is to improve the smudge/clean filter interface\" -- of course, but realistically, do you think git devs can be persuaded to do [[this|todo/git_smudge_clean_interface_suboptiomal]] sometime soon?  Even if yes, it still seems better to avoid adding a step to common git workflows, than to make the step fast.
+
+
+[[!format sh \"\"\"
+(master_env_v164_py36) 11:14  [t1] $ ls
+bar  foo
+(master_env_v164_py36) 11:14  [t1] $ git init
+Initialized empty Git repository in /tmp/t1/.git/
+(master_env_v164_py36) 11:14  [t1] $ git annex init
+init  (scanning for unlocked files...)
+ok
+(recording state in git...)
+(master_env_v164_py36) 11:14  [t1] $ git annex add foo
+add foo ok
+(recording state in git...)
+(master_env_v164_py36) 11:14  [t1] $ git annex add bar
+add bar ok
+(recording state in git...)
+(master_env_v164_py36) 11:14  [t1] $ ls -alt
+total 0
+drwxrwxr-x  8 ilya ilya 141 Nov 24 11:14 .git
+drwxrwxr-x  3 ilya ilya  40 Nov 24 11:14 .
+lrwxrwxrwx  1 ilya ilya 108 Nov 24 11:14 bar -> .git/annex/objects/jx/MV/MD5E-s4--c157a79031e1c40f85931829bc5fc552/MD5E-s4--c157a79031\
+e1c40f85931829bc5fc552
+lrwxrwxrwx  1 ilya ilya 108 Nov 24 11:14 foo -> .git/annex/objects/00/zZ/MD5E-s4--d3b07384d113edec49eaa6238ad5ff00/MD5E-s4--d3b07384d1\
+13edec49eaa6238ad5ff00
+drwxrwxrwt 12 root root 282 Nov 24 11:14 ..
+(master_env_v164_py36) 11:14  [t1] $ git annex unlock bar
+unlock bar ok
+(recording state in git...)
+(master_env_v164_py36) 11:16  [t1] $ git mv bar foo
+fatal: destination exists, source=bar, destination=foo
+(master_env_v164_py36) 11:17  [t1] $
+
+
+
+\"\"\"]]
+"""]]

diff --git a/CHANGELOG b/CHANGELOG
index f9ec53295..c762e7ec6 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,6 +1,6 @@
 git-annex (7.20191115) UNRELEASED; urgency=medium
 
-  * Sped up many git-annex commands that operare on many files, by 
+  * Sped up many git-annex commands that operate on many files, by 
     avoiding reserialization of keys.
     find is 7% faster; whereis is 3% faster; and git-annex get when
     all files are already present is 5% faster
diff --git a/doc/devblog/day_609__optimisation.mdwn b/doc/devblog/day_609__optimisation.mdwn
new file mode 100644
index 000000000..d89fd6617
--- /dev/null
+++ b/doc/devblog/day_609__optimisation.mdwn
@@ -0,0 +1,11 @@
+Today, sped up many git-annex commands by around 5%. Often git-annex
+traverses the work tree and deserializes keys to its Key data type, only to
+turn around and do something with a Key that needs it to be serialized
+again. So caching the original serialization of a key avoids that work. I
+had started on this in January but had to throw my first attempt away.
+
+The big bytestring conversion in January only yielded a 5-15% speedup,
+so an extra 5% is a nice bonus for so relativly little work today.
+It also feels like this optimisation approach is nearly paid out though;
+only converting all filepath operations to bytestrings seems likely to
+yield a similar widespread improvement.

diff --git a/doc/todo/git_status_smudges_unncessarily_after_unlock.mdwn b/doc/todo/git_status_smudges_unncessarily_after_unlock.mdwn
new file mode 100644
index 000000000..b1b59f086
--- /dev/null
+++ b/doc/todo/git_status_smudges_unncessarily_after_unlock.mdwn
@@ -0,0 +1,11 @@
+After unlocking a file, `git status` runs the smudge filter. That is
+unnecessary, and when many files were unlocked, it can take a long time
+because [[git_smudge_clean_interface_suboptiomal]] means it runs git-annex
+once per file.
+
+It should be possible to avoid that, as was done with git drop in [[!commit
+1113caa53efedbe7ab1d98b74010160f20473e8d]]. I tried making Command.Unlock
+use restagePointerFile, but that did not help; git update-index does then
+smudge it during the `git annex unlock`, which is no faster (but at least
+doing it then would avoid the surprise of a slow `git status` or `git
+commit -a`). Afterwards, `git status` then smudged it again, unsure why!

diff --git a/doc/profiling/comment_6_ca4ac016a0fb0132fc5c746dfb6fefb3._comment b/doc/profiling/comment_6_ca4ac016a0fb0132fc5c746dfb6fefb3._comment
new file mode 100644
index 000000000..7f8d3a257
--- /dev/null
+++ b/doc/profiling/comment_6_ca4ac016a0fb0132fc5c746dfb6fefb3._comment
@@ -0,0 +1,70 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 6"""
+ date="2019-11-22T22:18:02Z"
+ content="""
+After caching serialized Keys.
+
+		Fri Nov 22 19:06 2019 Time and Allocation Profiling Report  (Final)
+	
+		   git-annex +RTS -p -RTS find
+	
+		total time  =        3.00 secs   (2997 ticks @ 1000 us, 1 processor)
+		total alloc = 1,890,060,432 bytes  (excludes profiling overheads)
+	
+	COST CENTRE                      MODULE                         SRC                                                 %time %alloc
+	
+	inAnnex'.\                       Annex.Content                  Annex/Content.hs:(103,61)-(118,31)                   28.2   40.6
+	keyFile'                         Annex.Locations                Annex/Locations.hs:(564,1)-(574,30)                   3.8    5.3
+	splitc                           Utility.Split                  Utility/Split.hs:(24,1)-(26,25)                       3.7    5.1
+	_encodeFilePath                  Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:(111,1)-(114,49)        3.1    2.7
+	encodeW8                         Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:(189,1)-(191,70)        3.0    3.6
+	w82s                             Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:217:1-15                2.8    5.1
+	>>=.\.succ'                      Data.Attoparsec.Internal.Types Data/Attoparsec/Internal/Types.hs:146:13-76           2.8    0.2
+	>>=.\                            Data.Attoparsec.Internal.Types Data/Attoparsec/Internal/Types.hs:(146,9)-(147,44)    2.5    0.7
+	getAnnexLinkTarget'.probesymlink Annex.Link                     Annex/Link.hs:78:9-62                                 2.4    2.5
+	fileKey'                         Annex.Locations                Annex/Locations.hs:(583,1)-(593,41)                   2.1    1.5
+	parseLinkTarget                  Annex.Link                     Annex/Link.hs:(254,1)-(262,25)                        2.0    3.8
+	keyFile'.esc                     Annex.Locations                Annex/Locations.hs:(570,9)-(574,30)                   1.9    4.7
+	s2w8                             Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:214:1-15                1.7    3.5
+	keyPath                          Annex.Locations                Annex/Locations.hs:(603,1)-(605,23)                   1.6    3.5
+	getState                         Annex                          Annex.hs:(251,1)-(254,27)                             1.6    0.3
+	parseKeyVariety                  Types.Key                      Types/Key.hs:(322,1)-(370,42)                         1.4    0.0
+	withMutablePtrHint               Basement.Block.Base            Basement/Block/Base.hs:(468,1)-(482,50)               1.3    0.4
+	hashUpdates.\.\.\                Crypto.Hash                    Crypto/Hash.hs:85:48-99                               1.2    0.0
+	parseLinkTargetOrPointer         Annex.Link                     Annex/Link.hs:(239,1)-(243,25)                        1.1    0.1
+	primitive                        Basement.Monad                 Basement/Monad.hs:72:5-18                             1.1    0.0
+	withPtr.makeTrampoline           Basement.Block.Base            Basement/Block/Base.hs:(401,5)-(404,31)               1.0    0.6
+	assertLocal                      Git                            Git.hs:(123,1)-(129,28)                               0.8    1.6
+	decodeBS'                        Utility.FileSystemEncoding     Utility/FileSystemEncoding.hs:151:1-31                0.4    2.4
+
+Runtime improved by 5% or so, and getAnnexLinkTarget moved up, otherwise
+not a lot of change. keyFile is looking like an optimization target,
+although its percent of the runtime actually reduced. 
+However that's specific to this repo which has a lot of URL keys that
+contain '/' and so need to be escaped.
+
+		Fri Nov 22 19:09 2019 Time and Allocation Profiling Report  (Final)
+	
+		   git-annex +RTS -p -RTS find --not --in web
+	
+		total time  =        8.42 secs   (8421 ticks @ 1000 us, 1 processor)
+		total alloc = 1,887,547,744 bytes  (excludes profiling overheads)
+	
+	COST CENTRE                      MODULE                           SRC                                                 %time %alloc
+	
+	catObjectDetails.\               Git.CatFile                      Git/CatFile.hs:(83,88)-(91,97)                        8.1    4.1
+	catchMaybeIO                     Utility.Exception                Utility/Exception.hs:53:1-63                          7.6    2.2
+	parseResp                        Git.CatFile                      Git/CatFile.hs:(145,1)-(156,28)                       5.2    5.6
+	>>=.\                            Data.Attoparsec.Internal.Types   Data/Attoparsec/Internal/Types.hs:(146,9)-(147,44)    5.0    2.0
+	>>=.\.succ'                      Data.Attoparsec.Internal.Types   Data/Attoparsec/Internal/Types.hs:146:13-76           4.7    1.0
+	MAIN                             MAIN                             <built-in>                                            4.2    0.4
+	getState                         Annex                            Annex.hs:(251,1)-(254,27)                             2.0    1.1
+	getAnnexLinkTarget'.probesymlink Annex.Link                       Annex/Link.hs:78:9-62                                 1.9    2.6
+	splitc                           Utility.Split                    Utility/Split.hs:(24,1)-(26,25)                       1.9    5.2
+	_encodeFilePath                  Utility.FileSystemEncoding       Utility/FileSystemEncoding.hs:(111,1)-(114,49)        1.8    2.8
+	query.send                       Git.CatFile                      Git/CatFile.hs:141:9-32                               1.7    0.5
+	keyFile'                         Annex.Locations                  Annex/Locations.hs:(564,1)-(574,30)                   1.6    5.4
+
+Ditto.
+"""]]

diff --git a/doc/git-annex-benchmark.mdwn b/doc/git-annex-benchmark.mdwn
index c81839f5d..73bb01eb8 100644
--- a/doc/git-annex-benchmark.mdwn
+++ b/doc/git-annex-benchmark.mdwn
@@ -45,13 +45,14 @@ instead of a command. N is the number of items to benchmark.
 # OUTPUT
 
 The output of the commands being benchmarked goes to standard output and
-standard error as usual. It's often a good idea to sink it to /dev/null to
-avoid the display of the output skewing the benchmark results. Of course
---quiet can also be used to avoid most git-annex output, as long as you
-don't want to benchmark the generation of that output.
+standard error as usual. It's often a good idea to use --quiet to avoid
+unncessary output, unless the generation of that output is part of what
+you want to benchmark.
 
 The benchmark report is output to standard output by default, although
-criterion options can be used to redirect it to a file.
+criterion options can be used to redirect it to a file. For example:
+
+	git annex benchmark -o bench -- find >/dev/null
 
 # SEE ALSO
 

diff --git a/doc/todo/split_off_clean__47__smudge_filter__63__.mdwn b/doc/todo/split_off_clean__47__smudge_filter__63__.mdwn
index 161cdb577..c568d9b46 100644
--- a/doc/todo/split_off_clean__47__smudge_filter__63__.mdwn
+++ b/doc/todo/split_off_clean__47__smudge_filter__63__.mdwn
@@ -1 +1,5 @@
 If running the clean/smudge filter once per file is a [[bottleneck|forum/Adding_files_to_git__58___Very_long___34__recording_state_in_git__34___phase]], might it speed things up to split them off into something more lightweight than the full git-annex binary?
+
+> This is a duplicate of stuff discussed at
+> [[todo/git_smudge_clean_interface_suboptiomal]], so closing. [[done]]
+> --[[Joey]]

diff --git a/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter.mdwn b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter.mdwn
index 764be9e20..873edf8c6 100644
--- a/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter.mdwn
+++ b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter.mdwn
@@ -1 +1,3 @@
 Right now, non-annexed files get passed through the `annex` clean/smudge filter (see [[forum/Adding_files_to_git__58___Very_long___34__recording_state_in_git__34___phase]]).  It would be better if `git-annex` configure the filter only for the annexed unlocked files, in the `.gitattributes` file at the root of the repository.
+
+> not a viable solution, [[done]] --[[Joey]]
diff --git a/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_1_c614ab0aeab4a8d27d3a1da3db3c7e05._comment b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_1_c614ab0aeab4a8d27d3a1da3db3c7e05._comment
new file mode 100644
index 000000000..4996e933a
--- /dev/null
+++ b/doc/todo/only_pass_unlocked_files_through_the_clean__47__smudge_filter/comment_1_c614ab0aeab4a8d27d3a1da3db3c7e05._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2019-11-22T16:01:26Z"
+ content="""
+It immediately occurs to me that the proposal would break this:
+
+	git annex add foo
+	git annex add bar
+	git annex unlock bar
+	git mv bar foo
+	git commit -m add
+
+Since foo was a locked file, gitattributes would prevent from being
+smudged, so the large content that was in bar gets committed directly to git.
+
+The right solution is to improve the smudge/clean filter interface to it's
+not so slow, which there is copious discussion of elsewhere.
+"""]]

diff --git a/CHANGELOG b/CHANGELOG
index 4cb6c989a..f33e291be 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -10,6 +10,11 @@ git-annex (7.20191115) UNRELEASED; urgency=medium
   * sync, assistant: Pull and push from git-lfs remotes.
   * Fix bug that made bare repos be treated as non-bare when --git-dir
     was used.
+  * benchmark: Changed --databases to take a parameter specifiying the size
+    of the database to benchmark.
+  * benchmark --databases: Display size of the populated database.
+  * benchmark --databases: Improve the "addAssociatedFile to (new)"
+    benchmark to really add new values, not overwriting old values.
 
  -- Joey Hess <id@joeyh.name>  Fri, 15 Nov 2019 11:57:19 -0400
 
diff --git a/Command/Benchmark.hs b/Command/Benchmark.hs
index 7ecbf338d..0395383ff 100644
--- a/Command/Benchmark.hs
+++ b/Command/Benchmark.hs
@@ -26,7 +26,7 @@ cmd generator = command "benchmark" SectionTesting
 
 data BenchmarkOptions 
 	= BenchmarkOptions CmdParams CriterionMode
-	| BenchmarkDatabases CriterionMode
+	| BenchmarkDatabases CriterionMode Integer
 
 optParser :: CmdParamsDesc -> Parser BenchmarkOptions
 optParser desc = benchmarkoptions <|> benchmarkdatabases
@@ -36,10 +36,11 @@ optParser desc = benchmarkoptions <|> benchmarkdatabases
 		<*> criterionopts
 	benchmarkdatabases = BenchmarkDatabases
 		<$> criterionopts
-		<* flag' () 
-			( long "databases"
+		<*> option auto
+	                ( long "databases" 
+			<> metavar paramNumber
 			<> help "benchmark sqlite databases"
-			)
+                	)
 #ifdef WITH_BENCHMARK
 	criterionopts = parseWith defaultConfig
 #else
@@ -51,7 +52,7 @@ seek :: BenchmarkGenerator -> BenchmarkOptions -> CommandSeek
 seek generator (BenchmarkOptions ps mode) = do
 	runner <- generator ps
 	liftIO $ runMode mode [ bench (unwords ps) $ nfIO runner ]
-seek _ (BenchmarkDatabases mode) = benchmarkDbs mode
+seek _ (BenchmarkDatabases mode n) = benchmarkDbs mode n
 #else
 seek _ _ = giveup "git-annex is not built with benchmarking support"
 #endif
diff --git a/Database/Benchmark.hs b/Database/Benchmark.hs
index 7f0929757..da2a69339 100644
--- a/Database/Benchmark.hs
+++ b/Database/Benchmark.hs
@@ -20,6 +20,7 @@ import Database.Types
 import Utility.Tmp.Dir
 import Git.FilePath
 import Types.Key
+import Utility.DataUnits
 
 import Criterion.Main
 import Control.Monad.IO.Class (liftIO)
@@ -27,17 +28,12 @@ import qualified Data.ByteString.Char8 as B8
 import System.Random
 #endif
 
-benchmarkDbs :: CriterionMode -> Annex ()
+benchmarkDbs :: CriterionMode -> Integer -> Annex ()
 #ifdef WITH_BENCHMARK
-benchmarkDbs mode = withTmpDirIn "." "benchmark" $ \tmpdir -> do
-	-- benchmark different sizes of databases
-	dbs <- mapM (benchDb tmpdir)
-		[ 1000
-		, 10000
-		-- , 100000
-		]
+benchmarkDbs mode n = withTmpDirIn "." "benchmark" $ \tmpdir -> do
+	db <- benchDb tmpdir n
 	liftIO $ runMode mode
-		[ bgroup "keys database" $ flip concatMap dbs $ \db ->
+		[ bgroup "keys database"
 			[ getAssociatedFilesHitBench db
 			, getAssociatedFilesMissBench db
 			, getAssociatedKeyHitBench db
@@ -81,22 +77,22 @@ addAssociatedFileOldBench (BenchDb h num) = bench ("addAssociatedFile to " ++ sh
 addAssociatedFileNewBench :: BenchDb -> Benchmark
 addAssociatedFileNewBench (BenchDb h num) = bench ("addAssociatedFile to " ++ show num ++ " (new)") $ nfIO $ do
 	n <- getStdRandom (randomR (1,num))
-	SQL.addAssociatedFile (toIKey (keyN n)) (fileN (n+1)) (SQL.WriteHandle h)
+	SQL.addAssociatedFile (toIKey (keyN n)) (fileN (num+n)) (SQL.WriteHandle h)
 	H.flushDbQueue h
 
-populateAssociatedFiles :: H.DbQueue -> Int -> IO ()
+populateAssociatedFiles :: H.DbQueue -> Integer -> IO ()
 populateAssociatedFiles h num = do
 	forM_ [1..num] $ \n ->
 		SQL.addAssociatedFile (toIKey (keyN n)) (fileN n) (SQL.WriteHandle h)
 	H.flushDbQueue h
 
-keyN :: Int -> Key
+keyN :: Integer -> Key
 keyN n = stubKey
 	{ keyName = B8.pack $ "key" ++ show n
 	, keyVariety = OtherKey "BENCH"
 	}
 
-fileN :: Int -> TopFilePath
+fileN :: Integer -> TopFilePath
 fileN n = asTopFilePath ("file" ++ show n)
 
 keyMiss :: Key
@@ -105,14 +101,17 @@ keyMiss = keyN 0 -- 0 is never stored
 fileMiss :: TopFilePath
 fileMiss = fileN 0 -- 0 is never stored
 
-data BenchDb = BenchDb H.DbQueue Int
+data BenchDb = BenchDb H.DbQueue Integer
 
-benchDb :: FilePath -> Int -> Annex BenchDb
+benchDb :: FilePath -> Integer -> Annex BenchDb
 benchDb tmpdir num = do
-	liftIO $ putStrLn $ "setting up database with " ++ show num
+	liftIO $ putStrLn $ "setting up database with " ++ show num ++ " items"
 	initDb db SQL.createTables
 	h <- liftIO $ H.openDbQueue H.MultiWriter db SQL.containedTable
 	liftIO $ populateAssociatedFiles h num
+	sz <- liftIO $ getFileSize db
+	liftIO $ putStrLn $ "size of database on disk: " ++ 
+		roughSize storageUnits False sz
 	return (BenchDb h num)
   where
 	db = tmpdir </> show num </> "db"
diff --git a/doc/git-annex-benchmark.mdwn b/doc/git-annex-benchmark.mdwn
index 66f9c6525..c81839f5d 100644
--- a/doc/git-annex-benchmark.mdwn
+++ b/doc/git-annex-benchmark.mdwn
@@ -4,7 +4,7 @@ git-annex benchmark - benchmark git-annex commands
 
 # SYNOPSIS
 
-git annex benchmark [criterionopts] ( -- commmand [; command] | --databases )
+git annex benchmark [criterionopts] ( -- commmand [; command] | --databases=N )
 
 # DESCRIPTION
 
@@ -39,8 +39,8 @@ used.
 Any options that git-annex usually accepts can be included after the
 command to benchmark.
 
-The --databases option benchmark's git-annex's use of sqlite databases,
-instead of a command.
+The --databases=N option benchmark's git-annex's use of sqlite databases,
+instead of a command. N is the number of items to benchmark.
 
 # OUTPUT
 

diff --git a/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_14_2274479a1eef9ffc6a53ffb65e2c3511._comment b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_14_2274479a1eef9ffc6a53ffb65e2c3511._comment
new file mode 100644
index 000000000..7249adc93
--- /dev/null
+++ b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_14_2274479a1eef9ffc6a53ffb65e2c3511._comment
@@ -0,0 +1,52 @@
+[[!comment format=mdwn
+ username="xwvvvvwx"
+ avatar="http://cdn.libravatar.org/avatar/7198160b33539b5b1b2d56ca85c562d9"
+ subject="comment 14"
+ date="2019-11-21T17:32:31Z"
+ content="""
+I just reproduced this when pushing to a gcrypt remote on rsync.net using the assistant. There is only one client pushing to the gcrypt remote.
+
+It was during the initial sync of a moderately large amount of data (~22G), perhaps this has something to do with it?
+
+I could reproduce the issue by cloning with gcrypt directly (`git clone gcrypt::ssh://....`).
+
+I was able to recover by following the steps outlined in Schnouki's comment (#12), but this is obviously quite an unsatisfactory fix. 
+
+I am using annex to replicate important personal data, and I find this issue highly concerning.
+
+Foolishly, I did not keep a copy of the bad repo before I forced pushed over it on the remote, so I do not have a copy available to experiment with :(
+
+---
+
+## logs
+
+`daemon.log` excerpt: [https://ipfs.io/ipfs/QmcoPuTLY2v5FWPABQLVwgyqW5WdsvkBbVS33cJh6zjzi4](https://ipfs.io/ipfs/QmcoPuTLY2v5FWPABQLVwgyqW5WdsvkBbVS33cJh6zjzi4)
+
+
+`git clone` output:
+
+```
+[annex@xwvvvvwx:~]$ git clone gcrypt::ssh://<URL> remote
+Cloning into 'remote'...
+gcrypt: Decrypting manifest
+gpg: Signature made Thu 21 Nov 2019 04:02:40 PM CET
+gpg:                using RSA key 92E9F58E9F8C6845423C251AACD9A98951774194
+gpg: Good signature from \"git-annex <annex@xwvvvvwx.com>\" [ultimate]
+gcrypt: Remote ID is :id:tWrcOFKu2yX7y+jLDLxm
+gcrypt: Packfile e7b619864585f3c921b491fd041127cf0ae33c4480810610dcb2e37ec46a82be does not match digest!
+fatal: early EOF
+```
+
+`git annex version`:
+
+```
+git-annex version: 7.20191114
+build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
+dependency versions: aws-0.21.1 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.2.0.1 ghc-8.6.5 http-client-0.6.4 persistent-sqlite-2.9.3 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
+operating system: linux x86_64
+supported repository versions: 7
+upgrade supported from repository versions: 0 1 2 3 4 5 6
+```
+"""]]

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_4_379ff2e840514a3b4f4a426e2aae4a9a._comment b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_4_379ff2e840514a3b4f4a426e2aae4a9a._comment
deleted file mode 100644
index 95c51e5c9..000000000
--- a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_4_379ff2e840514a3b4f4a426e2aae4a9a._comment
+++ /dev/null
@@ -1,24 +0,0 @@
-[[!comment format=mdwn
- username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
- nickname="git-annex"
- avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
- subject="git add or git commit does not trigger assistant, but git rm does"
- date="2019-11-20T01:37:41Z"
- content="""
-More diagnostics on this one..
-
-I create a file that is below the size threshold for annex, but I do have a gitlab upstream.
-
-1) echo \"foo\" > foo  -  Assistant (or webapp) does not fire
-2) git add foo -  Assistant/webapp does not fire
-3) git commit -am \"random testing\"  - assistant does not fire
-4) git annex sync - foo ends up in gitlab
-
-I remove a file
-
-1) git rm foo  - Assistant fires and file is removed from gitlab
-
-Is this the expected behavior?  If so, why does a git rm trigger annex assistant/webapp, but a git add does not? 
-
-
-"""]]

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_4_379ff2e840514a3b4f4a426e2aae4a9a._comment b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_4_379ff2e840514a3b4f4a426e2aae4a9a._comment
new file mode 100644
index 000000000..95c51e5c9
--- /dev/null
+++ b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_4_379ff2e840514a3b4f4a426e2aae4a9a._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
+ nickname="git-annex"
+ avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
+ subject="git add or git commit does not trigger assistant, but git rm does"
+ date="2019-11-20T01:37:41Z"
+ content="""
+More diagnostics on this one..
+
+I create a file that is below the size threshold for annex, but I do have a gitlab upstream.
+
+1) echo \"foo\" > foo  -  Assistant (or webapp) does not fire
+2) git add foo -  Assistant/webapp does not fire
+3) git commit -am \"random testing\"  - assistant does not fire
+4) git annex sync - foo ends up in gitlab
+
+I remove a file
+
+1) git rm foo  - Assistant fires and file is removed from gitlab
+
+Is this the expected behavior?  If so, why does a git rm trigger annex assistant/webapp, but a git add does not? 
+
+
+"""]]

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_3_1556ebbb2a8db4aa859e3852160df044._comment b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_3_1556ebbb2a8db4aa859e3852160df044._comment
new file mode 100644
index 000000000..65d4bf485
--- /dev/null
+++ b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_3_1556ebbb2a8db4aa859e3852160df044._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
+ nickname="git-annex"
+ avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
+ subject="git add or git commit does not trigger assistant, but git rm does"
+ date="2019-11-20T01:37:15Z"
+ content="""
+More diagnostics on this one..
+
+I create a file that is below the size threshold for annex, but I do have a gitlab upstream.
+
+1) echo \"foo\" > foo  -  Assistant (or webapp) does not fire
+2) git add foo -  Assistant/webapp does not fire
+3) git commit -am \"random testing\"  - assistant does not fire
+4) git annex sync - foo ends up in gitlab
+
+I remove a file
+
+1) git rm foo  - Assistant fires and file is removed from gitlab
+
+Is this the expected behavior?  If so, why does a git rm trigger annex assistant/webapp, but a git add does not? 
+
+
+"""]]

diff --git a/doc/todo/remove_legacy_import_directory_interface.mdwn b/doc/todo/remove_legacy_import_directory_interface.mdwn
new file mode 100644
index 000000000..d8f8a316c
--- /dev/null
+++ b/doc/todo/remove_legacy_import_directory_interface.mdwn
@@ -0,0 +1,47 @@
+The old `git annex import /dir` interface should be removed, in favor of
+the new import from special remote interface, which can be used with a
+directory special remote to do the same kind of operation.
+
+There have always been complaints about the old interface being surprising
+and/or not doing quite what some users want.
+Tried to find a principled way to address some of that with the "duplicate"
+options, but users just complain they're confusing (which they certianly
+are) and don't quite do what they want.
+
+The fundamental mistake that the old interface made is it conflated
+copying content into the repository, deleting content from the directory, 
+and updating the working tree. The new interface decouples all 3,
+only doing the first, and updating a tracking branch. The user is then free
+to merge the tracking branch as-is, or otherwise modify before merging.
+There are some options to manipulate the tracking branch in commonly
+wanted ways, which just boil down do git branch manipulation. Less common
+desires can be handled using all of git's facilities. As for deleting from
+the directory, that's an export of a branch, which can just be an empty
+branch if they want to delete everything, or again they can use all of git
+to construct the branch with the changes they desire.
+
+So while it's not been used as much as the old interface, I think the new
+interface will be much more flexible to the varied needs of users. What's
+less clear is if it can well support every way that the old interface can
+be used.
+
+Of course the first pain point is that the user has to set up a directory
+special remote. Which may be annoying if they are importing from a variety
+of different directories ad-hoc.
+
+Another likely pain point is ad-hoc importing of individual files or
+files matched by wildcard. The new interface is much more about importing
+whole trees, perhaps configured by preferred content settings
+
+One approach would be to make the old interface be implemented using the
+new interface, and paper over the cracks, by eg setting up a directory
+special remote automatically.
+
+Or, the old interface could warn when used, linking to some documentation
+about how to accomplish the same tasks with the new interface.
+
+Either could be done incrementally, eg start with the most common import
+cases, convert to the new interface, and keep others using the old
+interface.
+
+--[[Joey]]

diff --git a/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_3_ee247d9316df963fd584df2199b80fe2._comment b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_3_ee247d9316df963fd584df2199b80fe2._comment
new file mode 100644
index 000000000..d272eb88a
--- /dev/null
+++ b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_3_ee247d9316df963fd584df2199b80fe2._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="using hardlinks"
+ date="2019-11-19T17:51:08Z"
+ content="""
+I again don't have a full answer, but maybe you could customize git's [post-checkout hook](https://git-scm.com/docs/githooks#_post_checkout)?  (You'd need to still call the hook that git-annex installs.)
+
+Also, I've thrown together a [[FUSE filesystem|https://github.com/broadinstitute/viral-ngs/blob/is-dx-benchmarks/tools/git-annex-remotes/git-annex-on-demand.py]] that fetches git-annexed files on demand, maybe that could be adapted.  It only works with locked files and symlinks though.
+
+What is the reason you don't want to use locked files?   You can have different [[worktrees|tips/Using_git-worktree_with_annex]] with symlinks of locked files pointing into the same annex.
+"""]]

diff --git a/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default/comment_2_f2d436822490e74544bf58a4f1c9ee79._comment b/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default/comment_2_f2d436822490e74544bf58a4f1c9ee79._comment
new file mode 100644
index 000000000..c4c86551d
--- /dev/null
+++ b/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default/comment_2_f2d436822490e74544bf58a4f1c9ee79._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="import/export"
+ date="2019-11-19T17:39:47Z"
+ content="""
+Makes sense.  It's certainly better if import/export did complementary things.  Maybe, move the old `git-annex-import` functionality to a new command called `git-annex-ingest`?
+
+\"everything not involving importing from a special remote should be deprecated\" -- i.e. to ingest a directory you'd first create a directory special remote for it, and then `git-annex-import` from that?
+"""]]

diff --git a/Command/Import.hs b/Command/Import.hs
index ca4448808..0488ef4cb 100644
--- a/Command/Import.hs
+++ b/Command/Import.hs
@@ -39,7 +39,7 @@ cmd :: Command
 cmd = notBareRepo $
 	withGlobalOptions [jobsOption, jsonOptions, fileMatchingOptions] $
 		command "import" SectionCommon 
-			"import files from elsewhere into the repository"
+			"add a tree of files to the repository"
 			(paramPaths ++ "|BRANCH[:SUBDIR]")
 			(seek <$$> optParser)
 
diff --git a/doc/git-annex-import.mdwn b/doc/git-annex-import.mdwn
index 89a30e82c..1b7239f09 100644
--- a/doc/git-annex-import.mdwn
+++ b/doc/git-annex-import.mdwn
@@ -1,16 +1,16 @@
 # NAME
 
-git-annex import - add files from a non-versioned directory or a special remote
+git-annex import - add a tree of files to the repository
 
 # SYNOPSIS
 
-git annex import `[path ...]` | git annex import --from remote branch[:subdir]
+git annex import --from remote branch[:subdir] | `[path ...]`
 
 # DESCRIPTION
 
-This command is a way to import files from elsewhere into your git-annex
-repository. It can import files from a directory into your repository, 
-or it can import files from a git-annex special remote.
+This command is a way to import a tree of files from elsewhere into your
+git-annex repository. It can import files from a git-annex special remote,
+or from a directory.
 
 # IMPORTING FROM A SPECIAL REMOTE
 
@@ -30,7 +30,8 @@ remote.
 
 You can only import from special remotes that were configured with
 `importtree=yes` when set up with [[git-annex-initremote]](1). Only some
-kinds of special remotes will let you configure them this way.
+kinds of special remotes will let you configure them this way. A perhaps
+non-exhastive list is the directory, s3, and adb special remotes.
 
 To import from a special remote, you must specify the name of a branch.
 A corresponding remote tracking branch will be updated by `git annex
@@ -88,6 +89,10 @@ things that depend on the key. Preferred content expressions containing
 When run with a path, `git annex import` moves files from somewhere outside
 the git working copy, and adds them to the annex.
 
+This is a legacy interface. It is still supported, but please consider
+switching to importing from a directory special remote instead, using the
+interface documented above.
+
 Individual files to import can be specified. If a directory is specified,
 the entire directory is imported.
   
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 43f656048..b191e3a5b 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -146,10 +146,9 @@ subdirectories).
   
   See [[git-annex-rmurl]](1) for details.
 
-* `import [path ...]`
+* `import --from remote branch[:subdir] | [path ...]`
 
-  Add files from a non-version-controlled directory or a 
-  special remote into the annex.
+  Add a tree of files to the repository.
   
   See [[git-annex-import]](1) for details.
 

diff --git a/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default/comment_1_4eb794daaeef843b104bd480e11f7b42._comment b/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default/comment_1_4eb794daaeef843b104bd480e11f7b42._comment
new file mode 100644
index 000000000..7e1005036
--- /dev/null
+++ b/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default/comment_1_4eb794daaeef843b104bd480e11f7b42._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2019-11-19T17:09:00Z"
+ content="""
+My general feeling about git-annex import is that everything not involving
+importing from a special remote should be deprecated and eventually
+removed.
+
+The --duplicate option probably does what you want, but if the interface is
+going to be changed, such as making that the default, I'd rather the
+interface change move toward the goal of deprecating the old mode.
+
+The fundamental mistake that the legacy interface made is it conflated
+copying content into the repository, dropping content from the directory, 
+and updating the working tree. The new interface decouples all 3,
+only doing the first, and updating a tracking branch, which the user is then
+free to merge as-is, or otherwise modify before merging. Dropping requires
+an export of a new tree, which is the main pain point in emulating
+the old interface, but you happen to not want to drop the content from the
+directory, so that pain point shouldn't affect you.
+"""]]

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_f6196f814ed388d276947ffcc92ff37c._comment b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_f6196f814ed388d276947ffcc92ff37c._comment
new file mode 100644
index 000000000..456146b11
--- /dev/null
+++ b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_f6196f814ed388d276947ffcc92ff37c._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
+ nickname="git-annex"
+ avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
+ subject="comment 2"
+ date="2019-11-19T17:18:18Z"
+ content="""
+bump?  Any thoughts?
+"""]]

diff --git a/doc/bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place/comment_1_d385d0fffdd6ac18f38828f805e4daff._comment b/doc/bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place/comment_1_d385d0fffdd6ac18f38828f805e4daff._comment
new file mode 100644
index 000000000..a34534e6f
--- /dev/null
+++ b/doc/bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place/comment_1_d385d0fffdd6ac18f38828f805e4daff._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2019-11-19T17:12:41Z"
+ content="""
+I think that you can accomplish what you want by making the directory
+you're importing from be a directory special remote with exporttree=yes
+importtree=yes and use the new `git annex import master --from remote`
+
+If that does not do what you want, I'd prefer to look at making it be able
+to do so. I hope to eventually remove the legacy git-annex import from
+directory, since we have this new more general interface.
+"""]]

diff --git a/doc/todo/add_import_--to_command/comment_4_4a30627ac78b32911604c3377b958cd0._comment b/doc/todo/add_import_--to_command/comment_4_4a30627ac78b32911604c3377b958cd0._comment
new file mode 100644
index 000000000..20d03ae2c
--- /dev/null
+++ b/doc/todo/add_import_--to_command/comment_4_4a30627ac78b32911604c3377b958cd0._comment
@@ -0,0 +1,40 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2019-11-19T16:58:06Z"
+ content="""
+Since git-annex does now support importtree from directory special remotes,
+you can almost get what you said you want by:
+
+	git annex initremote usb-drive type=directory directory=/mnt/usb-drive/myfiles \
+		exporttree=yes importtree=yes encryption=none
+
+Then `git annex import master --from usb-drive` will import the files
+into a usb-drive/master branch that you can merge. And you can run it
+repeatedly to import new and changed files from the directory.
+
+So then you have the files sitting in a special remote like you wanted.
+Namely the directory special remote on the USB drive. Only problem is that
+importing the files does also copy them into the git-annex repo. So you'd
+have to drop the files again, assuming you had disk space for them all
+to begin with.
+
+I wonder, if it were possible to import the files without add their content
+to the repo you ran the import from, leaving them on the special remote,
+would that meet your use case? That seems like something it would be
+possible to add.
+
+It would still probably have to copy the file into the local repo, in order
+to hash it, and then just delete the content from the local repo. Of course
+when the file is in a directory on the local system, that's not strictly
+necessary; it could do the hashing of the file in place. But that would
+need an extension to the special remote API to hash a file.
+
+But like I said in my other comment, I'd just clone my git-annex repo onto the
+drive and add the files to the repo there. Avoids all this complication.
+You'd need to provide a good justification for why you can't do that for
+me to pursue this any further.
+
+(As far as adding a --to switch to import, [[transitive_transfers]]
+discusses this kind of thing, and some issues with implementing that.)
+"""]]
diff --git a/doc/todo/git-annex-export_--from_option/comment_1_10f107aa0094d5ee4886878f5b1aaf06._comment b/doc/todo/git-annex-export_--from_option/comment_1_10f107aa0094d5ee4886878f5b1aaf06._comment
new file mode 100644
index 000000000..2f536eacd
--- /dev/null
+++ b/doc/todo/git-annex-export_--from_option/comment_1_10f107aa0094d5ee4886878f5b1aaf06._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2019-11-19T16:52:00Z"
+ content="""
+`git annex export --from` would be basically the same as
+[[transitive_transfers]] and the comments there detail
+the problems with trying to support that.
+
+What you can do is download the files onto the computer that is connected
+to the phone, export them to the phone, and then drop the files from the
+computer.
+"""]]

diff --git a/doc/index.mdwn b/doc/index.mdwn
new file mode 100644
index 000000000..77b61da9d
--- /dev/null
+++ b/doc/index.mdwn
@@ -0,0 +1,40 @@
+[[!inline raw=yes pages="summary"]]
+
+[[!sidebar content="""
+[[!inline feeds=no template=bare pages=sidebar]]
+"""]]
+
+<table>
+<tr>
+<td width="33%" valign="top">[[!inline feeds=no template=bare pages=links/key_concepts]]</td>
+<td width="33%" valign="top">[[!inline feeds=no template=bare pages=links/the_details]]</td>
+<td width="33%" valign="top">[[!inline feeds=no template=bare pages=links/other_stuff]]</td>
+</tr>
+</table>
+
+<table>
+<tr>
+<td width="50%" valign="top">[[!inline feeds=no template=bare pages=use_case/bob]]</td>
+<td width="50%" valign="top">[[!inline feeds=no template=bare pages=use_case/alice]]</td>
+</tr>
+</table>
+
+If that describes you, or if you're some from column A and some from column
+B, then git-annex may be the tool you've been looking for to expand from
+keeping all your small important files in git, to managing your large
+files with git.
+
+<table>
+<tr>
+<td width="50%" valign="top">[[!inline feeds=no template=bare pages=footer/column_a]]</td>
+<td width="50%" valign="top">[[!inline feeds=no template=bare pages=footer/column_b]]</td>
+</tr>
+</table>
+
+----
+
+git-annex is [[Free Software|license]], written in [Haskell](http://www.haskell.org/).
+You can [[contribute]]!
+
+git-annex's wiki is powered by [Ikiwiki](http://ikiwiki.info/) and
+hosted by [Branchable](http://branchable.com/).

diff --git a/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_2_e4bfea21f664a0cb1aa8ec18ee88fc50._comment b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_2_e4bfea21f664a0cb1aa8ec18ee88fc50._comment
new file mode 100644
index 000000000..359626b21
--- /dev/null
+++ b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_2_e4bfea21f664a0cb1aa8ec18ee88fc50._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="satya.ortiz-gagne@a4c92de91eb4fd5ae8fc9893bb4fd674a19f2e59"
+ nickname="satya.ortiz-gagne"
+ avatar="http://cdn.libravatar.org/avatar/79c93025f174cd2aff98fbb952702c09"
+ subject="Re: using hardlinks"
+ date="2019-11-18T20:46:22Z"
+ content="""
+Thanks for your comment. I've looked into [local caching of annexed files](https://git-annex.branchable.com/tips/local_caching_of_annexed_files) and most of it can be found in the scenario [described in the test gist](https://gist.github.com/satyaog/b08a6e5d1eee75217ba823d38b84fb8b).
+
+The two settings `annex.thin` and `annex.hardlink` are also set in the two git-annex repositories of the test. Thanks for letting me know about the caveats. Based on the tests that I've executed, it would seam that [`git-annex unlock`](https://git-annex.branchable.com/git-annex-unlock/) now copies the file to avoid the mentioned issue as I noticed different inodes? I understand that this prevents unwanted lost of data while using git-annex but I would actually like to have a hardlink instead of a copy. I'm wondering if it's possible.
+"""]]

diff --git a/doc/devblog/day_608__easier_git-lfs_setup.mdwn b/doc/devblog/day_608__easier_git-lfs_setup.mdwn
new file mode 100644
index 000000000..4c1f6da54
--- /dev/null
+++ b/doc/devblog/day_608__easier_git-lfs_setup.mdwn
@@ -0,0 +1,25 @@
+The git-lfs support I added to git-annex had one small problem: People
+expect to be able to clone a git repo and get right to using it, but after
+cloning a git-annex repo that's on a server that uses git-lfs, there
+was an extra `git annex enableremote` step to be able to use it as a git-lfs
+special remote. And, you ended up with a "origin" git remote and a git-lfs
+special remote with some other name.
+
+Now, it's this simple to set up a git-lfs repo on eg, github:
+
+	git annex initremote github type=git-lfs encryption=none url=https://github.com/joeyh/lfstest
+	git annex sync github
+	git annex copy --to github ...
+
+And then for others to clone and use it is even simpler:
+
+	git clone https://github.com/joeyh/lfstest
+	cd lfstest
+	git annex get
+
+The only gotcha is that git-annex has to know the url that's used for
+the remote. Cloning any other url any other way (eg http instead of https)
+will result in git-annex not using it. This is a consequence of git-lfs
+not having any equivilant of a git-annex repository UUID, so git-annex
+can't probe for the UUID and has to compare urls. This can be worked
+around using `initremote --sameas` to tell git-annex about other urls.

diff --git a/CHANGELOG b/CHANGELOG
index 7fc2c7712..ab28d365a 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,7 @@ git-annex (7.20191115) UNRELEASED; urgency=medium
     an url. initremote --sameas can be used to add additional urls.
   * git-lfs: When there's a git remote with an url that's known to be
     used for git-lfs, automatically enable the special remote.
+  * sync, assistant: Pull and push from git-lfs remotes.
 
  -- Joey Hess <id@joeyh.name>  Fri, 15 Nov 2019 11:57:19 -0400
 
diff --git a/Remote/List.hs b/Remote/List.hs
index 5f3016b25..3e7ca9fa7 100644
--- a/Remote/List.hs
+++ b/Remote/List.hs
@@ -128,4 +128,8 @@ updateRemote remote = do
 {- Checks if a remote is syncable using git. -}
 gitSyncableRemote :: Remote -> Bool
 gitSyncableRemote r = remotetype r `elem`
-	[ Remote.Git.remote, Remote.GCrypt.remote, Remote.P2P.remote ]
+	[ Remote.Git.remote
+	, Remote.GCrypt.remote
+	, Remote.P2P.remote
+	, Remote.GitLFS.remote
+	]
diff --git a/doc/todo/sync_git-lfs_special_remote_should_sync_git_too.mdwn b/doc/todo/sync_git-lfs_special_remote_should_sync_git_too.mdwn
index 13df9dcfd..ebefc2c59 100644
--- a/doc/todo/sync_git-lfs_special_remote_should_sync_git_too.mdwn
+++ b/doc/todo/sync_git-lfs_special_remote_should_sync_git_too.mdwn
@@ -1,6 +1,5 @@
 git annex sync with a git-lfs special remote does not pull or push.
 It should.
-
-(Does gcrypt have the same problem, also being a special remote that's a
-git repo?)
 --[[Joey]]
+
+> [[fixed|done]] --[[Joey]]

diff --git a/Annex/SpecialRemote.hs b/Annex/SpecialRemote.hs
index 828eb6e77..37e0e2129 100644
--- a/Annex/SpecialRemote.hs
+++ b/Annex/SpecialRemote.hs
@@ -34,21 +34,9 @@ findExisting name = do
 	t <- trustMap
 	headMaybe
 		. sortBy (comparing $ \(u, _, _) -> Down $ M.lookup u t)
-		. findByName name
+		. findByRemoteConfig (\c -> lookupName c == Just name)
 		<$> Logs.Remote.readRemoteLog
 
-findByName :: RemoteName ->  M.Map UUID RemoteConfig -> [(UUID, RemoteConfig, Maybe (ConfigFrom UUID))]
-findByName n = map sameasuuid . filter (matching . snd) . M.toList
-  where
-	matching c = case lookupName c of
-		Nothing -> False
-		Just n'
-			| n' == n -> True
-			| otherwise -> False
-	sameasuuid (u, c) = case M.lookup sameasUUIDField c of
-		Nothing -> (u, c, Nothing)
-		Just u' -> (toUUID u', c, Just (ConfigFrom u))
-
 newConfig
 	:: RemoteName
 	-> Maybe (Sameas UUID)
diff --git a/Annex/SpecialRemote/Config.hs b/Annex/SpecialRemote/Config.hs
index 73688569c..e09ae8ecc 100644
--- a/Annex/SpecialRemote/Config.hs
+++ b/Annex/SpecialRemote/Config.hs
@@ -101,3 +101,11 @@ removeSameasInherited :: RemoteConfig -> RemoteConfig
 removeSameasInherited c = case M.lookup sameasUUIDField c of
 	Nothing -> c
 	Just _ -> M.withoutKeys c sameasInherits
+
+{- Finds remote uuids with matching RemoteConfig. -}
+findByRemoteConfig :: (RemoteConfig -> Bool) -> M.Map UUID RemoteConfig -> [(UUID, RemoteConfig, Maybe (ConfigFrom UUID))]
+findByRemoteConfig matching = map sameasuuid . filter (matching . snd) . M.toList
+  where
+	sameasuuid (u, c) = case M.lookup sameasUUIDField c of
+		Nothing -> (u, c, Nothing)
+		Just u' -> (toUUID u', c, Just (ConfigFrom u))
diff --git a/CHANGELOG b/CHANGELOG
index 182448d54..7fc2c7712 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -2,6 +2,11 @@ git-annex (7.20191115) UNRELEASED; urgency=medium
 
   * Stop displaying rsync progress, and use git-annex's own progress display
     for local-to-local repo transfers.
+  * git-lfs: The url provided to initremote/enableremote will now be
+    stored in the git-annex branch, allowing enableremote to be used without
+    an url. initremote --sameas can be used to add additional urls.
+  * git-lfs: When there's a git remote with an url that's known to be
+    used for git-lfs, automatically enable the special remote.
 
  -- Joey Hess <id@joeyh.name>  Fri, 15 Nov 2019 11:57:19 -0400
 
diff --git a/Git/Config.hs b/Git/Config.hs
index f250af73d..9ebd4bd0f 100644
--- a/Git/Config.hs
+++ b/Git/Config.hs
@@ -94,6 +94,14 @@ store s repo = do
 		, fullconfig = M.unionWith (++) c (fullconfig repo)
 		}
 
+{- Stores a single config setting in a Repo, returning the new version of
+ - the Repo. Config settings can be updated incrementally. -}
+store' :: String -> String -> Repo -> Repo
+store' k v repo = repo
+	{ config = M.singleton k v `M.union` config repo
+	, fullconfig = M.unionWith (++) (M.singleton k [v]) (fullconfig repo)
+	}
+
 {- Updates the location of a repo, based on its configuration.
  -
  - Git.Construct makes LocalUknown repos, of which only a directory is
diff --git a/Git/Remote.hs b/Git/Remote.hs
index 9b05a86fb..fa336013e 100644
--- a/Git/Remote.hs
+++ b/Git/Remote.hs
@@ -51,6 +51,7 @@ makeLegalName s = case filter legal $ replace "/" "_" s of
 	legal c = isAlphaNum c
 	
 data RemoteLocation = RemoteUrl String | RemotePath FilePath
+	deriving (Eq)
 
 remoteLocationIsUrl :: RemoteLocation -> Bool
 remoteLocationIsUrl (RemoteUrl _) = True
diff --git a/Remote/Git.hs b/Remote/Git.hs
index 059a1673d..6e1b31f74 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -143,7 +143,9 @@ configRead autoinit r = do
 		(True, _, _)
 			| remoteAnnexCheckUUID gc -> tryGitConfigRead autoinit r
 			| otherwise -> return r
-		(False, _, NoUUID) -> tryGitConfigRead autoinit r
+		(False, _, NoUUID) -> configSpecialGitRemotes r >>= \case
+			Nothing -> tryGitConfigRead autoinit r
+			Just r' -> return r'
 		_ -> return r
 
 gen :: Git.Repo -> UUID -> RemoteConfig -> RemoteGitConfig -> RemoteStateHandle -> Annex (Maybe Remote)
@@ -231,7 +233,7 @@ repoAvail r
 tryGitConfigRead :: Bool -> Git.Repo -> Annex Git.Repo
 tryGitConfigRead autoinit r 
 	| haveconfig r = return r -- already read
-	| Git.repoIsSsh r = store $ do
+	| Git.repoIsSsh r = storeUpdatedRemote $ do
 		v <- Ssh.onRemote NoConsumeStdin r
 			(pipedconfig, return (Left $ giveup "configlist failed"))
 			"configlist" [] configlistfields
@@ -240,10 +242,10 @@ tryGitConfigRead autoinit r
 				| haveconfig r' -> return r'
 				| otherwise -> configlist_failed
 			Left _ -> configlist_failed
-	| Git.repoIsHttp r = store geturlconfig
+	| Git.repoIsHttp r = storeUpdatedRemote geturlconfig
 	| Git.GCrypt.isEncrypted r = handlegcrypt =<< getConfigMaybe (remoteConfig r "uuid")
 	| Git.repoIsUrl r = return r
-	| otherwise = store $ liftIO $ 
+	| otherwise = storeUpdatedRemote $ liftIO $ 
 		readlocalannexconfig `catchNonAsync` (const $ return r)
   where
 	haveconfig = not . M.null . Git.config
@@ -278,18 +280,6 @@ tryGitConfigRead autoinit r
 				set_ignore "not usable by git-annex" False
 				return r
 
-	store = observe $ \r' -> do
-		l <- Annex.getGitRemotes
-		let rs = exchange l r'
-		Annex.changeState $ \s -> s { Annex.gitremotes = Just rs }
-
-	exchange [] _ = []
-	exchange (old:ls) new
-		| Git.remoteName old == Git.remoteName new =
-			new : exchange ls new
-		| otherwise =
-			old : exchange ls new
-
 	{- Is this remote just not available, or does
 	 - it not have git-annex-shell?
 	 - Find out by trying to fetch from the remote. -}
@@ -319,7 +309,7 @@ tryGitConfigRead autoinit r
 		g <- gitRepo
 		case Git.GCrypt.remoteRepoId g (Git.remoteName r) of
 			Nothing -> return r
-			Just v -> store $ liftIO $ setUUID r $
+			Just v -> storeUpdatedRemote $ liftIO $ setUUID r $
 				genUUIDInNameSpace gCryptNameSpace v
 
 	{- The local repo may not yet be initialized, so try to initialize
@@ -337,6 +327,31 @@ tryGitConfigRead autoinit r
 		then [(Fields.autoInit, "1")]
 		else []
 
+{- Handles special remotes that can be enabled by the presence of
+ - regular git remotes.
+ -
+ - When a remote repo is found to be such a special remote, its
+ - UUID is cached in the git config, and the repo returned with
+ - the UUID set.
+ -}
+configSpecialGitRemotes :: Git.Repo -> Annex (Maybe Git.Repo)
+configSpecialGitRemotes r = Remote.GitLFS.configKnownUrl r >>= \case
+	Nothing -> return Nothing
+	Just r' -> Just <$> storeUpdatedRemote (return r')
+
+storeUpdatedRemote :: Annex Git.Repo -> Annex Git.Repo
+storeUpdatedRemote = observe $ \r' -> do
+	l <- Annex.getGitRemotes
+	let rs = exchange l r'
+	Annex.changeState $ \s -> s { Annex.gitremotes = Just rs }
+  where
+	exchange [] _ = []
+	exchange (old:ls) new
+		| Git.remoteName old == Git.remoteName new =
+			new : exchange ls new
+		| otherwise =
+			old : exchange ls new
+
 {- Checks if a given remote has the content for a key in its annex. -}
 inAnnex :: Remote -> State -> Key -> Annex Bool
 inAnnex rmt st key = do
diff --git a/Remote/GitLFS.hs b/Remote/GitLFS.hs
index d62978819..01f76a5a8 100644
--- a/Remote/GitLFS.hs
+++ b/Remote/GitLFS.hs
@@ -5,7 +5,7 @@
  - Licensed under the GNU AGPL version 3 or higher.
  -}
 
-module Remote.GitLFS (remote, gen) where
+module Remote.GitLFS (remote, gen, configKnownUrl) where
 
 import Annex.Common
 import Types.Remote

(Diff truncated)

diff --git a/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_1_b59a72142e8f752d7b68a63a6ac1bbfe._comment b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_1_b59a72142e8f752d7b68a63a6ac1bbfe._comment
new file mode 100644
index 000000000..d341a02d3
--- /dev/null
+++ b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks/comment_1_b59a72142e8f752d7b68a63a6ac1bbfe._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="using hardlinks"
+ date="2019-11-18T19:21:18Z"
+ content="""
+I don't have a full answer, but [[tips/local_caching_of_annexed_files]] might have relevant info.
+
+There is also the `annex.thin` setting; but check some [caveats](https://git-annex.branchable.com/bugs/annex.thin_can_cause_corrupt___40__not_just_missing__41___data/) related to it.
+"""]]

diff --git a/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks.mdwn b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks.mdwn
new file mode 100644
index 000000000..403e2cafa
--- /dev/null
+++ b/doc/forum/How_to_prevent_copies_on_a_single_device_and_use_only_hardlinks.mdwn
@@ -0,0 +1,13 @@
+Hi,
+
+Thank you very much for this software. I'm working in a research institute and we are very interested into using git-annex with DataLad to manage our datasets.
+
+We aim to provide a datasets repository accessible through the local network on a single file system. Some of our datasets are multi TB with a few millions of files. It will be managed by a few people but the primary users, the researchers, will only have read access. We would like to use hardlinks everywhere to avoid infrequent reading errors related to symlinks and save space when we want to propose different versions of the datasets with slight changes. The file system will be backed-up so we don't really need multi copies of the same files on a single file system.
+
+We seam to be able to achieve this using the `direct` mode in git-annex version 5 but it seams that the `unlock` mode in version 7 does copies instead of hardlinks. I'm wondering how we could achieve the same behaviour as in version 5. I believe I've read in the doc that there's a maximum of 2 hardlinks for a single file but I can't remember where or see if that is still the case. If that is still the case, I couldn't find if there is a setting to set or remove this maximum.
+
+We've tested with git-annex local version 5 / build 7.20190819, local version 7 / build 7.20190819 and local version 7 / build 7.20191106. [Here is a gist](https://gist.github.com/satyaog/b08a6e5d1eee75217ba823d38b84fb8b) containing test scripts for each setup. The `.annex-cache` part can be ignored for this topic. I've used Miniconda3-4.3.30 on Ubuntu 18.04.2 LTS to setup the environments.
+
+Thank you,
+
+Satya

diff --git a/doc/todo/sync_git-lfs_special_remote_should_sync_git_too.mdwn b/doc/todo/sync_git-lfs_special_remote_should_sync_git_too.mdwn
new file mode 100644
index 000000000..13df9dcfd
--- /dev/null
+++ b/doc/todo/sync_git-lfs_special_remote_should_sync_git_too.mdwn
@@ -0,0 +1,6 @@
+git annex sync with a git-lfs special remote does not pull or push.
+It should.
+
+(Does gcrypt have the same problem, also being a special remote that's a
+git repo?)
+--[[Joey]]

diff --git a/doc/special_remotes/git-lfs.mdwn b/doc/special_remotes/git-lfs.mdwn
index e48a76cf4..00e0d2940 100644
--- a/doc/special_remotes/git-lfs.mdwn
+++ b/doc/special_remotes/git-lfs.mdwn
@@ -10,13 +10,6 @@ the git-lfs special remote:
 
 * `url` - Required. The url to the git-lfs repository to use.
   Can be either a ssh url (scp-style is also accepted) or a http url.
-  But currently, a http url accesses the git-lfs repository without
-  authentication. To authenticate, you will need to use a ssh url.
-
-  This parameter needs to be specified in the initial `git annex
-  initremote` but also each time you `git annex enableremote`
-  an existing git-lfs special remote. It's fine to use different urls
-  at different times as long as they point to the same git-lfs repository.
 
 * `encryption` - One of "none", "hybrid", "shared", or "pubkey".
   Required. See [[encryption]]. Also see the encryption notes below.

diff --git a/doc/index.mdwn b/doc/index.mdwn
deleted file mode 100644
index 77b61da9d..000000000
--- a/doc/index.mdwn
+++ /dev/null
@@ -1,40 +0,0 @@
-[[!inline raw=yes pages="summary"]]
-
-[[!sidebar content="""
-[[!inline feeds=no template=bare pages=sidebar]]
-"""]]
-
-<table>
-<tr>
-<td width="33%" valign="top">[[!inline feeds=no template=bare pages=links/key_concepts]]</td>
-<td width="33%" valign="top">[[!inline feeds=no template=bare pages=links/the_details]]</td>
-<td width="33%" valign="top">[[!inline feeds=no template=bare pages=links/other_stuff]]</td>
-</tr>
-</table>
-
-<table>
-<tr>
-<td width="50%" valign="top">[[!inline feeds=no template=bare pages=use_case/bob]]</td>
-<td width="50%" valign="top">[[!inline feeds=no template=bare pages=use_case/alice]]</td>
-</tr>
-</table>
-
-If that describes you, or if you're some from column A and some from column
-B, then git-annex may be the tool you've been looking for to expand from
-keeping all your small important files in git, to managing your large
-files with git.
-
-<table>
-<tr>
-<td width="50%" valign="top">[[!inline feeds=no template=bare pages=footer/column_a]]</td>
-<td width="50%" valign="top">[[!inline feeds=no template=bare pages=footer/column_b]]</td>
-</tr>
-</table>
-
-----
-
-git-annex is [[Free Software|license]], written in [Haskell](http://www.haskell.org/).
-You can [[contribute]]!
-
-git-annex's wiki is powered by [Ikiwiki](http://ikiwiki.info/) and
-hosted by [Branchable](http://branchable.com/).

diff --git a/doc/todo/git-annex-export_--from_option.mdwn b/doc/todo/git-annex-export_--from_option.mdwn
new file mode 100644
index 000000000..c9d07ee9d
--- /dev/null
+++ b/doc/todo/git-annex-export_--from_option.mdwn
@@ -0,0 +1,7 @@
+I just wanted to have a way to manage data copying / syncing between a fileserver and my android phone. So I pushed some files on my fileserver into a git remote and added the files with the annex subcommands then cloned the git tree from my workstation which is connected to my smartphone.
+
+Now I followed the documentation about the special remote adb and created that remote with the initremote command. When I then export I get (not available) failed errors.
+
+Which is caused by the fact that I didn't have checked out the files on my workstation. I don't need the files on this pc so it would be stupid to checkout partially huge files there or in other words I don't need the files at that place, I don't get why the export command not has a --from option where it can get the files?
+
+Is there a reason that does not exist and if so what would be a way to do sending files to the android device without ssh-ing into my server?

diff --git a/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default.mdwn b/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default.mdwn
new file mode 100644
index 000000000..4fc5706e2
--- /dev/null
+++ b/doc/todo/change_git-annex-import_not_to_delete_original_files_by_default.mdwn
@@ -0,0 +1,5 @@
+[[git-annex-import]] by default deletes the original files.  Keeping them by default would be better. "import" in many other tools (e.g. the bioinformatics tool [Geneious](https://www.geneious.com/)) means a non-destructive import.  The short description of `git-annex-import` on its man page says it "adds" files to the repo, which does not suggest erasure.  When I first used `git-annex-import`, I was surprised by the default behavior, and others may be too.  Also, the command has now been "overloaded" for importing from a special remote, and in that mode the originals are not erased; giving the import-from-dir mode the same default would be more consistent.  In general, erasing data by default seems dangerous: what if it was being imported into a temporary or untrusted repo?
+
+Changing the default would also let one [[repeatedly re-import a directory while keeping original files in place|bugs/impossible__40____63____41___to_continuously_re-import_a_directory_while_keeping_original_files_in_place]].
+
+I realize this would be a breaking change for some workflows; warning of it [[like git does|todo/warn_of_breaking_changes_same_way_git_does]] would mitigate the breakage.

diff --git a/doc/news/version_7.20191009.mdwn b/doc/news/version_7.20191009.mdwn
deleted file mode 100644
index c9647bf87..000000000
--- a/doc/news/version_7.20191009.mdwn
+++ /dev/null
@@ -1,29 +0,0 @@
-git-annex 7.20191009 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Fix bug in handling of annex.largefiles that use largerthan/smallerthan.
-     When adding a modified file, it incorrectly used the file size of the
-     old version of the file, not the current size.
-   * Added --mimetype and --mimeencoding file matching options.
-   * Added --unlocked and --locked file matching options.
-   * Added adjust --lock, to enter an adjusted branch where files are locked.
-   * git-lfs: Added support for http basic auth.
-   * git-lfs: Only do endpoint discovery once when concurrency is enabled.
-   * fsck --incremental/--more: Fix bug that prevented the incremental fsck
-     information from being updated every 5 minutes as it was supposed to be;
-     it was only updated after 1000 files were checked, which may be more
-     files that are possible to fsck in a given fsck time window.
-     Thanks to Peter Simons for help with analysis of this bug.
-   * Test: Use more robust directory removal when built with directory-1.2.7.
-   * Close sqlite databases more robustly.
-   * remotedaemon: Don't list --stop in help since it's not supported.
-   * enable-tor: Run kdesu with -c option.
-   * enable-tor: Use pkexec to run command as root when gksu and kdesu are not
-     available.
-   * When dropping an unlocked file, preserve its mtime, which avoids
-     git status unncessarily running the clean filter on the file.
-   * uninit: Remove several git hooks that git-annex init sets up.
-   * uninit: Remove the smudge and clean filters that git-annex init sets up.
-   * Work around git cat-file --batch's odd stripping of carriage return
-     from the end of the line (some windows infection), avoiding crashing
-     when the repo contains a filename ending in a carriage return.
-   * git-annex-standalone.rpm: Fix the git-annex-shell symlink."""]]
\ No newline at end of file
diff --git a/doc/news/version_7.20191114.mdwn b/doc/news/version_7.20191114.mdwn
new file mode 100644
index 000000000..6dfb3401e
--- /dev/null
+++ b/doc/news/version_7.20191114.mdwn
@@ -0,0 +1,10 @@
+git-annex 7.20191114 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Added annex.allowsign option.
+   * Make --json-error-messages capture more errors,
+     particularly url download errors.
+   * Fix a crash (STM deadlock) when -J is used with multiple files
+     that point to the same key.
+   * linuxstandalone: Fix a regression that broke git-remote-https.
+   * OSX git-annex.app: Fix a problem that prevented using the bundled
+     git-remote-https, git-remote-http, and git-shell."""]]
\ No newline at end of file

diff --git a/Build/OSXMkLibs.hs b/Build/OSXMkLibs.hs
index 17af6592b..8a11c88ee 100644
--- a/Build/OSXMkLibs.hs
+++ b/Build/OSXMkLibs.hs
@@ -50,8 +50,12 @@ installLibs appbase replacement_libs libmap = do
 		let symdest = appbase </> shortlib
 		-- This is a hack; libraries need to be in the same
 		-- directory as the program, so also link them into the
-		-- extra directory.
-		let symdestextra = appbase </> "extra" </> shortlib
+		-- extra and git-core directories so programs in those will
+		-- find them.
+		let symdestextra = 
+			[ appbase </> "extra" </> shortlib
+			, appbase </> "git-core" </> shortlib
+			]
 		ifM (doesFileExist dest)
 			( return Nothing
 			, do
@@ -59,9 +63,11 @@ installLibs appbase replacement_libs libmap = do
 				putStrLn $ "installing " ++ pathlib ++ " as " ++ shortlib
 				unlessM (boolSystem "cp" [File pathlib, File dest]
 					<&&> boolSystem "chmod" [Param "644", File dest]
-					<&&> boolSystem "ln" [Param "-s", File fulllib, File symdest]
-					<&&> boolSystem "ln" [Param "-s", File (".." </> fulllib), File symdestextra]) $
+					<&&> boolSystem "ln" [Param "-s", File fulllib, File symdest]) $
 					error "library install failed"
+				forM_ symdestextra $ \d ->
+					unlessM (boolSystem "ln" [Param "-s", File (".." </> fulllib), File d]) $
+						error "library linking failed"
 				return $ Just appbase
 			)
 	return (catMaybes libs, replacement_libs', libmap')
diff --git a/CHANGELOG b/CHANGELOG
index e775f6737..716bcb983 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -6,8 +6,8 @@ git-annex (7.20191107) UNRELEASED; urgency=medium
   * Fix a crash (STM deadlock) when -J is used with multiple files
     that point to the same key.
   * linuxstandalone: Fix a regression that broke git-remote-https.
-  * OSX git-annex.app: Fix a regression that broke git-remote-https,
-    git-remote-http, and git-shell.
+  * OSX git-annex.app: Fix a problem that prevented using the bundled
+    git-remote-https, git-remote-http, and git-shell.
 
  -- Joey Hess <id@joeyh.name>  Mon, 11 Nov 2019 15:59:47 -0400
 
diff --git a/doc/bugs/OSX_dmg_git-core_binaries_do_not_link.mdwn b/doc/bugs/OSX_dmg_git-core_binaries_do_not_link.mdwn
new file mode 100644
index 000000000..d976be1fd
--- /dev/null
+++ b/doc/bugs/OSX_dmg_git-core_binaries_do_not_link.mdwn
@@ -0,0 +1,12 @@
+The OSX .dmg contains a few binaries in git-core like git-remote-http.
+They have been adjusted by otool to link to libraries in the same directory
+as the binary. However, the libraries are not located in the git-core
+directory, but in its parent directory, and so the git-core binaries don't
+link.
+
+I don't think this is a new regression, but not entirely sure.
+
+Seems that OSXMkLibs could symlink ../lib into git-core.
+--[[Joey]] 
+
+> [[fixed|done]] --[[Joey]]

diff --git a/Build/Standalone.hs b/Build/Standalone.hs
index 6f1dab369..e02e70a66 100644
--- a/Build/Standalone.hs
+++ b/Build/Standalone.hs
@@ -56,16 +56,24 @@ installGitLibs topdir = do
 		if issymlink
 			then do
 				-- many git-core files may symlink to eg
-				-- ../../git. The link targets are put
-				-- into a subdirectory so all links to 
-				-- .../git get the same binary.
+				-- ../../bin/git, which is located outside
+				-- the git-core directory. The target of
+				-- such links is installed into a bin
+				-- directory, and the links repointed to it.
+				--
+				-- Other git-core files symlink to a file
+				-- beside them in the directory. Those
+				-- links can be copied as-is.
 				linktarget <- readSymbolicLink f
-				let linktarget' = gitcoredestdir </> "bin" </> takeFileName linktarget
-				createDirectoryIfMissing True (takeDirectory linktarget')
-				L.readFile f >>= L.writeFile linktarget'
-				nukeFile destf
-				rellinktarget <- relPathDirToFile (takeDirectory destf) linktarget'
-				createSymbolicLink rellinktarget destf
+				if takeFileName linktarget == linktarget
+					then cp f destf
+					else do
+						let linktarget' = gitcoredestdir </> "bin" </> takeFileName linktarget
+						createDirectoryIfMissing True (takeDirectory linktarget')
+						L.readFile f >>= L.writeFile linktarget'
+						nukeFile destf
+						rellinktarget <- relPathDirToFile (takeDirectory destf) linktarget'
+						createSymbolicLink rellinktarget destf
 			else cp f destf
 	
 	-- install git's template files
diff --git a/CHANGELOG b/CHANGELOG
index c6a628a6e..f2d4fe15a 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -5,6 +5,7 @@ git-annex (7.20191107) UNRELEASED; urgency=medium
     particularly url download errors.
   * Fix a crash (STM deadlock) when -J is used with multiple files
     that point to the same key.
+  * linuxstandalone: Fix a regression that broke git-remote-https.
 
  -- Joey Hess <id@joeyh.name>  Mon, 11 Nov 2019 15:59:47 -0400
 
diff --git a/Makefile b/Makefile
index 58b3f586c..944437e79 100644
--- a/Makefile
+++ b/Makefile
@@ -166,6 +166,7 @@ linuxstandalone:
 	cp /usr/share/file/magic.mgc "$(LINUXSTANDALONE_DEST)/magic"
 	cp /usr/share/i18n -a "$(LINUXSTANDALONE_DEST)"
 	
+	read me
 	./Build/LinuxMkLibs "$(LINUXSTANDALONE_DEST)"
 	
 	$(MAKE) install-mans DESTDIR="$(LINUXSTANDALONE_DEST)"
diff --git a/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
index acdcb73e8..9ec6964e7 100644
--- a/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
+++ b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
@@ -75,3 +75,5 @@ Unfortunately in datalad we had no test testing cloning over https, so I added s
 
 [[!meta author=yoh]]
 [[!tag projects/datalad]]
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time/comment_1_1735409e62ce82f7ba7258b0167fda06._comment b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time/comment_1_1735409e62ce82f7ba7258b0167fda06._comment
new file mode 100644
index 000000000..1c80ed5ec
--- /dev/null
+++ b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time/comment_1_1735409e62ce82f7ba7258b0167fda06._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2019-11-14T18:14:35Z"
+ content="""
+It will either be caused by 5463f97ca216cd261f7a1da08aa8a62cef415a71 or by
+a new version of git reorging files (or both).
+"""]]

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_a77d982399c91c2513ae74fb346243f0._comment b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_a77d982399c91c2513ae74fb346243f0._comment
deleted file mode 100644
index b24289907..000000000
--- a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_a77d982399c91c2513ae74fb346243f0._comment
+++ /dev/null
@@ -1,11 +0,0 @@
-[[!comment format=mdwn
- username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
- nickname="git-annex"
- avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
- subject="Sometimes it sorta works"
- date="2019-11-14T17:24:29Z"
- content="""
-So, sometimes (often), it says it synched in the webapp, but no changes show up in any branch on gitlab.  However, sometimes it works.  For example, sometimes after killing everything off (including any ssh sessions) and restarting the webapp, a file I added to the repo locally as a test will show up in the gitlab repo.  When I then move that file, the webapp/assistant wakes up, syncs (says it was successful), and the \"delete\" is processed in the gitlab repo (the old file is deleted), but the \"add\" (the file showing up with a new name) doesn't.  
-
-
-"""]]

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_a77d982399c91c2513ae74fb346243f0._comment b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_a77d982399c91c2513ae74fb346243f0._comment
new file mode 100644
index 000000000..b24289907
--- /dev/null
+++ b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_2_a77d982399c91c2513ae74fb346243f0._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
+ nickname="git-annex"
+ avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
+ subject="Sometimes it sorta works"
+ date="2019-11-14T17:24:29Z"
+ content="""
+So, sometimes (often), it says it synched in the webapp, but no changes show up in any branch on gitlab.  However, sometimes it works.  For example, sometimes after killing everything off (including any ssh sessions) and restarting the webapp, a file I added to the repo locally as a test will show up in the gitlab repo.  When I then move that file, the webapp/assistant wakes up, syncs (says it was successful), and the \"delete\" is processed in the gitlab repo (the old file is deleted), but the \"add\" (the file showing up with a new name) doesn't.  
+
+
+"""]]

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_1_b61139816f55dbc4f7eac22207f243c0._comment b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_1_b61139816f55dbc4f7eac22207f243c0._comment
new file mode 100644
index 000000000..71e8b6fe6
--- /dev/null
+++ b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo/comment_1_b61139816f55dbc4f7eac22207f243c0._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
+ nickname="git-annex"
+ avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
+ subject="Sometimes it sorta works"
+ date="2019-11-14T17:24:11Z"
+ content="""
+So, sometimes (often), it says it synched in the webapp, but no changes show up in any branch on gitlab.  However, sometimes it works.  For example, sometimes after killing everything off (including any ssh sessions) and restarting the webapp, a file I added to the repo locally as a test will show up in the gitlab repo.  When I then move that file, the webapp/assistant wakes up, syncs (says it was successful), and the \"delete\" is processed in the gitlab repo (the old file is deleted), but the \"add\" (the file showing up with a new name) doesn't.  
+
+
+"""]]

diff --git a/Annex/Concurrent.hs b/Annex/Concurrent.hs
index 4626a9294..1ff8e0c73 100644
--- a/Annex/Concurrent.hs
+++ b/Annex/Concurrent.hs
@@ -90,10 +90,20 @@ enteringStage newstage a = Annex.getState Annex.workers >>= \case
 	Nothing -> a
 	Just tv -> do
 		mytid <- liftIO myThreadId
-		let set = changeStageTo mytid tv newstage
-		let restore = maybe noop (void . changeStageTo mytid tv)
+		let set = changeStageTo mytid tv (const newstage)
+		let restore = maybe noop (void . changeStageTo mytid tv . const)
 		bracket set restore (const a)
 
+{- Transition the current thread to the initial stage.
+ - This is done once the thread is ready to begin work.
+ -}
+enteringInitialStage :: Annex ()
+enteringInitialStage = Annex.getState Annex.workers >>= \case
+	Nothing -> noop
+	Just tv -> do
+		mytid <- liftIO myThreadId
+		void $ changeStageTo mytid tv initialStage
+
 {- This needs to leave the WorkerPool with the same number of
  - idle and active threads, and with the same number of threads for each
  - WorkerStage. So, all it can do is swap the WorkerStage of our thread's
@@ -110,14 +120,15 @@ enteringStage newstage a = Annex.getState Annex.workers >>= \case
  - in the pool than spareVals. That does not prevent other threads that call
  - this from using them though, so it's fine.
  -}
-changeStageTo :: ThreadId -> TMVar (WorkerPool AnnexState) -> WorkerStage -> Annex (Maybe WorkerStage)
-changeStageTo mytid tv newstage = liftIO $
+changeStageTo :: ThreadId -> TMVar (WorkerPool AnnexState) -> (UsedStages -> WorkerStage) -> Annex (Maybe WorkerStage)
+changeStageTo mytid tv getnewstage = liftIO $
 	replaceidle >>= maybe
 		(return Nothing)
 		(either waitidle (return . Just))
   where
 	replaceidle = atomically $ do
 		pool <- takeTMVar tv
+		let newstage = getnewstage (usedStages pool)
 		let notchanging = do
 			putTMVar tv pool
 			return Nothing
@@ -128,7 +139,7 @@ changeStageTo mytid tv newstage = liftIO $
 						Nothing -> do
 							putTMVar tv $
 								addWorkerPool (IdleWorker oldstage) pool'
-							return $ Just $ Left (myaid, oldstage)
+							return $ Just $ Left (myaid, newstage, oldstage)
 						Just pool'' -> do
 							-- optimisation
 							putTMVar tv $
@@ -139,27 +150,26 @@ changeStageTo mytid tv newstage = liftIO $
 				_ -> notchanging
 			else notchanging
 	
-	waitidle (myaid, oldstage) = atomically $ do
+	waitidle (myaid, newstage, oldstage) = atomically $ do
 		pool <- waitIdleWorkerSlot newstage =<< takeTMVar tv
 		putTMVar tv $ addWorkerPool (ActiveWorker myaid newstage) pool
 		return (Just oldstage)
 
--- | Waits until there's an idle worker in the worker pool
--- for its initial stage, removes it from the pool, and returns its state.
+-- | Waits until there's an idle StartStage worker in the worker pool,
+-- removes it from the pool, and returns its state.
 --
 -- If the worker pool is not already allocated, returns Nothing.
-waitInitialWorkerSlot :: TMVar (WorkerPool Annex.AnnexState) -> STM (Maybe (Annex.AnnexState, WorkerStage))
-waitInitialWorkerSlot tv = do
+waitStartWorkerSlot :: TMVar (WorkerPool Annex.AnnexState) -> STM (Maybe (Annex.AnnexState, WorkerStage))
+waitStartWorkerSlot tv = do
 	pool <- takeTMVar tv
-	let stage = initialStage (usedStages pool)
-	st <- go stage pool
-	return $ Just (st, stage)
+	st <- go pool
+	return $ Just (st, StartStage)
   where
-	go wantstage pool = case spareVals pool of
+	go pool = case spareVals pool of
 		[] -> retry
 		(v:vs) -> do
 			let pool' = pool { spareVals = vs }
-			putTMVar tv =<< waitIdleWorkerSlot wantstage pool'
+			putTMVar tv =<< waitIdleWorkerSlot StartStage pool'
 			return v
 
 waitIdleWorkerSlot :: WorkerStage -> WorkerPool Annex.AnnexState -> STM (WorkerPool Annex.AnnexState)
diff --git a/CHANGELOG b/CHANGELOG
index 03d8e7b11..c6a628a6e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -3,6 +3,8 @@ git-annex (7.20191107) UNRELEASED; urgency=medium
   * Added annex.allowsign option.
   * Make --json-error-messages capture more errors,
     particularly url download errors.
+  * Fix a crash (STM deadlock) when -J is used with multiple files
+    that point to the same key.
 
  -- Joey Hess <id@joeyh.name>  Mon, 11 Nov 2019 15:59:47 -0400
 
diff --git a/CmdLine/Action.hs b/CmdLine/Action.hs
index 87298a95f..67a7618e4 100644
--- a/CmdLine/Action.hs
+++ b/CmdLine/Action.hs
@@ -63,7 +63,7 @@ commandAction start = Annex.getState Annex.concurrency >>= \case
 	runconcurrent = Annex.getState Annex.workers >>= \case
 		Nothing -> runnonconcurrent
 		Just tv -> 
-			liftIO (atomically (waitInitialWorkerSlot tv)) >>=
+			liftIO (atomically (waitStartWorkerSlot tv)) >>=
 				maybe runnonconcurrent (runconcurrent' tv)
 	runconcurrent' tv (workerst, workerstage) = do
 		aid <- liftIO $ async $ snd <$> Annex.run workerst
@@ -99,12 +99,13 @@ commandAction start = Annex.getState Annex.concurrency >>= \case
 					case mkActionItem startmsg' of
 						OnlyActionOn k' _ | k' /= k ->
 							concurrentjob' workerst startmsg' perform'
-						_ -> mkjob workerst startmsg' perform'
+						_ -> beginjob workerst startmsg' perform'
 				Nothing -> noop
-		_ -> mkjob workerst startmsg perform
+		_ -> beginjob workerst startmsg perform
 	
-	mkjob workerst startmsg perform = 
-		inOwnConsoleRegion (Annex.output workerst) $
+	beginjob workerst startmsg perform =
+		inOwnConsoleRegion (Annex.output workerst) $ do
+			enteringInitialStage
 			void $ accountCommandAction startmsg $
 				performconcurrent startmsg perform
 
diff --git a/Types/WorkerPool.hs b/Types/WorkerPool.hs
index 178c30166..8a6816313 100644
--- a/Types/WorkerPool.hs
+++ b/Types/WorkerPool.hs
@@ -40,7 +40,12 @@ instance Show (Worker t) where
 	show (ActiveWorker _ s) = "ActiveWorker " ++ show s
 
 data WorkerStage
-	= PerformStage
+	= StartStage
+	-- ^ All threads start in this stage, and then transition away from
+	-- it to the initialStage when they begin doing work. This should
+	-- never be included in UsedStages, because transition from some
+	-- other stage back to this one could result in a deadlock.
+	| PerformStage
 	-- ^ Running a CommandPerform action.
 	| CleanupStage
 	-- ^ Running a CommandCleanup action.
@@ -102,12 +107,13 @@ workerAsync (ActiveWorker aid _) = Just aid
 allocateWorkerPool :: t -> Int -> UsedStages -> WorkerPool t
 allocateWorkerPool t n u = WorkerPool
 	{ usedStages = u
-	, workerList = take totalthreads $ map IdleWorker stages
+	, workerList = map IdleWorker $
+		take totalthreads $ concat $ repeat stages
 	, spareVals = replicate totalthreads t
 	}
   where
-	stages = concat $ repeat $ S.toList $ stageSet u
-	totalthreads = n * S.size (stageSet u)
+	stages = StartStage : S.toList (stageSet u)
+	totalthreads = n * length stages
 
 addWorkerPool :: Worker t -> WorkerPool t -> WorkerPool t
 addWorkerPool w pool = pool { workerList = w : workerList pool }
diff --git a/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction.mdwn b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction.mdwn
index 837cfecb1..1aa191ed8 100644
--- a/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction.mdwn
+++ b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction.mdwn
@@ -69,3 +69,5 @@ I felt like it is an old issue but failed to find a trace of it upon a quick loo
 
 [[!meta author=yoh]]
 [[!tag projects/datalad]]
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_6_e33df0ec76069deb069b94c944d62c76._comment b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_6_e33df0ec76069deb069b94c944d62c76._comment
new file mode 100644
index 000000000..7f348fc9f
--- /dev/null
+++ b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_6_e33df0ec76069deb069b94c944d62c76._comment
@@ -0,0 +1,69 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 6"""
+ date="2019-11-14T15:20:13Z"
+ content="""
+Added tracing of changes to the WorkerPool.
+
+	joey@darkstar:/tmp/dst>git annex get -J1 1 2 --json
+	("initial pool",WorkerPool UsedStages {initialStage = TransferStage, stageSet = fromList [TransferStage,VerifyStage]} [IdleWorker TransferStage,IdleWorker VerifyStage] 2)
+	("starting worker",WorkerPool UsedStages {initialStage = TransferStage, stageSet = fromList [TransferStage,VerifyStage]} [ActiveWorker TransferStage,IdleWorker VerifyStage] 1)
+
+Transfer starts for file 1
+
+	(("change stage from",TransferStage,"to",VerifyStage),WorkerPool UsedStages {initialStage = TransferStage, stageSet = fromList [TransferStage,VerifyStage]} [IdleWorker TransferStage,ActiveWorker VerifyStage] 1)

(Diff truncated)

diff --git a/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo.mdwn b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo.mdwn
new file mode 100644
index 000000000..c7f821b2b
--- /dev/null
+++ b/doc/forum/Issues_with_webapp___47___assistant_and_gitlab_as_a_metadata_only_repo.mdwn
@@ -0,0 +1,63 @@
+Greetings,
+
+Two more issues have come up.  
+
+1) webapp (and assistant) don't always seem to inherit ssh-agent keys (I've setup a passwordless key for gitlab).  It usually takes multiple cycles of killing off ssh and annex-related daemons before a new instance will not fail auth with gitlab - is there a more solid way of doing this?
+
+2) When I do get auth, the assistant / webui will notice file changes (i.e moving a small file that doesn't match annex.largefiles, but the changes don't get committed to origin (gitlab - metadata only).  The log file is below - thougts?
+
+[2019-11-13 20:24:33.378362] main: starting assistant version 7.20191106
+[2019-11-13 20:24:33.439501] TransferScanner: Syncing with origin
+(scanning...) [2019-11-13 20:24:33.529954] Watcher: Performing startup scan
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+fatal: Pathspec 'workflow/cc-archive-exif/LICENSE' is in submodule 'workflow/cc-archive-exif'
+
+git cat-file EOF: user error
+
+fd:39: hFlush: resource vanished (Broken pipe)
+
+fd:39: hFlush: resource vanished (Broken pipe)
+(started...)
+[2019-11-13 20:24:34.481957] Committer: Committing changes to git
+(recording state in git...)
+> GitLab: Disallowed command
+ControlSocket .git/annex/ssh/git@gitlab.com already exists, disabling multiplexing
+[2019-11-13 20:24:35.42995] Pusher: Syncing with origin
+Everything up-to-date
+> GitLab: Disallowed command
+Everything up-to-date
+> GitLab: Disallowed command
+> GitLab: Disallowed command
+> GitLab: Disallowed command
+> GitLab: Disallowed command
+> GitLab: Disallowed command
+
+fd:39: hFlush: resource vanished (Broken pipe)
+
+fd:39: hFlush: resource vanished (Broken pipe)
+[2019-11-13 20:26:09.256217] main: Syncing with origin
+Everything up-to-date
+> GitLab: Disallowed command
+
+fd:39: hFlush: resource vanished (Broken pipe)
+[2019-11-13 20:27:31.922901] Committer: Committing changes to git
+(recording state in git...)
+[2019-11-13 20:27:31.956249] Pusher: Syncing with origin
+Everything up-to-date
+
+fd:39: hFlush: resource vanished (Broken pipe)
+[2019-11-13 20:28:35.846741] Committer: Committing changes to git
+(recording state in git...)
+[2019-11-13 20:28:35.87987] Pusher: Syncing with origin
+Everything up-to-date
+> GitLab: Disallowed command
+> GitLab: Disallowed command

diff --git a/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
index 153a7e2c8..acdcb73e8 100644
--- a/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
+++ b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
@@ -67,6 +67,8 @@ git-credential-cache@  git-http-fetch@                git-remote-tor-annex@  rsy
 
 """]]
 
+</details>
+
 so may be that is related.
 
 Unfortunately in datalad we had no test testing cloning over https, so I added such integration test in https://github.com/datalad/datalad/pull/3867 to at least detect such regressions in the future before hitting the userland

diff --git a/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
new file mode 100644
index 000000000..153a7e2c8
--- /dev/null
+++ b/doc/bugs/regression__58___standalone_build_is_deficient_on_linux_after_7.20190819+git2-g908476a9b-1__126__ndall+1_some_time.mdwn
@@ -0,0 +1,75 @@
+### Please describe the problem.
+
+There were a few changes introduced since then to Makefile (I will not guess which one broke it) which resulted in git within git-annex-standalone of neurodebian to be unable to clone from https://:
+
+
+[[!format sh """
+$> /usr/lib/git-annex.linux/git clone https://github.com/afni/afni_ci_test_data.git                            
+Cloning into 'afni_ci_test_data'...
+fatal: unable to find remote helper for 'https'
+
+"""]]
+
+<details>
+<summary>diff between list of files in 7.20190819+git60-gcdb679818 and 7.20191017+git2-g7b13db551 package builds shows many git-* missing</summary>
+[[!format sh """
+lena:/tmp
+$> ls 7.2019*/usr/lib/git-annex.linux/exe/                                         
+7.20190819/usr/lib/git-annex.linux/exe/:
+cp@                            git-diff-index@          git-mktag@                git-sh-i18n--envsubst@
+curl@                          git-diff-tree@           git-mktree@               git-shell@
+git@                           git-difftool@            git-multi-pack-index@     git-shortlog@
+git-add@                       git-fast-export@         git-mv@                   git-show@
+git-am@                        git-fast-import@         git-name-rev@             git-show-branch@
+git-annex@                     git-fetch@               git-notes@                git-show-index@
+git-annex-shell@               git-fetch-pack@          git-pack-objects@         git-show-ref@
+git-annotate@                  git-fmt-merge-msg@       git-pack-redundant@       git-stage@
+git-apply@                     git-for-each-ref@        git-pack-refs@            git-status@
+git-archive@                   git-format-patch@        git-patch-id@             git-stripspace@
+git-bisect--helper@            git-fsck@                git-prune@                git-submodule--helper@
+git-blame@                     git-fsck-objects@        git-prune-packed@         git-symbolic-ref@
+git-branch@                    git-gc@                  git-pull@                 git-tag@
+git-bundle@                    git-get-tar-commit-id@   git-push@                 git-unpack-file@
+git-cat-file@                  git-grep@                git-range-diff@           git-unpack-objects@
+git-check-attr@                git-hash-object@         git-read-tree@            git-update-index@
+git-check-ignore@              git-help@                git-rebase@               git-update-ref@
+git-check-mailmap@             git-http-backend@        git-rebase--interactive@  git-update-server-info@
+git-check-ref-format@          git-http-fetch@          git-receive-pack@         git-upload-archive@
+git-checkout@                  git-http-push@           git-reflog@               git-upload-pack@
+git-checkout-index@            git-imap-send@           git-remote@               git-var@
+git-cherry@                    git-index-pack@          git-remote-ext@           git-verify-commit@
+git-cherry-pick@               git-init@                git-remote-fd@            git-verify-pack@
+git-clean@                     git-init-db@             git-remote-ftp@           git-verify-tag@
+git-clone@                     git-interpret-trailers@  git-remote-ftps@          git-whatchanged@
+git-column@                    git-log@                 git-remote-http@          git-worktree@
+git-commit@                    git-ls-files@            git-remote-https@         git-write-tree@
+git-commit-graph@              git-ls-remote@           git-remote-testsvn@       localedef@
+git-commit-tree@               git-ls-tree@             git-remote-tor-annex@     lsof@
+git-config@                    git-mailinfo@            git-repack@               rsync@
+git-count-objects@             git-mailsplit@           git-replace@              sh@
+git-credential@                git-merge@               git-rerere@               ssh@
+git-credential-cache@          git-merge-base@          git-reset@                ssh-keygen@
+git-credential-cache--daemon@  git-merge-file@          git-rev-list@             tar@
+git-credential-store@          git-merge-index@         git-rev-parse@            uname@
+git-daemon@                    git-merge-ours@          git-revert@               xargs@
+git-describe@                  git-merge-recursive@     git-rm@
+git-diff@                      git-merge-subtree@       git-send-pack@
+git-diff-files@                git-merge-tree@          git-serve@
+
+7.20191017/usr/lib/git-annex.linux/exe/:
+cp@                    git-credential-cache--daemon@  git-http-push@         git-sh-i18n--envsubst@  sh@
+curl@                  git-credential-store@          git-imap-send@         git-shell@              ssh@
+git@                   git-daemon@                    git-receive-pack@      git-upload-pack@        ssh-keygen@
+git-annex@             git-fast-import@               git-remote-http@       localedef@              tar@
+git-annex-shell@       git-http-backend@              git-remote-testsvn@    lsof@                   uname@
+git-credential-cache@  git-http-fetch@                git-remote-tor-annex@  rsync@                  xargs@
+
+
+"""]]
+
+so may be that is related.
+
+Unfortunately in datalad we had no test testing cloning over https, so I added such integration test in https://github.com/datalad/datalad/pull/3867 to at least detect such regressions in the future before hitting the userland
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

diff --git a/doc/forum/git-annex_clients_with_only_a_gitLab_repo_and_GDrive_special_remote_in_common/comment_2_ff0dab8dc544f7aa8e4ebfbec8f2a081._comment b/doc/forum/git-annex_clients_with_only_a_gitLab_repo_and_GDrive_special_remote_in_common/comment_2_ff0dab8dc544f7aa8e4ebfbec8f2a081._comment
new file mode 100644
index 000000000..0e4a37c0e
--- /dev/null
+++ b/doc/forum/git-annex_clients_with_only_a_gitLab_repo_and_GDrive_special_remote_in_common/comment_2_ff0dab8dc544f7aa8e4ebfbec8f2a081._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="git-annex@17927e6dc041ab425c14217a97a685adf3ecf44f"
+ nickname="git-annex"
+ avatar="http://cdn.libravatar.org/avatar/66e5c6e044d726597ce5a0ad68f86fe4"
+ subject="comment 2"
+ date="2019-11-14T00:38:56Z"
+ content="""
+Thank's Joey - that's what I needed. 
+"""]]

diff --git a/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_6_880c962d5ca77c494c984e7f74725265._comment b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_6_880c962d5ca77c494c984e7f74725265._comment
new file mode 100644
index 000000000..f8d18b1cf
--- /dev/null
+++ b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_6_880c962d5ca77c494c984e7f74725265._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="catch-all deadlock breaker"
+ date="2019-11-13T22:33:59Z"
+ content="""
+Not sure if feasible, but maybe a [[catch-all deadlock breaker|todo/more_extensive_retries_to_mask_transient_failures]] could be implemented to mask this and other deadlocks?
+
+The moon landings software [[had something|https://www.ibiblio.org/apollo/hrst/archive/1033.pdf]] [[like this|https://history.nasa.gov/computers/Ch2-6.html]], and it worked [[pretty well|https://www.wsj.com/articles/apollo-11-had-a-hidden-hero-software-11563153001]]...
+"""]]

diff --git a/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_4_eecaa7f7b0279c56902c90ed58d1444f._comment b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_4_eecaa7f7b0279c56902c90ed58d1444f._comment
new file mode 100644
index 000000000..4c1ad5521
--- /dev/null
+++ b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_4_eecaa7f7b0279c56902c90ed58d1444f._comment
@@ -0,0 +1,83 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2019-11-13T21:22:07Z"
+ content="""
+Simplified version of patch above, that converts ensureOnlyActionOn to not use
+STM at all, and is significantly simpler.
+
+With this patch, the test case still STM deadlocks. So this seems to be
+proof that the actual problem is not in ensureOnlyActionOn.
+
+	diff --git a/Annex.hs b/Annex.hs
+	index 9eb4c5f39..9baf7755a 100644
+	--- a/Annex.hs
+	+++ b/Annex.hs
+	@@ -143,7 +143,7 @@ data AnnexState = AnnexState
+	 	, existinghooks :: M.Map Git.Hook.Hook Bool
+	 	, desktopnotify :: DesktopNotify
+	 	, workers :: Maybe (TMVar (WorkerPool AnnexState))
+	-	, activekeys :: TVar (M.Map Key ThreadId)
+	+	, activekeys :: MVar (M.Map Key (ThreadId, MVar ()))
+	 	, activeremotes :: MVar (M.Map (Types.Remote.RemoteA Annex) Integer)
+	 	, keysdbhandle :: Maybe Keys.DbHandle
+	 	, cachedcurrentbranch :: (Maybe (Maybe Git.Branch, Maybe Adjustment))
+	@@ -154,7 +154,7 @@ data AnnexState = AnnexState
+	 newState :: GitConfig -> Git.Repo -> IO AnnexState
+	 newState c r = do
+	 	emptyactiveremotes <- newMVar M.empty
+	-	emptyactivekeys <- newTVarIO M.empty
+	+	emptyactivekeys <- newMVar M.empty
+	 	o <- newMessageState
+	 	sc <- newTMVarIO False
+	 	return $ AnnexState
+	diff --git a/CmdLine/Action.hs b/CmdLine/Action.hs
+	index 87298a95f..a8c2bd205 100644
+	--- a/CmdLine/Action.hs
+	+++ b/CmdLine/Action.hs
+	@@ -22,7 +22,7 @@ import Remote.List
+	 import Control.Concurrent
+	 import Control.Concurrent.Async
+	 import Control.Concurrent.STM
+	-import GHC.Conc
+	+import GHC.Conc (getNumProcessors)
+	 import qualified Data.Map.Strict as M
+	 import qualified System.Console.Regions as Regions
+	 
+	@@ -267,17 +267,22 @@ ensureOnlyActionOn k a = debugLocks $
+	 	go (Concurrent _) = goconcurrent
+	 	go ConcurrentPerCpu = goconcurrent
+	 	goconcurrent = do
+	-		tv <- Annex.getState Annex.activekeys
+	-		bracket (setup tv) id (const a)
+	-	setup tv = liftIO $ do
+	+		mv <- Annex.getState Annex.activekeys
+	+		bracketIO (setup mv) id (const a)
+	+	setup mv =  do
+	 		mytid <- myThreadId
+	-		atomically $ do
+	-			m <- readTVar tv
+	-			case M.lookup k m of
+	-				Just tid
+	-					| tid /= mytid -> retry
+	-					| otherwise -> return $ return ()
+	-				Nothing -> do
+	-					writeTVar tv $! M.insert k mytid m
+	-					return $ liftIO $ atomically $
+	-						modifyTVar tv $ M.delete k
+	+		m <- takeMVar mv
+	+		let ready sem = do
+	+			putMVar mv $! M.insert k (mytid, sem) m
+	+			return $ do
+	+				modifyMVar_ mv $ pure . M.delete k
+	+				putMVar sem ()
+	+		case M.lookup k m of
+	+			Nothing -> ready =<< newEmptyMVar
+	+			Just (tid, sem)
+	+				| tid /= mytid -> do
+	+					takeMVar sem
+	+					ready sem
+	+				| otherwise -> do
+	+					putMVar mv m
+	+					return noop
+"""]]
diff --git a/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_5_052acf169f9c2b8b0233adddabb02559._comment b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_5_052acf169f9c2b8b0233adddabb02559._comment
new file mode 100644
index 000000000..4d3cc32da
--- /dev/null
+++ b/doc/bugs/parallel_get_to_the_files_for_the_same_key_would_fail_with__thread_blocked_indefinitely_in_an_STM_transaction/comment_5_052acf169f9c2b8b0233adddabb02559._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2019-11-13T21:42:58Z"
+ content="""
+finishCommandActions is reaching the retry case, and STM deadlocks there.
+The WorkerPool is getting into a state where allIdle is False, and is not
+leaving it, perhaps due to an earlier STM deadlock. (There seem to be two
+different ones.)
+
+Also, I notice with --json-error-messages:
+
+	{"command":"get","note":"from origin...\nchecksum...","success":false,"key":"SHA256E-s524288--07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541","error-messages":["git-annex: thread blocked indefinitely in an STM transaction"],"file":"1"}
+
+So the thread that actually gets to run on the key is somehow reaching a
+STM deadlock.
+
+Which made me wonder if that thread deadlocks on enteringStage.
+And it seems so. If Command.Get is changed to use commandStages
+rather than transferStages, the test case succeeds.
+
+Like finishCommandActions, enteringStage has a STM retry if it needs to
+wait for something to happen to the WorkerPool. So again it looks like
+the WorkerPool is getting screwed up.
+"""]]