diff --git a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_4_64c0d3c061ff463f9ce16b52d2a70a1f._comment b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_4_64c0d3c061ff463f9ce16b52d2a70a1f._comment
new file mode 100644
index 000000000..61300bb84
--- /dev/null
+++ b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_4_64c0d3c061ff463f9ce16b52d2a70a1f._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="cardoso-neto"
+ avatar="http://cdn.libravatar.org/avatar/d90a656df072f3a29da54302c190c696"
+ subject="Fixing URLs."
+ date="2020-08-06T03:14:57Z"
+ content="""
+Apparently none of them worked, so here they are again:
+
+[github/Lykos153/git-annex-remote-googledrive](https://github.com/Lykos153/git-annex-remote-googledrive)
+
+[github/cardoso-neto/git-annex-remote-manytars](https://github.com/cardoso-neto/git-annex-remote-manytars)
+
+[github/cardoso-neto/git-annex-remote-manyzips](https://github.com/cardoso-neto/git-annex-remote-manyzips)
+
+[github/Lykos153/AnnexRemote](https://github.com/Lykos153/AnnexRemote)
+"""]]

diff --git a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_3_93e91e2ab98c35d4cb6082eea01b6abc._comment b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_3_93e91e2ab98c35d4cb6082eea01b6abc._comment
new file mode 100644
index 000000000..6e53f06e2
--- /dev/null
+++ b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_3_93e91e2ab98c35d4cb6082eea01b6abc._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="cardoso-neto"
+ avatar="http://cdn.libravatar.org/avatar/d90a656df072f3a29da54302c190c696"
+ subject="Correction."
+ date="2020-08-06T03:09:12Z"
+ content="""
+I meant this Google Drive remote: <github.com/Lykos153/git-annex-remote-googledrive> not the gdrive one that was discontinued.
+"""]]

diff --git a/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files..mdwn b/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files..mdwn
new file mode 100644
index 000000000..aeeca06c7
--- /dev/null
+++ b/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files..mdwn
@@ -0,0 +1,56 @@
+I was git-annex-adding a folder of big files.
+I ran the command while inside a folder I didn't need anymore.
+I deleted the folder with nautilus (moved it to trash) which caused the cwd to change and that really confused git-annex.
+
+Root of repo is audio-recordings/
+
+While inside audio-recordings/a I ran `git annex add ../audio-files`
+
+Then I deleted audio-recordings/a with Nautilus which caused its path to change to /.Trash-1000/files/a
+
+As soon as git-annex finished hashing the file it was hashing, this happened:
+
+```
+add ../audio-files/moto-maxx/2018.04.17 1438.wav 
+100%  579.95 MiB      160 MiB/s 0s
+  ../audio-files/moto-maxx/2018.04.17 1438.wav changed while it was being added
+failed                            
+fatal: not a git repository: '../.git'
+error: unknown option `cached'
+usage: git diff --no-index [<options>] <path> <path>
+...
+[the entire help message]
+...
+--find-object <object-id>
+                          look for differences that change the number of occurrences of the specified object
+    --diff-filter [(A|C|D|M|R|T|U|X|B)...[*]]
+                          select files by diff type
+    --output <file>       Output to a specific file
+
+(recording state in git...)
+fatal: not a git repository: '../.git'
+^Ceral-pathspecs","add","--"] exited 123)
+
+```
+
+The file that was being added was left like this:
+
+ll "audio-files/moto-maxx/2018.04.17 1438.wav"
+
+-r--r--r-- 2 ### ### 608121644 abr 17  2018 'audio-files/moto-maxx/2018.04.17 1438.wav'
+
+It appears git-annex just chmoded it, but didn't symlinkify it.
+
+### What version of git-annex are you using? On what operating system?
+
+Ubuntu 20.04 focal
+
+Linux 5.8.0-050800-generic #202008022230 SMP Sun Aug 2 22:33:21 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
+
+git-annex version: 8.20200226
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+I am absolutely in love with it. I even built a special remote for it and I'm now building another because that one was stupid.
+
+Seriously, git-annex is amazing.

diff --git a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_2_789445dff9a5905ed105ac344f6949ad._comment b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_2_789445dff9a5905ed105ac344f6949ad._comment
new file mode 100644
index 000000000..e32342de1
--- /dev/null
+++ b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_2_789445dff9a5905ed105ac344f6949ad._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="cardoso-neto"
+ avatar="http://cdn.libravatar.org/avatar/d90a656df072f3a29da54302c190c696"
+ subject="Improperly documented code."
+ date="2020-08-06T02:52:50Z"
+ content="""
+I found out how to do it after reading his source code for the [Google Drive remote](github.com/Lykos153/git-annex-remote-gdrive).
+
+Feel free to have a look around my remotes <github.com/cardoso-neto/git-annex-remote-manytars> <github.com/cardoso-neto/git-annex-remote-manyzips> which use his [helper module](github.com/Lykos153/AnnexRemote).
+
+To summarize it, you should override __init__ and list your configs there.
+"""]]

diff --git a/doc/thanks/list b/doc/thanks/list
index 963261fe6..7712c1e42 100644
--- a/doc/thanks/list
+++ b/doc/thanks/list
@@ -99,3 +99,7 @@ Timothy Schumann,
 Mark Reidenbach, 
 Martin D, 
 James Greenaway, 
+Kanak Kshetri, 
+Andrew Gilbert, 
+Noam Kremen, 
+Pluralist Extremist, 

diff --git a/Annex/UntrustedFilePath.hs b/Annex/UntrustedFilePath.hs
index 2ec37842b..33233a6de 100644
--- a/Annex/UntrustedFilePath.hs
+++ b/Annex/UntrustedFilePath.hs
@@ -31,17 +31,26 @@ import System.FilePath
  - that case.
  -}
 sanitizeFilePath :: String -> FilePath
-sanitizeFilePath [] = "file"
-sanitizeFilePath f = leading (map sanitize f)
+sanitizeFilePath = sanitizeLeadingFilePathCharacter . sanitizeFilePathComponent
+
+{- For when the filepath is being built up out of components that should be
+ - individually sanitized, this can be used for each component, followed by
+ - sanitizeLeadingFilePathCharacter for the whole thing.
+ -}
+sanitizeFilePathComponent :: String -> String
+sanitizeFilePathComponent = map sanitize
   where
 	sanitize c
 		| c == '.' || c == '-' = c
 		| isSpace c || isPunctuation c || isSymbol c || isControl c || c == '/' = '_'
 		| otherwise = c
 
-	leading ('.':s) = '_':s
-	leading ('-':s) = '_':s
-	leading s = s
+sanitizeLeadingFilePathCharacter :: String -> FilePath
+sanitizeLeadingFilePathCharacter [] = "file"
+sanitizeLeadingFilePathCharacter ('.':s) = '_':s
+sanitizeLeadingFilePathCharacter ('-':s) = '_':s
+sanitizeLeadingFilePathCharacter ('/':s) = '_':s
+sanitizeLeadingFilePathCharacter s = s
 
 escapeSequenceInFilePath :: FilePath -> Bool
 escapeSequenceInFilePath f = '\ESC' `elem` f
diff --git a/CHANGELOG b/CHANGELOG
index c9d086278..2c856674f 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -18,6 +18,8 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
     standalone and OSX app.
     Thanks, Yaroslav Halchenko
   * Slightly sped up the linux standalone bundle.
+  * importfeed: Fix reversion that caused some '.' in filenames to be
+    replaced with '_'
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/Command/ImportFeed.hs b/Command/ImportFeed.hs
index e292ab73e..cd6415369 100644
--- a/Command/ImportFeed.hs
+++ b/Command/ImportFeed.hs
@@ -338,17 +338,18 @@ defaultTemplate = "${feedtitle}/${itemtitle}${extension}"
 {- Generates a filename to use for a feed item by filling out the template.
  - The filename may not be unique. -}
 feedFile :: Utility.Format.Format -> ToDownload -> String -> FilePath
-feedFile tmpl i extension = Utility.Format.format tmpl $
-	M.map sanitizeFilePath $ M.fromList $ extractFields i ++
-		[ ("extension", extension)
-		, extractField "itempubdate" [itempubdate]
-		, extractField "itempubyear" [itempubyear]
-		, extractField "itempubmonth" [itempubmonth]
-		, extractField "itempubday" [itempubday]
-		, extractField "itempubhour" [itempubhour]
-		, extractField "itempubminute" [itempubminute]
-		, extractField "itempubsecond" [itempubsecond]
-		]
+feedFile tmpl i extension = sanitizeLeadingFilePathCharacter $ 
+	Utility.Format.format tmpl $
+		M.map sanitizeFilePathComponent $ M.fromList $ extractFields i ++
+			[ ("extension", extension)
+			, extractField "itempubdate" [itempubdate]
+			, extractField "itempubyear" [itempubyear]
+			, extractField "itempubmonth" [itempubmonth]
+			, extractField "itempubday" [itempubday]
+			, extractField "itempubhour" [itempubhour]
+			, extractField "itempubminute" [itempubminute]
+			, extractField "itempubsecond" [itempubsecond]
+			]
   where
 	itm = item i
 
diff --git a/doc/bugs/Importfeed_replaces_all___34__.__34___characters_with___34____95____34___in_filename.mdwn b/doc/bugs/Importfeed_replaces_all___34__.__34___characters_with___34____95____34___in_filename.mdwn
index 0bb6d6044..db444ff43 100644
--- a/doc/bugs/Importfeed_replaces_all___34__.__34___characters_with___34____95____34___in_filename.mdwn
+++ b/doc/bugs/Importfeed_replaces_all___34__.__34___characters_with___34____95____34___in_filename.mdwn
@@ -28,3 +28,4 @@ git annex version 8.20200720.1-g1ccb6699a1
 
 
 
+> [[fixed|done]] --[[Joey]]

diff --git a/doc/bugs/Importfeed_replaces_all___34__.__34___characters_with___34____95____34___in_filename.mdwn b/doc/bugs/Importfeed_replaces_all___34__.__34___characters_with___34____95____34___in_filename.mdwn
new file mode 100644
index 000000000..0bb6d6044
--- /dev/null
+++ b/doc/bugs/Importfeed_replaces_all___34__.__34___characters_with___34____95____34___in_filename.mdwn
@@ -0,0 +1,30 @@
+### Please describe the problem.
+
+Beginning with 8.20200522, importfeed replaces all dot (".") characters in filenames with "_". As an example, before that release, the TWIT Security Now podcast had filenames that looked like this:
+
+SN_767__WiFi_6.mp3
+
+After the release, the filenames look like this:
+
+SN_769__Zoom_s_E2EE_Design_mp3
+
+This change occurred across all my downloaded podcasts. 
+
+I suspect the bug has something to do with the following changes described in the news for 8.20200522:
+
+- addurl, importfeed: Avoid adding filenames with leading '.', instead it will be replaced with '_'.
+- addurl, importfeed: Allow '-' in filenames, as long as it's not the first character.
+
+
+### What steps will reproduce the problem?
+
+git-annex importfeed --fast http://leoville.tv/podcasts/sn.xml
+
+### What version of git-annex are you using? On what operating system?
+
+Arch Linux
+
+git annex version 8.20200720.1-g1ccb6699a1
+
+
+

diff --git a/doc/todo/speed_up_git_annex_sync_--content_--all/comment_12_6e8863e4d039a6eb97751a9a58149151._comment b/doc/todo/speed_up_git_annex_sync_--content_--all/comment_12_6e8863e4d039a6eb97751a9a58149151._comment
new file mode 100644
index 000000000..71a44fb04
--- /dev/null
+++ b/doc/todo/speed_up_git_annex_sync_--content_--all/comment_12_6e8863e4d039a6eb97751a9a58149151._comment
@@ -0,0 +1,36 @@
+[[!comment format=mdwn
+ username="Lukey"
+ avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
+ subject="comment 12"
+ date="2020-08-04T14:10:32Z"
+ content="""
+I finally got around to test this and it is a lot faster. Interestingly it now -J2 is slower than without jobs.
+Benchmark with warm cache:
+
+    git annex sync --content --all
+    
+    8.20200330:
+    real    23m48.059s
+    user    20m23.320s
+    sys     3m20.854s
+    
+    8.20200720.2-g465842f3f
+    real    3m41.064s
+    user    3m23.319s
+    sys     0m49.159s
+
+
+    git annex sync --content --all -J2
+    
+    8.20200330:
+    real    10m47.484s
+    user    16m3.024s
+    sys     1m49.373s
+    
+    8.20200720.2-g465842f3f
+    real    8m55.808s
+    user    12m44.766s
+    sys     1m25.181s
+
+Thanks!
+"""]]

diff --git a/Build/LinuxMkLibs.hs b/Build/LinuxMkLibs.hs
index 930107089..a873d76ad 100644
--- a/Build/LinuxMkLibs.hs
+++ b/Build/LinuxMkLibs.hs
@@ -24,7 +24,7 @@ import Utility.Path
 import Utility.FileMode
 import Utility.CopyFile
 
-mklibs :: FilePath -> a -> IO ()
+mklibs :: FilePath -> a -> IO Bool
 mklibs top _installedbins = do
 	fs <- dirContentsRecursive top
 	exes <- filterM checkExe fs
@@ -48,6 +48,19 @@ mklibs top _installedbins = do
 	let linker = Prelude.head linkers
 	mapM_ (installLinkerShim top linker) exes
 	
+	return (any hwcaplibdir libdirs)
+  where
+	-- hwcap lib dirs are things like foo/tls and foo/x86.
+	-- Hard to know if a directory is, so this is a heuristic
+	-- looking for things that are certianly not. If this heuristic
+	-- fails, a minor optimisation will not happen, but there will be
+	-- no bad results.
+	hwcaplibdir d = not $ or
+		[ "lib" == takeFileName d
+		-- eg, "lib/x86_64-linux-gnu"
+		, "-linux-" `isInfixOf` takeFileName d
+		]
+
 {- If there are two libdirs that are the same except one is in
  - usr/lib and the other is in lib/, move the contents of the usr/lib one
  - into the lib/ one. This reduces the number of directories the linker
diff --git a/Build/OSXMkLibs.hs b/Build/OSXMkLibs.hs
index 39e7b16ca..4f7271d98 100644
--- a/Build/OSXMkLibs.hs
+++ b/Build/OSXMkLibs.hs
@@ -31,8 +31,10 @@ import qualified Data.Set as S
 
 type LibMap = M.Map FilePath String
 
-mklibs :: FilePath -> M.Map FilePath FilePath -> IO ()
-mklibs appbase installedbins = mklibs' appbase installedbins [] [] M.empty
+mklibs :: FilePath -> M.Map FilePath FilePath -> IO Bool
+mklibs appbase installedbins = do
+	mklibs' appbase installedbins [] [] M.empty
+	return True
 
 {- Recursively find and install libs, until nothing new to install is found. -}
 mklibs' :: FilePath -> M.Map FilePath FilePath -> [FilePath] -> [(FilePath, FilePath)] -> LibMap -> IO ()
diff --git a/Build/Standalone.hs b/Build/Standalone.hs
index 4353c7aa3..f8c43242c 100644
--- a/Build/Standalone.hs
+++ b/Build/Standalone.hs
@@ -152,13 +152,22 @@ installLocales _ = return ()
 installLocales topdir = cp "/usr/share/i18n" (topdir </> "i18n")
 #endif
 
-installWrapper :: FilePath -> FilePath -> IO ()
+installSkel :: FilePath -> FilePath -> IO ()
 #ifdef darwin_HOST_OS
-installWrapper topdir basedir = do
+installSkel topdir basedir = do
 	removeDirectoryRecursive basedir
-	createDirectoryIfMissing True (takeDirectory basedir)
 	unlessM (boolSystem "cp" [Param "-R", File "standalone/osx/git-annex.app", File basedir]) $
 		error "cp failed"
+#else
+installSkel topdir _basedir = do
+	removeDirectoryRecursive topdir
+	unlessM (boolSystem "cp" [Param "-R", File "standalone/linux/skel", File topdir]) $
+		error "cp failed"
+#endif
+
+installSkelRest :: FilePath -> FilePath -> Bool -> IO ()
+#ifdef darwin_HOST_OS
+installSkelRest topdir basedir _hwcaplibs = do
 	plist <- lines <$> readFile "standalone/osx/Info.plist.template"
 	version <- getVersion
 	writeFile (basedir </> "Contents" </> "Info.plist")
@@ -166,11 +175,7 @@ installWrapper topdir basedir = do
   where
 	expandversion v l = replace "GIT_ANNEX_VERSION" v l
 #else
-installWrapper topdir _basedir = do
-	removeDirectoryRecursive topdir
-	createDirectoryIfMissing True (takeDirectory topdir)
-	unlessM (boolSystem "cp" [Param "-R", File "standalone/linux/skel", File topdir]) $
-		error "cp failed"
+installSkelRest topdir _basedir hwcaplibs = do
 	runshell <- lines <$> readFile "standalone/linux/skel/runshell"
 	-- GIT_ANNEX_PACKAGE_INSTALL can be set by a distributor and
 	-- runshell will be modified
@@ -180,6 +185,12 @@ installWrapper topdir _basedir = do
 	modifyFileMode (topdir </> "runshell") (addModes executeModes)
   where
 	expandrunshell (Just gapi) l@"GIT_ANNEX_PACKAGE_INSTALL=" = l ++ gapi
+	-- This is an optimisation, that avoids the linker looking in
+	-- several directories for hwcap optimised libs, when there are
+	-- none.
+	expandrunshell _ l@"LD_HWCAP_MASK=" = l ++ if not hwcaplibs
+		then "0"
+		else ""
 	expandrunshell _ l = l
 #endif
 
@@ -203,11 +214,12 @@ main :: IO ()
 main = getArgs >>= go
   where
 	go (topdir:basedir:[]) = do
-		installWrapper topdir basedir
+		installSkel topdir basedir
 		installGitAnnex topdir
 		installedbins <- installBundledPrograms topdir
 		installGitLibs topdir
 		installMagic topdir
 		installLocales topdir
-		mklibs topdir installedbins
+		hwcaplibs <- mklibs topdir installedbins
+		installSkelRest topdir basedir hwcaplibs
 	go _ = error "specify topdir and basedir"
diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__.mdwn b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__.mdwn
index 6da8dc6d8..96fc9e0d2 100644
--- a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__.mdwn
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__.mdwn
@@ -61,3 +61,5 @@ git annex build: 8.20200617+git192-g5849bd634-1~ndall+1
 
 [[!meta author=yoh]]
 [[!tag projects/datalad]]
+
+> [[done]] --[[Joey]]
diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_19_13b21a0523b8837f4e27bb78e35ad5e0._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_19_13b21a0523b8837f4e27bb78e35ad5e0._comment
new file mode 100644
index 000000000..a22e02914
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_19_13b21a0523b8837f4e27bb78e35ad5e0._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 19"""
+ date="2020-08-03T18:16:50Z"
+ content="""
+Implemented the `LD_HWCAP_MASK=0` optimisation, which left only these:
+
+	strace -f ./git-annex version 2>&1 | grep ENOENT | grep openat
+	openat(AT_FDCWD, "/home/joey/src/git-annex/tmp/git-annex.linux//lib/x86_64-linux-gnu/tls/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/home/joey/src/git-annex/tmp/git-annex.linux//lib/x86_64-linux-gnu/tls/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/home/joey/src/git-annex/tmp/git-annex.linux//lib/x86_64-linux-gnu/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 697752] openat(AT_FDCWD, "/home/joey/src/git-annex/tmp/git-annex.linux//lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 697752] openat(AT_FDCWD, "/home/joey/src/git-annex/tmp/git-annex.linux//lib/x86_64-linux-gnu/tls/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 697752] openat(AT_FDCWD, "/home/joey/src/git-annex/tmp/git-annex.linux//lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+
+There are more failed opens now for locale files for commands like
+grep when running it than there are for libraries. So, no need to
+consider further prelinking.
+
+I think that rewriting runshell in C would be the
+logical next choice, but dunno if it would speed it up by enough to be 
+worth the effort. So I'm going to close this now.
+"""]]
diff --git a/standalone/linux/skel/runshell b/standalone/linux/skel/runshell
index 63bc27751..e18769cea 100755
--- a/standalone/linux/skel/runshell
+++ b/standalone/linux/skel/runshell
@@ -241,6 +241,9 @@ else
 	cmd=sh
 fi
 
+LD_HWCAP_MASK=
+export LD_HWCAP_MASK
+
 if [ -n "${orig_IFS}" ]; then
 	IFS="${orig_IFS}"
 	export IFS

diff --git a/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_8_3122eeb1f540bdde320beb2eda196221._comment b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_8_3122eeb1f540bdde320beb2eda196221._comment
new file mode 100644
index 000000000..cc46c7df0
--- /dev/null
+++ b/doc/bugs/gcrypt_remote__58___every_sync_uploads_huge_manifest/comment_8_3122eeb1f540bdde320beb2eda196221._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="git-annex.branchable.com@79d6855760f61f7fbe0a401b45d8c791ef49b500"
+ nickname="git-annex.branchable.com"
+ avatar="http://cdn.libravatar.org/avatar/4bf61f9feda20e8b4fc09d52ee48af39"
+ subject="comment 8"
+ date="2020-08-02T23:41:32Z"
+ content="""
+Is it possible to convert an existing `gcrypt::ssh://host/path` gcrypt special remote to one on top of rsync transport protocol to work around these performance issues?  The existing remote is obviously quite large, and re-uploading the content would take a very long time.
+"""]]

diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_4_297b462c0344b8c7e5e113fdda3642f4._comment b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_4_297b462c0344b8c7e5e113fdda3642f4._comment
new file mode 100644
index 000000000..290de6e99
--- /dev/null
+++ b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_4_297b462c0344b8c7e5e113fdda3642f4._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 4"
+ date="2020-08-02T01:31:52Z"
+ content="""
+Oh, I see the reason for confusion: I usually as `set -u` to my scripts, so they would fail if variable is undefined. Indeed in the default mode of operation it isn't needed.
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_18_7650d349ba50972b64cd6d81673bf1c9._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_18_7650d349ba50972b64cd6d81673bf1c9._comment
new file mode 100644
index 000000000..0551aac62
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_18_7650d349ba50972b64cd6d81673bf1c9._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 18"
+ date="2020-07-31T21:48:19Z"
+ content="""
+FWIW, [it did drop from 90 to 15!](https://github.com/datalad/datalad-extensions/pull/25/checks?check_run_id=933718043#step:4:1) so there is progress ;-)
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_17_77a2f7d18cbcf780d8e2bb4a3e40b4fa._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_17_77a2f7d18cbcf780d8e2bb4a3e40b4fa._comment
new file mode 100644
index 000000000..25ce4863f
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_17_77a2f7d18cbcf780d8e2bb4a3e40b4fa._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 17"
+ date="2020-07-31T21:37:45Z"
+ content="""
+> ... that there are no hwcap optimised libs being included in the bundle, otherwise it would probably be a pessimisation.
+
+Kyle, get ready for me to use a new word I just have learned! ;)
+
+as for \"... included in the bundle\".  For neurodebian git-annex-standalone ideally any check of such kinda should be done at package build time since it is when it could check just once and avoid any checks later when installed.
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_16_c43788c8730619c3590d1a0a2bb87148._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_16_c43788c8730619c3590d1a0a2bb87148._comment
new file mode 100644
index 000000000..bcb2af0e7
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_16_c43788c8730619c3590d1a0a2bb87148._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="kyle"
+ avatar="http://cdn.libravatar.org/avatar/7d6e85cde1422ad60607c87fa87c63f3"
+ subject="comment 16"
+ date="2020-07-31T21:30:31Z"
+ content="""
+> FTR: testing of git annex started to show some failures 2 days ago
+
+Already reported [here] and fixed by Joey with 5a5873e05 (fix bug
+caught by test suite, 2020-07-31).
+
+[here]: https://git-annex.branchable.com/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes/
+
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_15_013afa9d45970397bfcd60e57a519d67._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_15_013afa9d45970397bfcd60e57a519d67._comment
new file mode 100644
index 000000000..fab2af08a
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_15_013afa9d45970397bfcd60e57a519d67._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 15"
+ date="2020-07-31T21:23:55Z"
+ content="""
+Just a brief one:
+
+> It is premature optimisation to try to reduce these seeks when they have not been shown to reduce performance ...
+
+Have a look at the original issue description.  They introduced 1.3ms to seek a single library... We saved about 30% of runtime switching Travis CI from standalone to conda build of annex. So performance reduction is obvious IMHO.
+
+>  Implemented that.
+
+Awesome!  I have restarted [gh workflows job](https://github.com/datalad/datalad-extensions/pull/25/checks?check_run_id=919624426) which runs perspectively [added test](https://github.com/datalad/datalad-extensions/pull/25/files#diff-8364c688b76bfaf5df947cfd4d74eef7R111) running `annex init` and counting missing seeks for libpcre (was over 90)... unfortunately regular `git annex test` failed before it tried added \"custom\" ones (I should isolate them I guess), so dunno what is the status on that github action.  
+FTR: testing of git annex started to show some failures 2 days ago : https://github.com/datalad/datalad-extensions/actions/runs/187943962 (although only in crippled fs runs, although might be just the sign of lakiness)  so not necessarily a fresh one.  yet to download a fresh built package and see what it brings locally
+"""]]

diff --git a/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes/comment_1_7f23940cfa4451e4cd7137e2d7a46d86._comment b/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes/comment_1_7f23940cfa4451e4cd7137e2d7a46d86._comment
new file mode 100644
index 000000000..9e646fccb
--- /dev/null
+++ b/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes/comment_1_7f23940cfa4451e4cd7137e2d7a46d86._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="kyle"
+ avatar="http://cdn.libravatar.org/avatar/7d6e85cde1422ad60607c87fa87c63f3"
+ subject="comment 1"
+ date="2020-07-31T21:20:08Z"
+ content="""
+> -		in if S.last b' == fromIntegral (ord 'E')
+> +		in if not (S.null b') && S.last b' == fromIntegral (ord 'E')
+
+I had a feeling this could be a one-liner, but I wasn't able to come
+up with a patch.  Shoot!
+
+Thanks for the fix.
+
+
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_14_16aae09d0e1d80240fc3ae81e5b06972._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_14_16aae09d0e1d80240fc3ae81e5b06972._comment
new file mode 100644
index 000000000..2d0f855a9
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_14_16aae09d0e1d80240fc3ae81e5b06972._comment
@@ -0,0 +1,45 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 14"""
+ date="2020-07-31T20:25:01Z"
+ content="""
+Setting `LD_HWCAP_MASK=0` prevents the linker from looking for hardware
+optimised libraries. I compared with and without it, and there was a
+savings of 280 failed seeks across `git-annex init`.
+
+In particular ones with a double `x86_64` like this one:
+
+	git-annex.linux//lib/x86_64-linux-gnu/tls/x86_64/x86_64/libm.so.6
+
+While it still does ones like this, which I would have thought would also
+be disabled:
+
+	git-annex.linux//lib/x86_64-linux-gnu/tls/x86_64/libm.so.6
+
+And, it eliminates looking for libpcre twice in the same place
+(git-annex.linux//lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0)
+that it otherwise does while linking git.
+
+This is some weird behavior from the linker. I get the impression
+that it's looking in `x86_64` for two different reasons, hwcap and
+something else. And tls is not being filtered either.
+Found this message that I think explains why:
+<https://sourceware.org/pipermail/libc-alpha/2020-May/113878.html>
+
+> Two fake hwcap bits and corresponding subdirectory are added by the loader: the TLS
+> bit with the directory name "tls", and the platform bit, with the
+> AT_PLATFROM string provided by the kernel as the directory name.
+
+So, the kernel provides hwcaps, which we can filter, but then these
+fake ones are added on top of it.
+
+It seems likely that's a bug... What if I actually had a good
+reason to want to mask out those libraries from being used, and the linker
+used them anyway?
+
+Anyway, that reduces the runtime to 0.08-0.09s from 0.10-0.11s.
+
+Enabling this would need some way to detect that there are no hwcap
+optimised libs being included in the bundle, otherwise it would probably
+be a pessimisation. 
+"""]]

diff --git a/Types/Key.hs b/Types/Key.hs
index a4f49870d..7fd702046 100644
--- a/Types/Key.hs
+++ b/Types/Key.hs
@@ -344,7 +344,7 @@ parseKeyVariety "URL"          = URLKey
 parseKeyVariety b
 	| "X" `S.isPrefixOf` b = 
 		let b' = S.tail b
-		in if S.last b' == fromIntegral (ord 'E')
+		in if not (S.null b') && S.last b' == fromIntegral (ord 'E')
 			then ExternalKey (S.init b') (HasExt True)
 			else ExternalKey b' (HasExt False)
 	| otherwise = OtherKey b
diff --git a/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes.mdwn b/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes.mdwn
index 144f76a82..d3b596667 100644
--- a/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes.mdwn
+++ b/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes.mdwn
@@ -33,3 +33,5 @@ Thanks in advance.
 
 [[!meta author=kyle]]
 [[!tag projects/datalad]]
+
+> [[fixed|done]] and rest of test suite passing --[[Joey]]

diff --git a/doc/design/external_backend_protocol/comment_2_e6005a7563a65c0fd4615cc3f2c3034b._comment b/doc/design/external_backend_protocol/comment_2_e6005a7563a65c0fd4615cc3f2c3034b._comment
new file mode 100644
index 000000000..5ff341183
--- /dev/null
+++ b/doc/design/external_backend_protocol/comment_2_e6005a7563a65c0fd4615cc3f2c3034b._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-07-31T20:05:42Z"
+ content="""
+SHA512 backend has 128 bytes key name, that's where I got that suggestion
+from. Some filesystems have limits around 255 bytes for the name of a file,
+so that leaves plenty for extension, and the rest of the parts of the key.
+Realistically, the length of a SHA256 is a better goal.
+
+Of course, if you had a crazy 1025 byte hash and wanted to use it on IDK,
+GNU Hurd or something, you could do it, but your repo
+would not be portable to eg Linux with its 1024 byte filename limit.
+git-annex itself does not care though, and I *think* git would also not
+care.
+"""]]

diff --git a/Build/LinuxMkLibs.hs b/Build/LinuxMkLibs.hs
index b62fc762b..1041b7ab1 100644
--- a/Build/LinuxMkLibs.hs
+++ b/Build/LinuxMkLibs.hs
@@ -36,18 +36,51 @@ mklibs top = do
 	fs <- dirContentsRecursive top
 	exes <- filterM checkExe fs
 	libs <- parseLdd <$> readProcess "ldd" exes
+	
 	glibclibs <- glibcLibs
 	let libs' = nub $ libs ++ glibclibs
-	libdirs <- nub . catMaybes <$> mapM (installLib installFile top) libs'
+	let (linkers, otherlibs) = partition ("ld-linux" `isInfixOf`) libs'
+	libdirs <- nub . catMaybes <$> mapM (installLib installFile top) otherlibs
+	libdirs' <- consolidateUsrLib top libdirs
+
+	gconvlibs <- gconvLibs
+	mapM_ (installLib installFile top) gconvlibs
 
 	-- Various files used by runshell to set up env vars used by the
 	-- linker shims.
-	writeFile (top </> "libdirs") (unlines libdirs)
-	writeFile (top </> "gconvdir")
-		(parentDir $ Prelude.head $ filter ("/gconv/" `isInfixOf`) glibclibs)
+	writeFile (top </> "libdirs") (unlines libdirs')
+	writeFile (top </> "gconvdir") (parentDir $ Prelude.head gconvlibs)
 	
-	let linker = Prelude.head $ filter ("ld-linux" `isInfixOf`) libs'
+	mapM_ (installLib installFile top) linkers
+	let linker = Prelude.head linkers
 	mapM_ (installLinkerShim top linker) exes
+	
+{- If there are two libdirs that are the same except one is in
+ - usr/lib and the other is in lib/, move the contents of the usr/lib one
+ - into the lib/ one. This reduces the number of directories the linker
+ - needs to look in, and so reduces the number of failed stats
+ - and improves startup time.
+ -}
+consolidateUsrLib :: FilePath -> [FilePath] -> IO [FilePath]
+consolidateUsrLib top libdirs = map reverse <$> go [] (map reverse libdirs)
+  where
+	go c [] = return c
+	go c (x:[]) = return (x:c)
+	go c (x:y:rest)
+		| x == y ++ reverse ("/usr") = do
+			let x' = reverse x
+			let y' = reverse y
+			fs <- getDirectoryContents (inTop top x')
+			forM_ fs $ \f -> do
+				print f
+				unless (dirCruft f) $
+					renameFile 
+						(inTop top (x' </> f))
+						(inTop top (y' </> f))
+			go (y:c) rest
+		| otherwise = do
+			print (x,y)
+			go (x:c) (y:rest)
 
 {- Installs a linker shim script around a binary.
  -
diff --git a/CHANGELOG b/CHANGELOG
index 2181a1b4d..c9d086278 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -17,6 +17,7 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
   * Deal with unusual IFS settings in the shell scripts for linux
     standalone and OSX app.
     Thanks, Yaroslav Halchenko
+  * Slightly sped up the linux standalone bundle.
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/Utility/LinuxMkLibs.hs b/Utility/LinuxMkLibs.hs
index 8782ead6b..6115023a0 100644
--- a/Utility/LinuxMkLibs.hs
+++ b/Utility/LinuxMkLibs.hs
@@ -1,6 +1,6 @@
 {- Linux library copier and binary shimmer
  -
- - Copyright 2013 Joey Hess <id@joeyh.name>
+ - Copyright 2013-2020 Joey Hess <id@joeyh.name>
  -
  - License: BSD-2-clause
  -}
@@ -9,6 +9,7 @@ module Utility.LinuxMkLibs (
 	installLib,
 	parseLdd,
 	glibcLibs,
+	gconvLibs,
 	inTop,
 ) where
 
@@ -59,9 +60,15 @@ parseLdd = mapMaybe (getlib . dropWhile isSpace) . lines
   where
 	getlib l = headMaybe . words =<< lastMaybe (split " => " l)
 
-{- Get all glibc libs and other support files, including gconv files
+{- Get all glibc libs.
  -
  - XXX Debian specific. -}
 glibcLibs :: IO [FilePath]
 glibcLibs = lines <$> readProcess "sh"
-	["-c", "dpkg -L libc6:$(dpkg --print-architecture) libgcc1:$(dpkg --print-architecture) | egrep '\\.so|gconv'"]
+	["-c", "dpkg -L libc6:$(dpkg --print-architecture) libgcc1:$(dpkg --print-architecture) | egrep '\\.so' | grep -v /gconv/ | grep -v ld.so.conf | grep -v sotruss-lib"]
+
+{- Get gblibc's gconv libs, which are handled specially.. -}
+gconvLibs :: IO [FilePath]
+gconvLibs = lines <$> readProcess "sh"
+	["-c", "dpkg -L libc6:$(dpkg --print-architecture) | grep /gconv/"]
+
diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_13_5d65192366f7690c22d1a209fb1e8f78._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_13_5d65192366f7690c22d1a209fb1e8f78._comment
new file mode 100644
index 000000000..b9f24ce52
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_13_5d65192366f7690c22d1a209fb1e8f78._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 13"""
+ date="2020-07-31T17:41:55Z"
+ content="""
+`git-annex init` runs git something like 30 times, so it's close to the worst
+case for a single git-annex command, other than when smudge filters are run.
+
+I tried inlining runshell into both git and git-annex scripts, thinking
+that the overhead of starting the second shell script might be measurable.
+It was not; I saw `git-annex init` taking 0.10-0.14s before and after.
+
+I also tried trimming out some parts of the script that normally
+don't run, like the android support, but that didn't speed it up.
+
+With the consolidated lib dirs, I did see `git-annex init` drop to 0.07-0.10s.
+
+Ok, I found a way to consolidate the directories that will not include
+directories for optimised libs. Implemented that.
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_12_2363dac7701b9c36eb1896a09e68c17d._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_12_2363dac7701b9c36eb1896a09e68c17d._comment
new file mode 100644
index 000000000..f7eafe1ec
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_12_2363dac7701b9c36eb1896a09e68c17d._comment
@@ -0,0 +1,35 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 12"""
+ date="2020-07-31T17:10:08Z"
+ content="""
+It is premature optimisation to try to reduce these seeks when they have
+not been shown to reduce performance, and at least in my tests, have been
+shown to not affect it to the limits I can measure.
+
+I don't think it would be safe to consolidate all the libs into one
+directory, as I did in my test. Because it seems possible that there could
+be two versions of a .so file with the same name, in eg lib64/tls
+and lib64/x86_64. Installing only one of them would either lose actual
+optimisations in the other .so file, or cause breakage. I don't know when
+those optimised .so files exist, but implementating this consolidation
+would make the build more fragile.
+
+I don't know if `prelink(8)` is able to prelink things such that they can
+be relocated, as git-annex.linux can. I think it hardcodes a fixed path in
+the binary. Similarly, ld.so.cache (which is why this seeking doesn't
+happen on Debian AFAICS, not preliking) contains a list of directories and
+the libraries in them, so unless it were created by runshell the first
+time, it would not work. And creating it by runshell the first time would
+make the thing more complicated, and thus more fragile. (Also
+/sbin/ldconfig is 1mb in size so would increase the bundle rather a lot.)
+Also, I benchmarked prelinking back in 
+[[!commit c4229be9a7a2318ef71b9ae433bc14bf604c9caf]] and the speedup was not
+measurable.
+
+So these are not appealing with the information I have. It's a very
+different situation than the ghc bug that was adding one directory
+to rpath for every haskell library, which had an easily measurable
+performance impact because there are hundreds of those libraries and 30000
+seeks did add up to a measurable time.
+"""]]

diff --git a/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes.mdwn b/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes.mdwn
new file mode 100644
index 000000000..144f76a82
--- /dev/null
+++ b/doc/bugs/prop__95__isomorphic__95__key__95__encode_fails_with_external_backend_changes.mdwn
@@ -0,0 +1,35 @@
+Just a heads up that `prop_isomorphic_key_encode` is failing on master
+(for me locally and at <https://github.com/datalad/datalad-extensions>):
+
+```
+$ git show --no-patch --format=reference
+85e57d826 (Added a comment: streaming data and external backends, 2020-07-30)
+
+$ stack exec -- git annex test -p prop_isomorphic_key_encode
+Tests                     
+  QuickCheck
+    prop_isomorphic_key_encode: FAIL (0.03s)
+      *** Failed! (after 601 tests):
+      Exception:
+        Data.ByteString.last: empty ByteString
+        CallStack (from HasCallStack):
+          error, called at libraries/bytestring/Data/ByteString.hs:1877:23 in bytestring-0.10.8.2:Data.ByteString
+      Exception thrown while showing test case:
+        Data.ByteString.last: empty ByteString
+        CallStack (from HasCallStack):
+          error, called at libraries/bytestring/Data/ByteString.hs:1877:23 in bytestring-0.10.8.2:Data.ByteString
+      
+      Use --quickcheck-replay=896100 to reproduce.
+
+1 out of 1 tests failed (0.03s)
+  (Failures above could be due to a bug in git-annex, or an incompatibility
+   with utilities, such as git, installed on this system.)
+```
+
+I do not see this failure before f75be3216 (external backends wip,
+2020-07-29).
+
+Thanks in advance.
+
+[[!meta author=kyle]]
+[[!tag projects/datalad]]

diff --git a/doc/todo/external_backends/comment_15_7880557cb94706d82a6d2bfc785288ea._comment b/doc/todo/external_backends/comment_15_7880557cb94706d82a6d2bfc785288ea._comment
new file mode 100644
index 000000000..322e52f08
--- /dev/null
+++ b/doc/todo/external_backends/comment_15_7880557cb94706d82a6d2bfc785288ea._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="streaming data and external backends"
+ date="2020-07-30T15:58:21Z"
+ content="""
+\"I can't think of any situation where git-annex would GENKEY before it has the full content of a file available\" -- if [[todo/git-annex-cat]] were implemented, could then e.g. `git fsck --from REMOTE` without the data ever touching the disk.
+"""]]

diff --git a/doc/devblog/day_628__external_key_backends/comment_1_e6ff184245dd1a6cffff2ecdaeff6a66._comment b/doc/devblog/day_628__external_key_backends/comment_1_e6ff184245dd1a6cffff2ecdaeff6a66._comment
new file mode 100644
index 000000000..09126d2db
--- /dev/null
+++ b/doc/devblog/day_628__external_key_backends/comment_1_e6ff184245dd1a6cffff2ecdaeff6a66._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="thanks"
+ date="2020-07-30T15:51:13Z"
+ content="""
+Thanks a lot Joey for implementing this.
+
+\"only had to write around 400 lines of new code\" -- sign of good design.  At some point I'll learn enough Haskell to understand/learn from git-annex's design.
+
+"""]]

diff --git a/doc/design/external_backend_protocol/comment_1_7a55199030c3b9f455354139af7dbac0._comment b/doc/design/external_backend_protocol/comment_1_7a55199030c3b9f455354139af7dbac0._comment
new file mode 100644
index 000000000..09d889d80
--- /dev/null
+++ b/doc/design/external_backend_protocol/comment_1_7a55199030c3b9f455354139af7dbac0._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="key length"
+ date="2020-07-30T15:13:49Z"
+ content="""
+\"128 bytes maximum\" key length -- maybe a bit shorter to support the E variant?
+"""]]

diff --git a/doc/devblog/day_628__external_key_backends.mdwn b/doc/devblog/day_628__external_key_backends.mdwn
new file mode 100644
index 000000000..a10939b32
--- /dev/null
+++ b/doc/devblog/day_628__external_key_backends.mdwn
@@ -0,0 +1,10 @@
+Today I implemented [[external backends|design/external_backend_protocol]]
+for keys. So unusual new hashes can be used by writing a small program.
+
+Probably lots of other uses for this too; I don't know if I'll like
+them all. It has the potential to warp git-annex in some directions I don't
+want to deal with. Still, it's good to have this feature.
+
+I was able to reuse a lot of the external special remote code for this,
+and only had to write around 400 lines of new code. Dunno how that all
+happened in 8 hours, but it did!

diff --git a/CHANGELOG b/CHANGELOG
index 4f4e5e97c..2181a1b4d 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,5 +1,8 @@
 git-annex (8.20200720.2) UNRELEASED; urgency=medium
 
+  * Added support for external backend programs. So if you want a hash
+    that git-annex doesn't support, or something stranger, you can write a
+    small program to implement it.
   * Fix a lock file descriptor leak that could occur when running commands
     like git-annex add with -J. Bug was introduced as part of a different FD
     leak fix in version 6.20160318.
diff --git a/doc/todo/MD5E_keys_without_file_size.mdwn b/doc/todo/MD5E_keys_without_file_size.mdwn
index 12d50a9d9..a0ec56f1a 100644
--- a/doc/todo/MD5E_keys_without_file_size.mdwn
+++ b/doc/todo/MD5E_keys_without_file_size.mdwn
@@ -2,4 +2,7 @@ Would it be hard to support MD5E keys that omit the -sSIZE part, the way this is
 
 Another (and more generally useful) solution would be [[todo/alternate_keys_for_same_content/]].  Then can start with a URL-based key but then attach an MD5 to it as metadata, and have the key treated as a checksum-containing key, without needing to migrate the contents to a new key.
 
-[[!tag moreinfo]]
+> Closing, because [[external_backends]] is implemented, so you should be
+> able to roll your own backend for your use case here. Assuming you can't
+> just use regular MD5E and omit the file size field, which will work too.
+> --[[Joey]]
diff --git a/doc/todo/external_backends.mdwn b/doc/todo/external_backends.mdwn
index 01ec7a0db..af13e8b33 100644
--- a/doc/todo/external_backends.mdwn
+++ b/doc/todo/external_backends.mdwn
@@ -4,5 +4,6 @@ It would be good if one could define custom external [[backends]], the way one c
 
 Thoughts?
 
-[[!tag needsthought]]
 [[!tag projects/datalad]]
+
+> fully implemented. [[done]] --[[Joey]]
diff --git a/doc/todo/key_checksum_from_chunk_checksums.mdwn b/doc/todo/key_checksum_from_chunk_checksums.mdwn
index a736380c6..1635d7318 100644
--- a/doc/todo/key_checksum_from_chunk_checksums.mdwn
+++ b/doc/todo/key_checksum_from_chunk_checksums.mdwn
@@ -1,3 +1,4 @@
 Would it be hard to add a variantion to checksumming [[backends]], that would change how the checksum is computed: instead of computing it on the whole file, it would first be computed on file chunks of given size, and then the final checksum computed on the concatenation of the chunk checksums?  You'd add a new [[key field|internals/key_format]], say cNNNNN, specifying the chunking size (the last chunk might be shorter).  Then (1) for large files, checksum computation could be parallelized (there could be a config option specifying the default chunk size for newly added files); (2) I often have large files on a remote, for which I have md5 for each chunk, but not for the full file; this would enable me to register the location of these fies with git-annex without downloading them, while still using a checksum-based key.
 
-[[!tag needsthought]]
+> Closing, because [[external_backends]] is implemented, so you should be
+> able to roll your own backend for your use case here. --[[Joey]]
diff --git a/doc/todo/option_to_add_user-specified_string_to_key.mdwn b/doc/todo/option_to_add_user-specified_string_to_key.mdwn
index 1b392c7ca..4ed4ce107 100644
--- a/doc/todo/option_to_add_user-specified_string_to_key.mdwn
+++ b/doc/todo/option_to_add_user-specified_string_to_key.mdwn
@@ -12,4 +12,9 @@ This enables attaching metadata not to file contents, but to the file itself; or
 deduplication.  This loss may be acceptable.  The loss can be mitigated for local repo and non-special remotes: after storing an object with e.g. MD5 d41d8cd98f00b204e9800998ecf8427e under .git/annex/objects, check if there is a symlink .git/annex/contenthash/d41d8cd98f00b204e9800998ecf8427e ; if not, make this a symlink to the object just stored; if yes,
 erase the object just stored, and hardlink the symlink's target instead.
 
-[[!tag unlikely moreinfo]]
+> Closing since [[external_backends]] is implemented, and you could do this
+> using it. Whether that's a good idea, I'm fairly doubtful about. Be sure
+> to read "considerations for generating keys" in 
+> <https://git-annex.branchable.com/design/external_backend_protocol/#index7h2>
+>  
+> [[done]] --[[Joey]]
diff --git a/doc/todo/shorter_keys_through_better_encoding/comment_9_75f611c63bf1e9c8b4885fea8e9d467f._comment b/doc/todo/shorter_keys_through_better_encoding/comment_9_75f611c63bf1e9c8b4885fea8e9d467f._comment
new file mode 100644
index 000000000..7c226eb34
--- /dev/null
+++ b/doc/todo/shorter_keys_through_better_encoding/comment_9_75f611c63bf1e9c8b4885fea8e9d467f._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 9"""
+ date="2020-07-29T21:22:42Z"
+ content="""
+[[external_backends]] is now implemented, so you can write a program that
+makes keys use some other, shorter hash encoding.
+
+I don't know if that's really sufficient to close this.
+"""]]

diff --git a/Backend/External.hs b/Backend/External.hs
index 995445327..6b2d062ec 100644
--- a/Backend/External.hs
+++ b/Backend/External.hs
@@ -11,6 +11,7 @@ module Backend.External (makeBackend) where
 
 import Annex.Common
 import Annex.ExternalAddonProcess
+import Backend.Utilities
 import Types.Key
 import Types.Backend
 import Types.KeySource
@@ -19,6 +20,7 @@ import qualified Utility.SimpleProtocol as Proto
 
 import qualified Data.ByteString as S
 import qualified Data.Map.Strict as M
+import Data.Char
 import Control.Concurrent
 import System.IO.Unsafe (unsafePerformIO)
 import System.Log.Logger (debugM)
@@ -92,8 +94,7 @@ genKeyExternal ebname hasext ks meterupdate =
 	req = GENKEY (fromRawFilePath (contentLocation ks))
 	notavail = giveup $ "Cannot generate a key, since " ++ externalBackendProgram ebname ++ " is not available."
 	
-	-- TODO hasExt handling
-	go (GENKEY_SUCCESS (ProtoKey k)) = result k
+	go (GENKEY_SUCCESS pk) = Just $ Result <$> fromProtoKey pk hasext ks
 	go (GENKEY_FAILURE msg) = Just $ giveup $
 		"External backend program failed to generate a key: " ++ msg
 	go (PROGRESS bytesprocessed) = Just $ do
@@ -106,8 +107,7 @@ verifyKeyContentExternal ebname hasext meterupdate k f =
 	withExternalState ebname hasext $ \st ->
 		handleRequest st req notavail go
   where
-  	-- TODO hasExt handling
-	req = VERIFYKEYCONTENT (ProtoKey k) f
+	req = VERIFYKEYCONTENT (toProtoKey k) f
 
 	-- This should not be able to happen, because CANVERIFY is checked
 	-- before this function is enable, and so the external program 
@@ -273,6 +273,26 @@ withExternalState bname hasext a = do
 newtype ProtoKey = ProtoKey Key
 	deriving (Show)
 
+fromProtoKey :: ProtoKey -> HasExt -> KeySource -> Annex Key
+fromProtoKey (ProtoKey k) (HasExt False) _ = pure k
+fromProtoKey (ProtoKey k) hasext@(HasExt True) source =
+	addE source (setHasExt hasext) k
+
+toProtoKey :: Key -> ProtoKey
+toProtoKey k = ProtoKey $ alterKey k $ \d -> d
+	-- The extension can be easily removed, because the protocol
+	-- documentation does not allow '.' to be used in the keyName,
+	-- so the first one is the extension.
+	{ keyName = S.takeWhile (/= dot) (keyName d)
+	, keyVariety = setHasExt (HasExt False) (keyVariety d)
+	}
+  where
+	dot = fromIntegral (ord '.')
+
+setHasExt :: HasExt -> KeyVariety -> KeyVariety
+setHasExt hasext (ExternalKey name _) = ExternalKey name hasext
+setHasExt _ v = v
+
 instance Proto.Serializable ProtoKey where
 	serialize (ProtoKey k) = Proto.serialize k
 	deserialize = fmap ProtoKey . Proto.deserialize
diff --git a/Backend/Hash.hs b/Backend/Hash.hs
index 2b8bcd78d..e80ad4216 100644
--- a/Backend/Hash.hs
+++ b/Backend/Hash.hs
@@ -20,13 +20,11 @@ import Types.Backend
 import Types.KeySource
 import Utility.Hash
 import Utility.Metered
+import Backend.Utilities
 
 import qualified Data.ByteString as S
 import qualified Data.ByteString.Char8 as S8
 import qualified Data.ByteString.Lazy as L
-import qualified System.FilePath.ByteString as P
-import Data.Char
-import Data.Word
 import Control.DeepSeq
 import Control.Exception (evaluate)
 
@@ -114,29 +112,8 @@ keyValue hash source meterupdate = do
 {- Extension preserving keys. -}
 keyValueE :: Hash -> KeySource -> MeterUpdate -> Annex Key
 keyValueE hash source meterupdate =
-	keyValue hash source meterupdate >>= addE
-  where
-	addE k = do
-		maxlen <- annexMaxExtensionLength <$> Annex.getGitConfig
-		let ext = selectExtension maxlen (keyFilename source)
-		return $ alterKey k $ \d -> d
-			{ keyName = keyName d <> ext
-			, keyVariety = hashKeyVariety hash (HasExt True)
-			}
-
-selectExtension :: Maybe Int -> RawFilePath -> S.ByteString
-selectExtension maxlen f
-	| null es = ""
-	| otherwise = S.intercalate "." ("":es)
-  where
-	es = filter (not . S.null) $ reverse $
-		take 2 $ filter (S.all validInExtension) $
-		takeWhile shortenough $
-		reverse $ S.split (fromIntegral (ord '.')) (P.takeExtensions f)
-	shortenough e = S.length e <= fromMaybe maxExtensionLen maxlen
-
-maxExtensionLen :: Int
-maxExtensionLen = 4 -- long enough for "jpeg"
+	keyValue hash source meterupdate
+		>>= addE source (const $ hashKeyVariety hash (HasExt True))
 
 {- A key's checksum is checked during fsck when it's content is present
  - except for in fast mode. -}
@@ -166,13 +143,6 @@ checkKeyChecksum hash key file = catchIOErrorType HardwareFault hwfault $ do
 keyHash :: Key -> S.ByteString
 keyHash = fst . splitKeyNameExtension
 
-validInExtension :: Word8 -> Bool
-validInExtension c
-	| isAlphaNum (chr (fromIntegral c)) = True
-	| fromIntegral c == ord '.' = True
-	| c <= 127 = False -- other ascii: spaces, punctuation, control chars
-	| otherwise = True -- utf8 is allowed, also other encodings
-
 {- Upgrade keys that have the \ prefix on their hash due to a bug, or
  - that contain non-alphanumeric characters in their extension.
  -
@@ -310,10 +280,10 @@ testKeyBackend =
 	let b = genBackendE (SHA2Hash (HashSize 256))
 	    gk = case genKey b of
 		Nothing -> Nothing
-		Just f -> Just (\ks p -> addE <$> f ks p)
+		Just f -> Just (\ks p -> addTestE <$> f ks p)
 	in b { genKey = gk }
   where
-	addE k = alterKey k $ \d -> d
+	addTestE k = alterKey k $ \d -> d
 		{ keyName = keyName d <> longext
 		}
 	longext = ".this-is-a-test-key"
diff --git a/Backend/Utilities.hs b/Backend/Utilities.hs
index 0baaa476c..16bbbdc9f 100644
--- a/Backend/Utilities.hs
+++ b/Backend/Utilities.hs
@@ -1,17 +1,25 @@
 {- git-annex backend utilities
  -
- - Copyright 2012-2019 Joey Hess <id@joeyh.name>
+ - Copyright 2012-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
 
+{-# LANGUAGE OverloadedStrings #-}
+
 module Backend.Utilities where
 
 import Annex.Common
+import qualified Annex
 import Utility.Hash
+import Types.Key
+import Types.KeySource
 
 import qualified Data.ByteString as S
 import qualified Data.ByteString.Lazy as L
+import qualified System.FilePath.ByteString as P
+import Data.Char
+import Data.Word
 
 {- Generates a keyName from an input string. Takes care of sanitizing it.
  - If it's not too long, the full string is used as the keyName.
@@ -32,3 +40,34 @@ genKeyName s
 	sha256len = 64
 	md5len = 32
 
+{- Converts a key to a version that includes an extension from the
+ - file that the key was generated from.  -}
+addE :: KeySource -> (KeyVariety -> KeyVariety) -> Key -> Annex Key
+addE source sethasext k = do
+	maxlen <- annexMaxExtensionLength <$> Annex.getGitConfig
+	let ext = selectExtension maxlen (keyFilename source)
+	return $ alterKey k $ \d -> d
+		{ keyName = keyName d <> ext
+		, keyVariety = sethasext (keyVariety d)
+		}
+
+selectExtension :: Maybe Int -> RawFilePath -> S.ByteString
+selectExtension maxlen f
+	| null es = ""
+	| otherwise = S.intercalate "." ("":es)
+  where
+	es = filter (not . S.null) $ reverse $
+		take 2 $ filter (S.all validInExtension) $
+		takeWhile shortenough $
+		reverse $ S.split (fromIntegral (ord '.')) (P.takeExtensions f)

(Diff truncated)

diff --git a/Backend/External.hs b/Backend/External.hs
index d665555c0..995445327 100644
--- a/Backend/External.hs
+++ b/Backend/External.hs
@@ -21,6 +21,7 @@ import qualified Data.ByteString as S
 import qualified Data.Map.Strict as M
 import Control.Concurrent
 import System.IO.Unsafe (unsafePerformIO)
+import System.Log.Logger (debugM)
 
 newtype ExternalBackendName = ExternalBackendName S.ByteString
 	deriving (Show, Eq, Ord)
@@ -30,34 +31,50 @@ newtype ExternalBackendName = ExternalBackendName S.ByteString
 -- that cannot generate or verify keys, but that still lets the keys be
 -- basically used.
 makeBackend :: S.ByteString -> HasExt -> Annex Backend
-makeBackend bname hasext = withExternalState ebname $ \st -> do
-	withExternalAddon st (pure unavailbackend) $ \_ext -> do
-		canverify <- handleRequest st CANVERIFY (pure False) $ \case
-			CANVERIFY_YES -> result True
-			CANVERIFY_NO -> result False
-			_ -> Nothing
-		isstable <- handleRequest st ISSTABLE (pure False) $ \case
-			ISSTABLE_YES -> result True
-			ISSTABLE_NO -> result False
-			_ -> Nothing
-		iscryptographicallysecure <- handleRequest st ISCRYPTOGRAPHICALLYSECURE (pure False) $ \case
-			ISCRYPTOGRAPHICALLYSECURE_YES -> result True
-			ISCRYPTOGRAPHICALLYSECURE_NO -> result False
-			_ -> Nothing
-		return $ Backend
-			{ backendVariety = ExternalKey bname hasext
-			, genKey = Just $ genKeyExternal ebname
-			, verifyKeyContent = if canverify
-				then Just $ verifyKeyContentExternal ebname
-				else Nothing
-			, canUpgradeKey = Nothing
-			, fastMigrate = Nothing
-			, isStableKey = const isstable
-			, isCryptographicallySecure = const iscryptographicallysecure
-			}
+makeBackend bname hasext =
+	withExternalState ebname hasext (return . externalBackend)
   where
 	ebname = ExternalBackendName bname
-	unavailbackend = Backend
+
+makeBackend' :: ExternalBackendName -> HasExt -> Either ExternalAddonStartError ExternalAddonProcess -> Annex Backend
+makeBackend' ebname@(ExternalBackendName bname) hasext (Right p) = do
+	let st = ExternalState
+		{ externalAddonProcess = Right p
+		, externalBackend = unavailBackend ebname hasext
+		}
+	canverify <- handleRequest st CANVERIFY (pure False) $ \case
+		CANVERIFY_YES -> result True
+		CANVERIFY_NO -> result False
+		_ -> Nothing
+	isstable <- handleRequest st ISSTABLE (pure False) $ \case
+		ISSTABLE_YES -> result True
+		ISSTABLE_NO -> result False
+		_ -> Nothing
+	iscryptographicallysecure <- handleRequest st ISCRYPTOGRAPHICALLYSECURE (pure False) $ \case
+		ISCRYPTOGRAPHICALLYSECURE_YES -> result True
+		ISCRYPTOGRAPHICALLYSECURE_NO -> result False
+		_ -> Nothing
+	return $ Backend
+		{ backendVariety = ExternalKey bname hasext
+		, genKey = Just $ genKeyExternal ebname hasext
+		, verifyKeyContent = if canverify
+			then Just $ verifyKeyContentExternal ebname hasext
+				-- The protocol supports PROGRESS here,
+				-- but it's not actually used. It was put
+				-- in to avoid needing a protocol version
+				-- bump if progress handling is later added.
+				nullMeterUpdate
+			else Nothing
+		, canUpgradeKey = Nothing
+		, fastMigrate = Nothing
+		, isStableKey = const isstable
+		, isCryptographicallySecure = const iscryptographicallysecure
+		}
+makeBackend' ebname hasext (Left _) = return $ unavailBackend ebname hasext
+
+unavailBackend :: ExternalBackendName -> HasExt -> Backend
+unavailBackend (ExternalBackendName bname) hasext = 
+	Backend
 		{ backendVariety = ExternalKey bname hasext
 		, genKey = Nothing
 		, verifyKeyContent = Nothing
@@ -67,26 +84,63 @@ makeBackend bname hasext = withExternalState ebname $ \st -> do
 		, isCryptographicallySecure = const False
 		}
 
-genKeyExternal :: ExternalBackendName -> KeySource -> MeterUpdate -> Annex Key
-genKeyExternal bname ks p = error "TODO"
+genKeyExternal :: ExternalBackendName -> HasExt -> KeySource -> MeterUpdate -> Annex Key
+genKeyExternal ebname hasext ks meterupdate = 
+	withExternalState ebname hasext $ \st ->
+		handleRequest st req notavail go
+  where
+	req = GENKEY (fromRawFilePath (contentLocation ks))
+	notavail = giveup $ "Cannot generate a key, since " ++ externalBackendProgram ebname ++ " is not available."
+	
+	-- TODO hasExt handling
+	go (GENKEY_SUCCESS (ProtoKey k)) = result k
+	go (GENKEY_FAILURE msg) = Just $ giveup $
+		"External backend program failed to generate a key: " ++ msg
+	go (PROGRESS bytesprocessed) = Just $ do
+		liftIO $ meterupdate bytesprocessed
+		return $ GetNextMessage go
+	go _ = Nothing
+
+verifyKeyContentExternal :: ExternalBackendName -> HasExt -> MeterUpdate -> Key -> FilePath -> Annex Bool
+verifyKeyContentExternal ebname hasext meterupdate k f = 
+	withExternalState ebname hasext $ \st ->
+		handleRequest st req notavail go
+  where
+  	-- TODO hasExt handling
+	req = VERIFYKEYCONTENT (ProtoKey k) f
+
+	-- This should not be able to happen, because CANVERIFY is checked
+	-- before this function is enable, and so the external program 
+	-- is available. But if it does, fail the verification.
+	notavail = return False
 
-verifyKeyContentExternal :: ExternalBackendName -> Key -> FilePath -> Annex Bool
-verifyKeyContentExternal bname k f = error "TODO"
+	go VERIFYKEYCONTENT_SUCCESS = result True
+	go VERIFYKEYCONTENT_FAILURE = result False
+	go (PROGRESS bytesprocessed) = Just $ do
+		liftIO $ meterupdate bytesprocessed
+		return $ GetNextMessage go
+	go _ = Nothing
 
 -- State about a running external backend program.
 data ExternalState = ExternalState
 	{ externalAddonProcess :: Either ExternalAddonStartError ExternalAddonProcess
+	, externalBackend :: Backend
 	}
 
 handleRequest :: ExternalState -> Request -> Annex a -> ResponseHandler a -> Annex a
 handleRequest st req whenunavail responsehandler =
 	withExternalAddon st whenunavail $ \p -> do
 		sendMessage p req
-		receiveResponse p responsehandler (Just . handleAsyncMessage)
+		let loop = receiveResponse p responsehandler
+			(Just . handleAsyncMessage loop)
+		loop
   where
-	handleAsyncMessage (ERROR err) = do
+	handleAsyncMessage _ (ERROR err) = do
 		warning ("external special remote error: " ++ err)
 		whenunavail
+	handleAsyncMessage loop (DEBUG msg) = do
+		liftIO $ debugM "external" msg
+		loop
 
 withExternalAddon :: ExternalState -> a -> (ExternalAddonProcess -> a) -> a
 withExternalAddon st whenunavail a = case externalAddonProcess st of
@@ -153,8 +207,8 @@ poolVar = unsafePerformIO $ newMVar M.empty
 -- Starts a new instance of an external backend.
 -- Does not add it to the poolVar; caller should add it once it's done
 -- using it.
-newExternalState :: ExternalBackendName -> ExternalAddonPID -> Annex ExternalState
-newExternalState (ExternalBackendName name) pid = do
+newExternalState :: ExternalBackendName -> HasExt -> ExternalAddonPID -> Annex ExternalState
+newExternalState ebname hasext pid = do
 	st <- startExternalAddonProcess basecmd pid
 	st' <- case st of
 		Left (ProgramNotInstalled msg) -> warnonce msg >> return st
@@ -172,17 +226,24 @@ newExternalState (ExternalBackendName name) pid = do
 					warnonce (basecmd ++ " uses an unsupported version of the external backend protocol")
 					return $ Left (ProgramFailure "bad protocol version")
 				else return (Right p)
-	return $ ExternalState { externalAddonProcess = st' }
+	backend <- makeBackend' ebname hasext st'
+	return $ ExternalState
+		{ externalAddonProcess = st'
+		, externalBackend = backend
+		}
   where
-	basecmd = "git-annex-backend-X" ++ decodeBS' name
+	basecmd = externalBackendProgram ebname
 	warnonce msg = when (pid == 1) $
 		warning msg
 
+externalBackendProgram :: ExternalBackendName -> String
+externalBackendProgram (ExternalBackendName bname) = "git-annex-backend-X" ++ decodeBS' bname
+
 -- Runs an action with an ExternalState, starting a new external backend
 -- process if necessary. It is returned to the pool once the action
 -- finishes successfully. On exception, it's shut down.
-withExternalState :: ExternalBackendName -> (ExternalState -> Annex a) -> Annex a
-withExternalState bname a = do
+withExternalState :: ExternalBackendName -> HasExt -> (ExternalState -> Annex a) -> Annex a
+withExternalState bname hasext a = do
 	st <- get
 	r <- a st `onException` shutdown st
 	put st -- only when no exception is thrown
@@ -194,7 +255,7 @@ withExternalState bname a = do

(Diff truncated)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 2f79e4037..ad0f62ed3 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -349,12 +349,13 @@ getViaTmpFromDisk rsp v key action = checkallowed $ do
 	-- RetrievalSecurityPolicy would cause verification to always fail.
 	checkallowed a = case rsp of
 		RetrievalAllKeysSecure -> a
-		RetrievalVerifiableKeysSecure
-			| Backend.isVerifiable key -> a
-			| otherwise -> ifM (annexAllowUnverifiedDownloads <$> Annex.getGitConfig)
+		RetrievalVerifiableKeysSecure -> ifM (Backend.isVerifiable key)
+			( a
+			, ifM (annexAllowUnverifiedDownloads <$> Annex.getGitConfig)
 				( a
 				, warnUnverifiableInsecure key >> return False
 				)
+			)
 
 {- Verifies that a file is the expected content of a key.
  -
@@ -373,12 +374,13 @@ getViaTmpFromDisk rsp v key action = checkallowed $ do
 verifyKeyContent :: RetrievalSecurityPolicy -> VerifyConfig -> Verification -> Key -> FilePath -> Annex Bool
 verifyKeyContent rsp v verification k f = case (rsp, verification) of
 	(_, Verified) -> return True
-	(RetrievalVerifiableKeysSecure, _)
-		| Backend.isVerifiable k -> verify
-		| otherwise -> ifM (annexAllowUnverifiedDownloads <$> Annex.getGitConfig)
+	(RetrievalVerifiableKeysSecure, _) -> ifM (Backend.isVerifiable k)
+		( verify
+		, ifM (annexAllowUnverifiedDownloads <$> Annex.getGitConfig)
 			( verify
 			, warnUnverifiableInsecure k >> return False
 			)
+		)
 	(_, UnVerified) -> ifM (shouldVerify v)
 		( verify
 		, return True
@@ -391,9 +393,11 @@ verifyKeyContent rsp v verification k f = case (rsp, verification) of
 		Just size -> do
 			size' <- liftIO $ catchDefaultIO 0 $ getFileSize f
 			return (size' == size)
-	verifycontent = case Types.Backend.verifyKeyContent =<< Backend.maybeLookupBackendVariety (fromKey keyVariety k) of
+	verifycontent = Backend.maybeLookupBackendVariety (fromKey keyVariety k) >>= \case
 		Nothing -> return True
-		Just verifier -> verifier k f
+		Just b -> case Types.Backend.verifyKeyContent b of
+			Nothing -> return True
+			Just verifier -> verifier k f
 
 warnUnverifiableInsecure :: Key -> Annex ()
 warnUnverifiableInsecure k = warning $ unwords
@@ -512,12 +516,13 @@ moveAnnex key src = ifM (checkSecureHashes' key)
 	alreadyhave = liftIO $ removeFile src
 
 checkSecureHashes :: Key -> Annex (Maybe String)
-checkSecureHashes key
-	| Backend.isCryptographicallySecure key = return Nothing
-	| otherwise = ifM (annexSecureHashesOnly <$> Annex.getGitConfig)
+checkSecureHashes key = ifM (Backend.isCryptographicallySecure key)
+	( return Nothing
+	, ifM (annexSecureHashesOnly <$> Annex.getGitConfig)
 		( return $ Just $ "annex.securehashesonly blocked adding " ++ decodeBS (formatKeyVariety (fromKey keyVariety key)) ++ " key"
 		, return Nothing
 		)
+	)
 
 checkSecureHashes' :: Key -> Annex Bool
 checkSecureHashes' key = checkSecureHashes key >>= \case
diff --git a/Annex/ExternalAddonProcess.hs b/Annex/ExternalAddonProcess.hs
index 11f9b3b11..94983160a 100644
--- a/Annex/ExternalAddonProcess.hs
+++ b/Annex/ExternalAddonProcess.hs
@@ -13,8 +13,8 @@ import Git.Env
 import Utility.Shell
 import Messages.Progress
 
-import Control.Concurrent.STM
 import Control.Concurrent.Async
+import System.Log.Logger (debugM)
 
 data ExternalAddonProcess = ExternalAddonProcess
 	{ externalSend :: Handle
@@ -23,6 +23,7 @@ data ExternalAddonProcess = ExternalAddonProcess
 	-- immediately.
 	, externalShutdown :: Bool -> IO ()
 	, externalPid :: ExternalAddonPID
+	, externalProgram :: String
 	}
 
 type ExternalAddonPID = Int
@@ -31,8 +32,8 @@ data ExternalAddonStartError
 	= ProgramNotInstalled String
 	| ProgramFailure String
 
-startExternalAddonProcess :: String -> TVar ExternalAddonPID-> Annex (Either ExternalAddonStartError ExternalAddonProcess)
-startExternalAddonProcess basecmd pidvar = do
+startExternalAddonProcess :: String -> ExternalAddonPID -> Annex (Either ExternalAddonStartError ExternalAddonProcess)
+startExternalAddonProcess basecmd pid = do
 	errrelayer <- mkStderrRelayer
 	g <- Annex.gitRepo
 	cmdpath <- liftIO $ searchPath basecmd
@@ -47,16 +48,12 @@ startExternalAddonProcess basecmd pidvar = do
 			}
 		p <- propgit g basep
 		tryNonAsync (createProcess p) >>= \case
-			Right v -> (Right <$> started errrelayer v)
+			Right v -> (Right <$> started cmd errrelayer v)
 				`catchNonAsync` const (runerr cmdpath)
 			Left _ -> runerr cmdpath
 	
-	started errrelayer pall@(Just hin, Just hout, Just herr, ph) = do
+	started cmd errrelayer pall@(Just hin, Just hout, Just herr, ph) = do
 		stderrelay <- async $ errrelayer herr
-		pid <- atomically $ do
-			n <- succ <$> readTVar pidvar
-			writeTVar pidvar n
-			return n
 		let shutdown forcestop = do
 			cancel stderrelay
 			if forcestop
@@ -71,8 +68,9 @@ startExternalAddonProcess basecmd pidvar = do
 			, externalReceive = hout
 			, externalPid = pid
 			, externalShutdown = shutdown
+			, externalProgram = cmd
 			}
-	started _ _ = giveup "internal"
+	started _ _ _ = giveup "internal"
 
 	propgit g p = do
 		environ <- propGitEnv g
@@ -85,3 +83,11 @@ startExternalAddonProcess basecmd pidvar = do
 		path <- intercalate ":" <$> getSearchPath
 		return $ Left $ ProgramNotInstalled $
 			"Cannot run " ++ basecmd ++ " -- It is not installed in PATH (" ++ path ++ ")"
+
+protocolDebug :: ExternalAddonProcess -> Bool -> String -> IO ()
+protocolDebug external sendto line = debugM "external" $ unwords
+	[ externalProgram external ++ 
+		"[" ++ show (externalPid external) ++ "]"
+	, if sendto then "<--" else "-->"
+	, line
+	]
diff --git a/Annex/Transfer.hs b/Annex/Transfer.hs
index 789a51f25..1bf50f022 100644
--- a/Annex/Transfer.hs
+++ b/Annex/Transfer.hs
@@ -177,14 +177,15 @@ runTransfer' ignorelock t afile retrydecider transferaction = enteringStage Tran
  - tend to be configured to reject it, so Upload is also prevented.
  -}
 checkSecureHashes :: Observable v => Transfer -> Annex v -> Annex v
-checkSecureHashes t a
-	| isCryptographicallySecure (transferKey t) = a
-	| otherwise = ifM (annexSecureHashesOnly <$> Annex.getGitConfig)
+checkSecureHashes t a = ifM (isCryptographicallySecure (transferKey t))
+	( a
+	, ifM (annexSecureHashesOnly <$> Annex.getGitConfig)
 		( do
 			warning $ "annex.securehashesonly blocked transfer of " ++ decodeBS (formatKeyVariety variety) ++ " key"
 			return observeFailure
 		, a
 		)
+	)
   where
 	variety = fromKey keyVariety (transferKey t)
 
diff --git a/Assistant/Upgrade.hs b/Assistant/Upgrade.hs
index 78a6b3705..2e7d8d0bf 100644
--- a/Assistant/Upgrade.hs
+++ b/Assistant/Upgrade.hs
@@ -117,7 +117,7 @@ distributionDownloadComplete d dest cleanup t
 	| otherwise = cleanup
   where
 	k = mkKey $ const $ distributionKey d
-	fsckit f = case Backend.maybeLookupBackendVariety (fromKey keyVariety k) of
+	fsckit f = Backend.maybeLookupBackendVariety (fromKey keyVariety k) >>= \case
 		Nothing -> return $ Just f
 		Just b -> case Types.Backend.verifyKeyContent b of
 			Nothing -> return $ Just f
diff --git a/Backend.hs b/Backend.hs
index 99f4c296f..f17ef9a7a 100644
--- a/Backend.hs
+++ b/Backend.hs
@@ -6,12 +6,13 @@
  -}
 
 module Backend (
-	list,
+	builtinList,
 	defaultBackend,
 	genKey,
 	getBackend,
 	chooseBackend,
 	lookupBackendVariety,
+	lookupBuiltinBackendVariety,
 	maybeLookupBackendVariety,
 	isStableKey,
 	isCryptographicallySecure,
@@ -26,16 +27,18 @@ import Types.KeySource

(Diff truncated)

diff --git a/doc/design/external_backend_protocol.mdwn b/doc/design/external_backend_protocol.mdwn
index 128ed5e49..0bade483f 100644
--- a/doc/design/external_backend_protocol.mdwn
+++ b/doc/design/external_backend_protocol.mdwn
@@ -42,7 +42,7 @@ The program responds.
 	VERSION 1
 
 git-annex will next query the program about the properties of the keys it
-uses (CANVERIFY, ISSTABLE, ISCRYPTOGRAPHICALLYSECURE), and the program will
+uses (`CANVERIFY`, `ISSTABLE`, `ISCRYPTOGRAPHICALLYSECURE`), and the program will
 respond to each query.
 
 Then git-annex may ask the program to generate a key.
@@ -97,7 +97,7 @@ reply with one of the listed replies.
   art advances, and should aim to stay ahead of the state of the art by a
   reasonable amount of time.
   * `ISCRYPTOGRAPHICALLYSECURE-YES`
-  * ISCRYPTOGRAPHICALLYSECURE-NO`
+  * `ISCRYPTOGRAPHICALLYSECURE-NO`
 
 ## main messages and replies
 
@@ -113,7 +113,7 @@ This is where work happens.
   The program should examine the ContentFile and verify that it has the
   content it would expect for the Key. While it is doing this, it can
   send any number of `PROGRESS` messages indication the position in the
-  file that it's gotten to. (If the program earlier sent CANVERIFY-NO,
+  file that it's gotten to. (If the program earlier sent `CANVERIFY-NO`,
   it will not be asked to do this.)
   * `VERIFYKEYCONTENT-SUCCESS`
   * `VERIFYKEYCONTENT-FAILURE`
@@ -129,8 +129,8 @@ These messages can be sent at any time by either git-annex or the program.
   Generic error. Can be sent at any time if things get too messed up to
   continue. When possible, use a more specific reply.  
   The program should exit after sending this, as git-annex will not talk to
-  it any further. If the program receives an ERROR from git-annex, it can
-  exit with its own ERROR.
+  it any further. If the program receives an `ERROR` from git-annex, it can
+  exit with its own `ERROR`.
 
 ## considerations for generating keys
 
@@ -142,7 +142,8 @@ is git-annex-backend-XFOO, it should generate a key starting with "XFOO-".
 
 The backend name (and program name) has to be all uppercase, and should be
 reasonably short (max 10 bytes or so), and should be entirely ascii
-alphanumerics. Eg, use similar names to other [[backends]].
+alphanumerics. Eg, use similar names to other [[backends]]. It must not end
+with "E" (see next paragraph for why).
 
 git-annex will automatically also support an "E" variant of the backend,
 which adds a filename extension to the end of the key. It does this
@@ -156,7 +157,7 @@ using them, or even cause problems due to filename length limits. 128 bytes
 maximum, but shorter is better.
 
 It's important that, if the program responds with
-ISCRYPTOGRAPHICALLYSECURE-YES, the key name contains only a hash, and not
+`ISCRYPTOGRAPHICALLYSECURE-YES`, the key name contains only a hash, and not
 other data from some other source. That other data could be used to try to
 mount a sha1 collision attack against git, by embedding colliding material
 in the key name, where users are unlikely to notice it. While git has

diff --git a/doc/forum/standalone_tarballs_for_specific_versions/comment_11_543bb94e6a6c1313a09328402a191995._comment b/doc/forum/standalone_tarballs_for_specific_versions/comment_11_543bb94e6a6c1313a09328402a191995._comment
new file mode 100644
index 000000000..915e89bd9
--- /dev/null
+++ b/doc/forum/standalone_tarballs_for_specific_versions/comment_11_543bb94e6a6c1313a09328402a191995._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="version correspondence of standalone builds and official releases"
+ date="2020-07-29T15:36:37Z"
+ content="""
+To answer @joeyh's question from a [related bug report](https://git-annex.branchable.com/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected/#comment-06e9d73bdaff08d38d991ddc2b946f96) (\"maybe I'm missing something about how problematic the build being slightly behind until the next release is for you\"): it's problematic for [[conda-forge|install/conda]] packaging, because I want conda to install the non-standalone release if possible and automatically fall back on the standalone release if dependencies of the standard release conflict with other packages; for this to work, I need need the standalone release that corresponds to the same version as the standard release.  I could package a standalone-only conda-forge release, marking it as an alpha to keep it from being preferred to the official release; but then there's no seamless fallback if the user wants the specific official version.
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_11_b3c5851c3603976d59927b591ba1d126._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_11_b3c5851c3603976d59927b591ba1d126._comment
new file mode 100644
index 000000000..b4394c0c7
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_11_b3c5851c3603976d59927b591ba1d126._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 11"
+ date="2020-07-28T15:26:41Z"
+ content="""
+that was with a bit dated (8.20200501+git61-g64e081d58-1~ndall+1) version, with 8.20200720.1-1~ndall+1 it looks a bit better:
+
+```
+$> strace -f git-annex version 2>&1 | awk '/libpcre.*so.*ENOENT/{print}' | wc -l
+39
+```
+
+but still a more pertinent test/demonstration of current situation would be
+
+```shell
+$> strace -f git-annex init 2>&1 | awk '/libpcre.*so.*ENOENT/{print}' | wc -l   
+651
+```
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_10_20db4ecf9debcc2662ec69a6ad9b37c6._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_10_20db4ecf9debcc2662ec69a6ad9b37c6._comment
new file mode 100644
index 000000000..2522a8f7a
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_10_20db4ecf9debcc2662ec69a6ad9b37c6._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 10"
+ date="2020-07-28T15:18:40Z"
+ content="""
+WOW -- a random discovery and possible note to myself:  I am getting all those libpcre misses when I run `git annex version` within a git repository and not otherwise:
+
+```
+$> strace -f git-annex version 2>&1 | awk '/libpcre.*so.*/{print}' | nl | tail -n 1
+     1	[pid 3702554] openat(AT_FDCWD, \"/lib/x86_64-linux-gnu/libpcre.so.3\", O_RDONLY|O_CLOEXEC) = 3
+
+$> mkdir repo; cd repo; git init
+Initialized empty Git repository in /tmp/repo/.git/
+
+$> strace -f git-annex version 2>&1 | awk '/libpcre.*so.*/{print}' | nl | tail -n 1
+    41	[pid 3702907] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = 3
+```
+
+Not sure why `git annex version` needs to run `git` when inside a git repo for its `version`. (same happens for `git annex --help`)
+"""]]

diff --git a/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected/comment_2_e976cda6c4cc2103b2a83ce185a71f06._comment b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected/comment_2_e976cda6c4cc2103b2a83ce185a71f06._comment
new file mode 100644
index 000000000..9819a3040
--- /dev/null
+++ b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected/comment_2_e976cda6c4cc2103b2a83ce185a71f06._comment
@@ -0,0 +1,32 @@
+[[!comment format=mdwn
+ username="leej"
+ avatar="http://cdn.libravatar.org/avatar/eb1c6bd57680f694fb4658388e6de4ed"
+ subject="comment 2"
+ date="2020-07-28T14:57:58Z"
+ content="""
+Yes, I've, too, had the feeling that I'm not getting something about your process.  Thank you for your continued patience.  Here’s a bit more about the assumptions under which I’ve been operating: 
+
+So far, in 2020, my Autopkg instance has pulled down these builds and imported them into Munki for testing and internal distribution to clients.  
+
+* 7.20191231 -> (build)
+* 7.20200205 -> (build)
+* 8.20200226 -> (documented git-annex release)
+* 8.20200309 -> (documented git-annex release)
+* 8.20200310 -> (build)
+* 8.20200331 -> (build)
+* 8.20200502 -> (build)
+* 8.20200617 -> (documented git-annex release)
+* 8.20200618 -> (build)
+
+If your intermediate builds that appear on kitenet (e.g.,  20200618 between the announced releases 20200617 and 20200720) are, in fact, ship quality.  And, there are no behavioral surprises introduced in these builds between public releases, then I would be happy to just accept the undocumented, intermediate build that Autopkg is finding whenever the version is incremented on the dmg on kitenet.   
+
+However, if they are more works-in-progress, as I have been assuming, and have the potential for introducing yet undocumented changes, then out of caution, I would (continue to) wait for a build matching one of your public releases that you document with a news entry.
+
+Let me know your thoughts as to the quality of these intermediate builds.  
+
+And, also, if there's a better way of letting you, or someone else, know simply that the macOS auto builder needs a nudge, other than cutting a bug, please let me know.  
+
+Note finally: I’m still seeing distributionVersion = \"8.20200618\", distributionReleasedate = 2020-07-20 18:49:32.302161119 UTC up on downloads.kitenet
+
+As always, thank you for your help and the value you’ve put into this incredibly useful program. 
+"""]]

diff --git a/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_3_8f94f5d1ee5c92e945ba2344cdea8307._comment b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_3_8f94f5d1ee5c92e945ba2344cdea8307._comment
new file mode 100644
index 000000000..493f63732
--- /dev/null
+++ b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_3_8f94f5d1ee5c92e945ba2344cdea8307._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="nicholsn"
+ nickname="nolan.nichols"
+ avatar="http://cdn.libravatar.org/avatar/12e81148e905851b4794288e8bf336e9"
+ subject="Thank you"
+ date="2020-07-28T03:57:32Z"
+ content="""
+This fix is much appreciated!
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_9_ce5fb75f944816b134fd2dc7a389b913._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_9_ce5fb75f944816b134fd2dc7a389b913._comment
new file mode 100644
index 000000000..6e326dad1
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_9_ce5fb75f944816b134fd2dc7a389b913._comment
@@ -0,0 +1,180 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 9"
+ date="2020-07-27T21:40:53Z"
+ content="""
+quick comment:
+
+>  The rest of the seeking seems reasonable.
+
+note that some directories seems to be considered twice, e.g.
+
+```
+[pid 481279] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+[pid 481279] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+```
+
+and
+
+```
+[pid 481279] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+[pid 481279] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+```
+
+I guess some paths get listed multiple times.
+
+> I guess these extra 14 seeks are not a major performance hit. ... 2 shell scripts have around 0.030s overhead themsevles
+
+sure not major but still wasteful if could be avoided.   
+
+> The way you run the shim does not put the bundled git in PATH. That kind of throws off your results, because git, not git-annex is what links to pcre.
+
+I do not think I did anything special, and here is my reproduced full example:
+
+<details>
+<summary>installing bleeding edge build (8.20200720.1+git52-gf5e65d680-1~ndall+1) from datalad-extensions</summary> 
+
+```shell
+(git)lena:~datalad/datalad[maint]
+$> tools/ci/install-annex.sh datalad-extensions-build
+I: top directory /home/yoh/.tmp/ga-4b1Aue7
+Using curl as \"curl --silent\"
+Getting artifacts_url from https://api.github.com/repos/datalad/datalad-extensions/actions/workflows/build-git-annex-debianstandalone.yaml/runs?status=success&branch=master into '/home/yoh/.tmp/tmp.n7XrEmCr3L'
+Getting archive download url from https://api.github.com/repos/datalad/datalad-extensions/actions/runs/183638203/artifacts
+Getting download url from https://api.github.com/repos/datalad/datalad-extensions/actions/artifacts/12229676/zip
+Downloading artifact package from https://pipelines.actions.githubusercontent.com/2UPlDxaVvvbkeFX4btxWorCjpJvj40zvWY5ogH2yZibhOMcU7O/_apis/pipelines/1/runs/1100/signedartifactscontent?artifactName=git-annex-debianstandalone-packages&urlExpires=2020-07-27T21%3A31%3A09.8843916Z&urlSigningMethod=HMACV1&urlSignature=K83aXlzZZoZcxFrz2diweljTVcdOsNv101ISfiNMcwk%3D
+Archive:  .artifact.zip
+  inflating: git-annex-build.log     
+  inflating: git-annex-standalone_8.20200720.1+git52-gf5e65d680-1~ndall+1_amd64.deb  
+  inflating: git-annex_8.20200720.1+git52-gf5e65d680-1~ndall+1.dsc  
+  inflating: git-annex_8.20200720.1+git52-gf5e65d680-1~ndall+1.tar.gz  
+  inflating: git-annex_8.20200720.1+git52-gf5e65d680-1~ndall+1_amd64.buildinfo  
+  inflating: git-annex_8.20200720.1+git52-gf5e65d680-1~ndall+1_amd64.changes  
+  inflating: git-annex_8.20200720.1+git52-gf5e65d680-1~ndall+1_source.buildinfo  
+  inflating: git-annex_8.20200720.1+git52-gf5e65d680-1~ndall+1_source.changes  
+[sudo] password for yoh: 
+(Reading database ... 760196 files and directories currently installed.)
+Preparing to unpack .../git-annex-standalone_8.20200720.1+git52-gf5e65d680-1~ndall+1_amd64.deb ...
+Unpacking git-annex-standalone (8.20200720.1+git52-gf5e65d680-1~ndall+1) over (8.20200617+git192-g5849bd634-1~ndall+1) ...
+Setting up git-annex-standalone (8.20200720.1+git52-gf5e65d680-1~ndall+1) ...
+Processing triggers for gnome-menus (3.36.0-1) ...
+Processing triggers for desktop-file-utils (0.24-1) ...
+Processing triggers for mime-support (3.64) ...
+Processing triggers for hicolor-icon-theme (0.17-2) ...
+Processing triggers for man-db (2.9.1-1) ...
+I: git-annex is available under '/usr/bin'
+
+$> grep GIT_ANNEX_PACK /usr/lib/git-annex.linux/runshell
+GIT_ANNEX_PACKAGE_INSTALL=1
+
+```
+</details>
+
+and it does not matter how I invoke (via outside git or directly through git-annex bundle)
+
+<details>
+<summary>I get those 46 failed lookups</summary> 
+
+```shell
+$> strace -f git annex version  2>&1 | grep 'libpcre2-8.*ENOENT' | nl
+     1	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     2	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     3	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     4	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     5	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     6	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     7	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     8	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     9	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    10	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    11	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    12	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    13	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    14	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    15	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    16	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    17	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    18	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    19	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    20	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    21	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    22	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    23	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    24	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    25	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    26	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    27	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    28	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    29	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    30	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    31	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    32	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    33	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    34	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    35	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    36	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    37	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    38	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    39	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    40	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    41	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    42	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    43	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    44	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    45	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    46	[pid 3664792] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+
+$> strace -f /usr/lib/git-annex.linux/git-annex version 2>&1 | grep 'libpcre2-8.*ENOENT' | nl 
+     1	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     2	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     3	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     4	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     5	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     6	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     7	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     8	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+     9	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    10	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    11	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    12	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    13	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    14	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    15	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    16	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/audit/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    17	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    18	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    19	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    20	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    21	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    22	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    23	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    24	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//etc/ld.so.conf.d/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    25	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    26	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    27	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    28	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    29	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    30	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    31	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    32	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    33	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    34	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    35	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    36	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    37	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    38	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    39	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    40	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//lib/x86_64-linux-gnu/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    41	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/tls/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    42	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/tls/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    43	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/tls/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    44	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/haswell/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    45	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/haswell/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+    46	[pid 3665083] openat(AT_FDCWD, \"/usr/lib/git-annex.linux//usr/lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0\", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+
+```
+</details>
+
+
+"""]]

diff --git a/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_8_b45863d81e3790ea44714a18c1721abd._comment b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_8_b45863d81e3790ea44714a18c1721abd._comment
new file mode 100644
index 000000000..9a5bc024d
--- /dev/null
+++ b/doc/todo/may_be___40__again__41___to_prelink_or_somehow_avoid_all_the_failing_opens__63__/comment_8_b45863d81e3790ea44714a18c1721abd._comment
@@ -0,0 +1,72 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2020-07-27T15:44:39Z"
+ content="""
+For details of the older todo, including some timings with and without
+prelinking, see [[!commit c4229be9a7a2318ef71b9ae433bc14bf604c9caf]].
+
+A ghc bug, since fixed, was causing it to look in IIRC, thousands of
+unncessary directories per library. This todo, by contrast, complains about
+less than 100 extra lookups total.
+
+The way you run the shim does not put the bundled git in PATH. That kind of
+throws off your results, because git, not git-annex is what links to pcre.
+I was not actually able to reproduce your result of it finding pcre without
+any failed seeks, but perhaps it ran a different git binary than the ones I
+have access to. Anyway, that all seems like a bit of a red herring due to
+that problem and puts your timings in doubt too.
+
+Simply moving all the libraries to a single directory would cut down on the
+failed seeking a lot:
+
+	strace -f ./git-annex version 2>&1 | grep ENOENT | grep openat
+	openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/x86_64/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/x86_64/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/x86_64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 481279] openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/x86_64/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 481279] openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 481279] openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 481279] openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/tls/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 481279] openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/x86_64/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 481279] openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+	[pid 481279] openat(AT_FDCWD, "/usr/lib/git-annex.linux//lib64/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
+
+The /lib64/tls/x86_64/x86_64/ seems like kind of weird behavior from
+ld-linux.so, I wonder if that's a bug on its part. The rest of the seeking
+seems reasonable. I guess these extra 14 seeks are not a major performance hit.
+
+But, that library consolidation does not seem to speed it up appreciably at
+all. Timings were almost identical before and after. 100 failed opens, when
+cache is hot, is just not that much overhead compared with the script's
+overhead. I don't think it's even worth implementing the library
+consolidiation based on this.
+
+Running `git-annex.linux/git-annex version`
+takes 0.060s. Compare with around 0.030s to run `/usr/bin/git-annex version`.
+(Sometimes it runs in more like 0.020s, but not often.. Probably have to
+catch the cache in the right mood.) 
+So, runshell and the other 2 shell scripts have around
+0.030s overhead themsevles.
+
+(This is with runshell modified so `GIT_ANNEX_PACKAGE_INSTALL` is set.
+IIRC the way the git-annex-standalone deb is built sets that. Otherwise,
+runshell does an additional 0.060s of locale setup stuff.)
+
+I tried putting set -x in all 3 levels of shell scripts and time stamping
+the output to see what was expensive. This was a bit surprising,
+because other than the abovementioned locale setup stuff, all the rest
+of the set -x output happened within 0.000965s. Which is 30 times faster
+than the timings above say it should be. Could be that the time stamping,
+which used `ts`, is not accurate enough.
+
+Anyway, runshell uses several unix utilities (and at least dirname is run
+redundantly between the git-annex script and runshell), 
+and there are 3 levels of shell scripts for the shell to parse and run.
+Combining them into a single shell script would eliminate some redundant work.
+Probably rewriting in C would be a bigger win.
+"""]]

diff --git a/doc/bugs/__39__Beyond_symbolic__39___link_error_when_link_is_upstream_of_repo.mdwn b/doc/bugs/__39__Beyond_symbolic__39___link_error_when_link_is_upstream_of_repo.mdwn
new file mode 100644
index 000000000..3b4d06838
--- /dev/null
+++ b/doc/bugs/__39__Beyond_symbolic__39___link_error_when_link_is_upstream_of_repo.mdwn
@@ -0,0 +1,79 @@
+2a8fdfc7d (Display a warning message when asked to operate on a file
+inside a directory that's a symbolic link to elsewhere, 2020-05-11)
+made `git annex add` error like `git add` does rather than silently
+doing nothing in a scenario like below.
+
+[[!format sh """
+# <repo>
+# |-- a
+# |   `-- f0
+# `-- alink -> a
+cd "$(mktemp -d "${TMPDIR:-/tmp}"/gx-link-XXXXXXX)"
+git init && git annex init
+mkdir a
+ln -sT a alink
+echo 0 >alink/f0
+git annex add alink/f0
+"""]]
+
+Unlike git, however, git-annex's warning also applies when the
+symbolic link is upstream of the repository and git-annex is passed an
+absolute path.
+
+[[!format sh """
+# .
+# |-- a
+# |   `-- repo
+# |       `-- f0
+# `-- alink -> a
+cd "$(mktemp -d "${TMPDIR:-/tmp}"/gx-link-XXXXXXX)"
+mkdir a
+ln -sT a alink
+git init alink/repo
+(
+    cd alink/repo
+    git annex init
+    echo 0 >f0
+    git annex add "$(pwd)"/f0
+)
+"""]]
+
+```
+[...]
+git-annex: /tmp/gx-link-Wxqnzo8/alink/repo/f0 is beyond a symbolic link
+add f0
+ok
+(recording state in git...)
+```
+
+The file does appear to be annexed and added to the index despite the
+warning:
+
+```
+$ git diff --cached
+diff --git a/f0 b/f0
+new file mode 120000
+index 0000000..7a725b9
+--- /dev/null
++++ b/f0
+@@ -0,0 +1 @@
++.git/annex/objects/Xj/V5/SHA256E-s2--9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa/SHA256E-s2--9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa
+\ No newline at end of file
+```
+
+Looking at `Seek.workTreeItems'`, my first thought was that the above
+scenario should not error: `relPathCwdToFile` should turn the absolute
+path into `f0` when it compares it against the current working
+directory, `/tmp/gx-link-Wxqnzo8/alink/repo/`.  However, it looks like
+`System.Directory.getCurrentDirectory` returns a resolved path for the
+current working directory, so `relPathCwdToFile` turns the absolute
+path into `../../alink/repo/f0`, which gets flagged by the
+`viasymlink` check.
+
+This error message is of course easy enough to work around on the
+caller's side, but since it looks like an unintended consequence of
+2a8fdfc7d, the change in behavior seems worth mentioning.
+
+
+[[!meta author=kyle]]
+[[!tag projects/datalad]]

diff --git a/doc/forum/best_way_to_move_a_git_annex_repo_trought_file_system/comment_3_00418ada32a4d93dfd3334686e391a23._comment b/doc/forum/best_way_to_move_a_git_annex_repo_trought_file_system/comment_3_00418ada32a4d93dfd3334686e391a23._comment
new file mode 100644
index 000000000..ac33c4c17
--- /dev/null
+++ b/doc/forum/best_way_to_move_a_git_annex_repo_trought_file_system/comment_3_00418ada32a4d93dfd3334686e391a23._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-07-27T15:38:22Z"
+ content="""
+Yeah, this is not really a problem. Do note that, when a git-annex repo is
+moved to a new drive, this does change some of the internal details that
+git-annex keeps track of (inode numbers). If the repository has unlocked
+files in it, this can make git-annex's handling of those a little slow at
+first, until it learns the new details.
+"""]]

diff --git a/doc/todo/Request__58___Date_limit_with_importfeed/comment_1_5d3d762eaf755410f5ee70a22fc95ab6._comment b/doc/todo/Request__58___Date_limit_with_importfeed/comment_1_5d3d762eaf755410f5ee70a22fc95ab6._comment
new file mode 100644
index 000000000..c8249dd39
--- /dev/null
+++ b/doc/todo/Request__58___Date_limit_with_importfeed/comment_1_5d3d762eaf755410f5ee70a22fc95ab6._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-07-27T15:34:53Z"
+ content="""
+What I do in this situation is use importfeed with --fast, and then delete
+the old episodes I am not interested in, and `git-annex get` the ones I do
+want. Or even use --relaxed if there are so many episodes that the http
+traffic to check their sizes takes too long.
+"""]]

diff --git a/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected.mdwn b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected.mdwn
index f3d5e8ee3..538de2fad 100644
--- a/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected.mdwn
+++ b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected.mdwn
@@ -34,3 +34,5 @@ $/usr/bin/sw_vers
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 Still rock solid maintaining the .dmgs and .pkgs for a 67GB [Munki](https://www.munki.org) repository.
+
+> [[done]] --[[Joey]]
diff --git a/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected/comment_1_adbceacb2f27d3527273686fae1b68d2._comment b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected/comment_1_adbceacb2f27d3527273686fae1b68d2._comment
new file mode 100644
index 000000000..bcb520186
--- /dev/null
+++ b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected/comment_1_adbceacb2f27d3527273686fae1b68d2._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-07-25T16:09:17Z"
+ content="""
+This is a near duplicate of a bug you filed before. I'm dubious about the
+value of these bug reports. As I said before:
+
+The build version being a little behind is not uncommon when the
+autobuilder has not been keeping up for whatever reason. I generally don't
+block releases on autobuilds if it's the fault of the autobuilder and not
+a bug in git-annex's build process or test suite, and just ship the latest
+available build.
+
+But maybe I'm missing something about how problimatic the build being
+slightly behind until the next release is for you?
+
+The autobuilder has since caught up again, so the daily
+build is current.
+"""]]

diff --git a/doc/design/external_backend_protocol.mdwn b/doc/design/external_backend_protocol.mdwn
index 69f0a6bca..128ed5e49 100644
--- a/doc/design/external_backend_protocol.mdwn
+++ b/doc/design/external_backend_protocol.mdwn
@@ -122,6 +122,9 @@ This is where work happens.
 
 These messages can be sent at any time by either git-annex or the program.
 
+* `DEBUG message`  
+  Tells git-annex to display the message if --debug is enabled.  
+  (git-annex does not send a reply to this message.)
 * `ERROR ErrorMsg`  
   Generic error. Can be sent at any time if things get too messed up to
   continue. When possible, use a more specific reply.  
diff --git a/doc/design/external_special_remote_protocol.mdwn b/doc/design/external_special_remote_protocol.mdwn
index e58264a5d..d6663be86 100644
--- a/doc/design/external_special_remote_protocol.mdwn
+++ b/doc/design/external_special_remote_protocol.mdwn
@@ -378,7 +378,7 @@ handling a request.
   may be empty to get all urls.
   (git-annex replies one or more times with VALUE for each url.
   The final VALUE has an empty value, indicating the end of the url list.)
-* `DEBUG message`
+* `DEBUG message`  
   Tells git-annex to display the message if --debug is enabled.  
   (git-annex does not send a reply to this message.)
 
diff --git a/doc/todo/external_backends/comment_14_6edc4008ce8eec17e283dd7db8dbcbf0._comment b/doc/todo/external_backends/comment_14_6edc4008ce8eec17e283dd7db8dbcbf0._comment
new file mode 100644
index 000000000..bb43080df
--- /dev/null
+++ b/doc/todo/external_backends/comment_14_6edc4008ce8eec17e283dd7db8dbcbf0._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 14"""
+ date="2020-07-27T15:20:00Z"
+ content="""
+> What is the advantage of a separate VERIFYCONTENT request, vs calling
+> GENKEY and comparing the result?
+
+Nothing for remotes using a hash. However, if the remote is using something
+other than a hash, or a hash combined with something else, it might not be
+able to regenerate the same key. It may still be able to detect
+corrupted content, eg using the hash part of the key.
+
+> Can the protocol specify that the file passed to GENKEY may be a named pipe
+
+I can't think of any situation where git-annex would GENKEY before
+it has the full content of a file available.
+
+> add DEBUG and INFO requests
+
+For INFO I'd rather wait for a use case. None of the current backends ever
+need to display any messages, except for in the case of an exceptional error,
+eg a hardware faulure where hashing. And ERROR would be fine for that.
+
+DEBUG sure.
+"""]]

diff --git a/doc/todo/Request__58___Date_limit_with_importfeed.mdwn b/doc/todo/Request__58___Date_limit_with_importfeed.mdwn
new file mode 100644
index 000000000..1448fe0c2
--- /dev/null
+++ b/doc/todo/Request__58___Date_limit_with_importfeed.mdwn
@@ -0,0 +1,3 @@
+I'm using importfeed with some podcasts, but I only want the most recent items. Importfeed seems to import all items in the feed, so I'm ending up with hundreds of them going back into last year and beyond. There doesn't seem to be a way to limit this.
+
+A switch to define the oldest file to download would be nice. Perhaps something with this format (for Jan 1st 2020): --oldest 20200101

diff --git a/doc/forum/best_way_to_move_a_git_annex_repo_trought_file_system/comment_2_e3e30ca123cd840288655c25b688cd66._comment b/doc/forum/best_way_to_move_a_git_annex_repo_trought_file_system/comment_2_e3e30ca123cd840288655c25b688cd66._comment
new file mode 100644
index 000000000..0dd3c5612
--- /dev/null
+++ b/doc/forum/best_way_to_move_a_git_annex_repo_trought_file_system/comment_2_e3e30ca123cd840288655c25b688cd66._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="ghen1"
+ avatar="http://cdn.libravatar.org/avatar/efd0e92b6198291138f0cd7aedbf86b6"
+ subject="comment 2"
+ date="2020-07-27T15:18:52Z"
+ content="""
+You can move the folder like any other folder.
+"""]]

diff --git a/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected.mdwn b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected.mdwn
new file mode 100644
index 000000000..f3d5e8ee3
--- /dev/null
+++ b/doc/bugs/kite_net_OSX__47__current__47___distribution_is_8.20200618_but_8.20200720.1_expected.mdwn
@@ -0,0 +1,36 @@
+### Please describe the problem.
+
+*As of filing:* 
+
+* [downloads.kitenet.net/git-annex/OSX/current/10.11_El_Capitan/git-annex.dmg.info](https://downloads.kitenet.net/git-annex/OSX/current/10.11_El_Capitan/git-annex.dmg.info) contains: `distributionVersion = "8.20200618"` 
+* whereas:  git-annex 8.20200720.1 (and 8.20200720) are the only versions published since 8.20200617. (There was no release matching 8.20200618) 
+
+*Expected behavior:*
+
+* `distributionVersion` and included `git-annex`'s version matches an announced released version
+
+### What steps will reproduce the problem?
+
+1. On macOS, download and deploy the current [git-annex.dmg](https://downloads.kitenet.net/git-annex/OSX/current/10.11_El_Capitan/git-annex.dmg)
+2. Open a terminal session and enter `git-annex version`
+
+### What version of git-annex are you using? On what operating system?
+
+[[!format sh """
+git-annex version: 8.20200617-gd46490c17
+
+$/usr/bin/sw_vers
+ ProductName:	Mac OS X
+ ProductVersion: 10.12.6
+ BuildVersion:	16G2136
+"""]]
+
+### Please provide any additional information below.
+
+* Assuming this is the same problem with the autobuilder described in [kite net OSX/current/ distribution is 7.20190913 but 7.20191009 expected](https://git-annex.branchable.com/bugs/kite_net_OSX__47__current__47___distribution_is_7.20190913_but_7.20191009_expected/)
+* Please give it a look and let us know when we have a distributionVersion matching release notes.  Would like to pickup the fixes described in 8.20200720(.1) news.
+
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Still rock solid maintaining the .dmgs and .pkgs for a 67GB [Munki](https://www.munki.org) repository.

diff --git a/doc/forum/standalone_tarballs_for_specific_versions/comment_10_3c41c48d40ae44129a80f2d2bef284ae._comment b/doc/forum/standalone_tarballs_for_specific_versions/comment_10_3c41c48d40ae44129a80f2d2bef284ae._comment
new file mode 100644
index 000000000..7124b3895
--- /dev/null
+++ b/doc/forum/standalone_tarballs_for_specific_versions/comment_10_3c41c48d40ae44129a80f2d2bef284ae._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="updating the standalone distribution"
+ date="2020-07-24T19:28:18Z"
+ content="""
+@joey, when you get a chance, could you add the standalone distribution for the current released version (8.20200720.1, not the bleeding-edge git) to the archive at downloads.kitenet.net?
+
+Would it be practical to document the process for creating a standalone distro, so that it can be built on conda-forge simultaneously with the corresponding non-standalone version?
+
+Thanks!
+"""]]

diff --git a/CHANGELOG b/CHANGELOG
index 4b1665c6e..4f4e5e97c 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -11,6 +11,9 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
   * move, copy --to: Sped up seeking files by 2x.
   * drop: Sped up seeking files to drop by 2x, and also some performance
     improvements to checking numcopies.
+  * Deal with unusual IFS settings in the shell scripts for linux
+    standalone and OSX app.
+    Thanks, Yaroslav Halchenko
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__.mdwn b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__.mdwn
index bee64d7f7..03e6a5fd9 100644
--- a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__.mdwn
+++ b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__.mdwn
@@ -24,3 +24,5 @@ most likely it is just a matter of sanitizing this variable in `runshell` or ali
 
 [[!meta author=yoh]]
 [[!tag projects/datalad]]
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_3_ba1b34111d47d58517ea64f537214e98._comment b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_3_ba1b34111d47d58517ea64f537214e98._comment
new file mode 100644
index 000000000..1416c7cd1
--- /dev/null
+++ b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_3_ba1b34111d47d58517ea64f537214e98._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-07-24T18:43:32Z"
+ content="""
+Applied a version of the patch without the :- , although it seems the :- is
+not a bashism after all. It still makes no sense to me unless there's some
+other setting that might make the shell blow up when IFS isn't set.
+"""]]
diff --git a/standalone/linux/skel/runshell b/standalone/linux/skel/runshell
index d44262759..4e3feb92e 100755
--- a/standalone/linux/skel/runshell
+++ b/standalone/linux/skel/runshell
@@ -4,6 +4,9 @@
 
 set -e
 
+orig_IFS="${IFS}"
+unset IFS
+
 os="$(uname -o 2>/dev/null || true)"
 base="$(dirname "$0")"
 
@@ -238,6 +241,11 @@ else
 	cmd=sh
 fi
 
+if [ -n "${orig_IFS}" ]; then
+	IFS="${orig_IFS}"
+	export IFS
+fi
+
 if [ -z "$tbase" ]; then
 	if [ "$useproot" ]; then
 		exec proot "$cmd" "$@"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/runshell b/standalone/osx/git-annex.app/Contents/MacOS/runshell
index 557c59e22..5572a9fd6 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/runshell
+++ b/standalone/osx/git-annex.app/Contents/MacOS/runshell
@@ -4,6 +4,9 @@
 
 set -e
 
+orig_IFS="${IFS}"
+unset IFS
+
 base="$(dirname "$0")"
 
 if [ ! -d "$base" ]; then
@@ -85,6 +88,11 @@ export GIT_ANNEX_DIR
 GIT_ANNEX_STANDLONE_ENV="PATH GIT_EXEC_PATH GIT_TEMPLATE_DIR"
 export GIT_ANNEX_STANDLONE_ENV
 
+if [ -n "${orig_IFS}" ]; then
+	IFS="${orig_IFS}"
+	export IFS
+fi
+
 if [ "$1" ]; then
 	cmd="$1"
 	shift 1

diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_2b7afff24ba025fe3e272ff6f607eadf._comment b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_2b7afff24ba025fe3e272ff6f607eadf._comment
new file mode 100644
index 000000000..fa6e1d5b4
--- /dev/null
+++ b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_2b7afff24ba025fe3e272ff6f607eadf._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-07-24T18:34:37Z"
+ content="""
+Hmm, that patch looks pretty good, but contains a bashism in the `${IFS:-}`
+and I don't actually understand what the point of that is, if seems to
+say it should expand to "" if IFS is not set, but it would expand to that
+anyway.
+"""]]

diff --git a/CHANGELOG b/CHANGELOG
index e56fac6c2..4b1665c6e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -11,7 +11,6 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
   * move, copy --to: Sped up seeking files by 2x.
   * drop: Sped up seeking files to drop by 2x, and also some performance
     improvements to checking numcopies.
-  * Unset IFS in shell scripts in the linux standalone build and OSX app.
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_7836dffec8167c51c91b96fba90bde27._comment b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_7836dffec8167c51c91b96fba90bde27._comment
deleted file mode 100644
index fc550f3f4..000000000
--- a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_7836dffec8167c51c91b96fba90bde27._comment
+++ /dev/null
@@ -1,26 +0,0 @@
-[[!comment format=mdwn
- username="joey"
- subject="""comment 2"""
- date="2020-07-24T18:17:12Z"
- content="""
-Hmm, so runshell has some things like
-
-	for lib in $(cat "$base/libdirs"); do
-		GIT_ANNEX_LD_LIBRARY_PATH=...
-
-Which, with an unusual IFS, probably set the environment variable to
-something bogus. Which seems the kind of thing that could break things
-badly enough to segfault.
-
-I have so far not been able to reproduce it, I tried:
-
-	tar zxvf ~/src/git-annex/tmp/git-annex-standalone-amd64.tar.gz
-	cd git-annex.linux/
-	IFS=$'\013' ./git-annex version
-
-But this may involve the system the standalone build is used in, if the
-problem is this is breaking its isolation from that system. So might need a
-particular version skew between the two.
-
-I've applied the obvious changes, so please verify the fix.
-"""]]
diff --git a/standalone/linux/skel/git b/standalone/linux/skel/git
index b2b09d9c9..8a6860f43 100755
--- a/standalone/linux/skel/git
+++ b/standalone/linux/skel/git
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-annex b/standalone/linux/skel/git-annex
index 668eb8ad2..ec9739a8b 100755
--- a/standalone/linux/skel/git-annex
+++ b/standalone/linux/skel/git-annex
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-annex-shell b/standalone/linux/skel/git-annex-shell
index 9414b2091..d66338491 100755
--- a/standalone/linux/skel/git-annex-shell
+++ b/standalone/linux/skel/git-annex-shell
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-annex-webapp b/standalone/linux/skel/git-annex-webapp
index 710f49634..914c5923c 100755
--- a/standalone/linux/skel/git-annex-webapp
+++ b/standalone/linux/skel/git-annex-webapp
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-receive-pack b/standalone/linux/skel/git-receive-pack
index 01d1f5925..f4f9f262a 100755
--- a/standalone/linux/skel/git-receive-pack
+++ b/standalone/linux/skel/git-receive-pack
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-shell b/standalone/linux/skel/git-shell
index 4947e47ed..0fc5cd7bd 100755
--- a/standalone/linux/skel/git-shell
+++ b/standalone/linux/skel/git-shell
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-upload-pack b/standalone/linux/skel/git-upload-pack
index 11dc9a98d..312996cec 100755
--- a/standalone/linux/skel/git-upload-pack
+++ b/standalone/linux/skel/git-upload-pack
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/runshell b/standalone/linux/skel/runshell
index ec44d3364..d44262759 100755
--- a/standalone/linux/skel/runshell
+++ b/standalone/linux/skel/runshell
@@ -3,7 +3,6 @@
 # libraries bundled with this app.
 
 set -e
-unset IFS
 
 os="$(uname -o 2>/dev/null || true)"
 base="$(dirname "$0")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git b/standalone/osx/git-annex.app/Contents/MacOS/git
index 69ff306a9..ccf2cd741 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-annex b/standalone/osx/git-annex.app/Contents/MacOS/git-annex
index 761630c08..bb82a2bd9 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-annex
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-annex
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell
index 6fa8a649a..29ebc2588 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp
index 55d56ce68..7ff18bada 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack b/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack
index e3099f0f2..efbbf2bb3 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-shell b/standalone/osx/git-annex.app/Contents/MacOS/git-shell
index a9317e364..7a74904f2 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-shell
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-shell
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack b/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack
index 66ed2dc63..5049371c9 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack
@@ -1,5 +1,4 @@
 #!/bin/sh
-unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/runshell b/standalone/osx/git-annex.app/Contents/MacOS/runshell
index f02a02e6b..557c59e22 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/runshell
+++ b/standalone/osx/git-annex.app/Contents/MacOS/runshell

(Diff truncated)

diff --git a/CHANGELOG b/CHANGELOG
index 4b1665c6e..e56fac6c2 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -11,6 +11,7 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
   * move, copy --to: Sped up seeking files by 2x.
   * drop: Sped up seeking files to drop by 2x, and also some performance
     improvements to checking numcopies.
+  * Unset IFS in shell scripts in the linux standalone build and OSX app.
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_7836dffec8167c51c91b96fba90bde27._comment b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_7836dffec8167c51c91b96fba90bde27._comment
new file mode 100644
index 000000000..fc550f3f4
--- /dev/null
+++ b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_2_7836dffec8167c51c91b96fba90bde27._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-07-24T18:17:12Z"
+ content="""
+Hmm, so runshell has some things like
+
+	for lib in $(cat "$base/libdirs"); do
+		GIT_ANNEX_LD_LIBRARY_PATH=...
+
+Which, with an unusual IFS, probably set the environment variable to
+something bogus. Which seems the kind of thing that could break things
+badly enough to segfault.
+
+I have so far not been able to reproduce it, I tried:
+
+	tar zxvf ~/src/git-annex/tmp/git-annex-standalone-amd64.tar.gz
+	cd git-annex.linux/
+	IFS=$'\013' ./git-annex version
+
+But this may involve the system the standalone build is used in, if the
+problem is this is breaking its isolation from that system. So might need a
+particular version skew between the two.
+
+I've applied the obvious changes, so please verify the fix.
+"""]]
diff --git a/standalone/linux/skel/git b/standalone/linux/skel/git
index 8a6860f43..b2b09d9c9 100755
--- a/standalone/linux/skel/git
+++ b/standalone/linux/skel/git
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-annex b/standalone/linux/skel/git-annex
index ec9739a8b..668eb8ad2 100755
--- a/standalone/linux/skel/git-annex
+++ b/standalone/linux/skel/git-annex
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-annex-shell b/standalone/linux/skel/git-annex-shell
index d66338491..9414b2091 100755
--- a/standalone/linux/skel/git-annex-shell
+++ b/standalone/linux/skel/git-annex-shell
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-annex-webapp b/standalone/linux/skel/git-annex-webapp
index 914c5923c..710f49634 100755
--- a/standalone/linux/skel/git-annex-webapp
+++ b/standalone/linux/skel/git-annex-webapp
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-receive-pack b/standalone/linux/skel/git-receive-pack
index f4f9f262a..01d1f5925 100755
--- a/standalone/linux/skel/git-receive-pack
+++ b/standalone/linux/skel/git-receive-pack
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-shell b/standalone/linux/skel/git-shell
index 0fc5cd7bd..4947e47ed 100755
--- a/standalone/linux/skel/git-shell
+++ b/standalone/linux/skel/git-shell
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/git-upload-pack b/standalone/linux/skel/git-upload-pack
index 312996cec..11dc9a98d 100755
--- a/standalone/linux/skel/git-upload-pack
+++ b/standalone/linux/skel/git-upload-pack
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink -f "$0" 2>/dev/null || readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/linux/skel/runshell b/standalone/linux/skel/runshell
index d44262759..ec44d3364 100755
--- a/standalone/linux/skel/runshell
+++ b/standalone/linux/skel/runshell
@@ -3,6 +3,7 @@
 # libraries bundled with this app.
 
 set -e
+unset IFS
 
 os="$(uname -o 2>/dev/null || true)"
 base="$(dirname "$0")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git b/standalone/osx/git-annex.app/Contents/MacOS/git
index ccf2cd741..69ff306a9 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-annex b/standalone/osx/git-annex.app/Contents/MacOS/git-annex
index bb82a2bd9..761630c08 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-annex
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-annex
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell
index 29ebc2588..6fa8a649a 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-shell
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp
index 7ff18bada..55d56ce68 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-annex-webapp
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack b/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack
index efbbf2bb3..e3099f0f2 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-receive-pack
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-shell b/standalone/osx/git-annex.app/Contents/MacOS/git-shell
index 7a74904f2..a9317e364 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-shell
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-shell
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack b/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack
index 5049371c9..66ed2dc63 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack
+++ b/standalone/osx/git-annex.app/Contents/MacOS/git-upload-pack
@@ -1,4 +1,5 @@
 #!/bin/sh
+unset IFS
 link="$(readlink "$0")" || true
 if [ -n "$link" ]; then
 	base="$(dirname "$link")"
diff --git a/standalone/osx/git-annex.app/Contents/MacOS/runshell b/standalone/osx/git-annex.app/Contents/MacOS/runshell
index 557c59e22..f02a02e6b 100755
--- a/standalone/osx/git-annex.app/Contents/MacOS/runshell
+++ b/standalone/osx/git-annex.app/Contents/MacOS/runshell

(Diff truncated)

diff --git a/doc/devblog/day_627__last_performance_work_for_now.mdwn b/doc/devblog/day_627__last_performance_work_for_now.mdwn
new file mode 100644
index 000000000..f3417f02d
--- /dev/null
+++ b/doc/devblog/day_627__last_performance_work_for_now.mdwn
@@ -0,0 +1,16 @@
+One more day working on performance, as I had a few known improvements I had
+not had time to get to. Managed to double the speed of `move --to, `copy
+--to`, and `drop` when seeking files to act on and a few percent more in
+general.
+
+My laptop's keyboard is failing, with more and more keys not working --
+luckily so far only ones in the number row -- so I'm stopping early and
+hoping the fix arrives quickly on Monday. At some point I know
+that [[this_todo|todo/faster_key_lookup_for_limits]] will be able to speed
+up using things like `--in` and `--copies` by a similar amount as the recent
+performance improvements.
+
+--
+
+Today's work was sponsored by Jake Vosloo
+[on Patreon](https://patreon.com/joeyh).

diff --git a/doc/todo/importing_from_special_remote_without_downloading/comment_11_06cd77a9265289decceb5c3b4024db4b._comment b/doc/todo/importing_from_special_remote_without_downloading/comment_11_06cd77a9265289decceb5c3b4024db4b._comment
new file mode 100644
index 000000000..93b1bff1c
--- /dev/null
+++ b/doc/todo/importing_from_special_remote_without_downloading/comment_11_06cd77a9265289decceb5c3b4024db4b._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 11"""
+ date="2020-07-24T17:50:03Z"
+ content="""
+Yes, it can also happen with addurl, but I think it's less likely that two
+users add the same url with and without --fast or --relaxed than that two
+users sync with the same remote with and without --content.
+
+Anyway, I opened [[sync_fast_import]].
+"""]]
diff --git a/doc/todo/sync_fast_import.mdwn b/doc/todo/sync_fast_import.mdwn
new file mode 100644
index 000000000..35f4b53f8
--- /dev/null
+++ b/doc/todo/sync_fast_import.mdwn
@@ -0,0 +1,34 @@
+git-annex import --no-content from a directory special remote is
+implemented, but git-annex sync, when run without --content, does not
+operate on import/export special remotes. This is inconsistent, and it
+would be useful if it did.
+
+<https://git-annex.branchable.com/todo/importing_from_special_remote_without_downloading/#comment-e3db95e073f01a05b205e26f422f5bc5>
+describes a problem with doing that, involving merge conflicts. That
+should not actually happen with the directory special remote, because
+it generates the same key with importKey as git-annex import would.
+But other special remotes later using this interface might generate a key
+using a different hash than usual.
+
+The suggestion there is that there could be a separate config that controls
+whether sync does a fast import or an import with content. Then when sync
+is run without --content, it can do a fast import. And when run with
+--content, it can do a fast import, followed by getting the content.
+
+Or maybe that should just be what it always does, when a remote supports
+importKey? (If so, git-annex import should do the same.) Yeah, this seems
+better than a config. Look at it like this: The special remote makes pseudo
+"commits" when changes are made to it. And maybe it choses to use a
+different kind of key than the local repository would use. Same could
+happen when pulling from someone else's repo, if they've configured
+git-annex to use a different backend.
+
+Except.. --no-content means annex.largefiles is not checked, so non-large
+files get added as annexed files. That's done because annex.largefiles
+can contain expressions that need to examine the content of the file.
+In particulat for mimetype and mimeencoding.
+So there would still be a conflict potential.
+
+May be worth removing support for matching annex.largefiles when the
+expression needs the file content, when importing from a special remote.
+Or could detect when those are used, and only import with --content then.

diff --git a/doc/install/Windows/comment_12_3a7080a31d30ccb32fede56a98bb635a._comment b/doc/install/Windows/comment_12_3a7080a31d30ccb32fede56a98bb635a._comment
new file mode 100644
index 000000000..09b70f9ca
--- /dev/null
+++ b/doc/install/Windows/comment_12_3a7080a31d30ccb32fede56a98bb635a._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="nix.zahlen@1211ac6c964ba2d68b70655f747bef1383032e77"
+ nickname="nix.zahlen"
+ avatar="http://cdn.libravatar.org/avatar/66cc45a05749fe8d4ca36d8c6071da51"
+ subject="It is really a pitty that the Windows port / version is unmaintained"
+ date="2020-07-24T17:48:07Z"
+ content="""
+Is there maybe a how-to somewhere which describes how we could compile the source on Windows to have more recent versions?!
+"""]]

diff --git a/CHANGELOG b/CHANGELOG
index d7b353ae7..4b1665c6e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -8,7 +8,7 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
     some weird inheriting of ssh FDs by sshd. Bug was introduced in
     git-annex version 7.20200202.7.
   * Fix a bug in find --branch in the previous version.
-  * move, copy: Some performance improvements.
+  * move, copy --to: Sped up seeking files by 2x.
   * drop: Sped up seeking files to drop by 2x, and also some performance
     improvements to checking numcopies.
 
diff --git a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
index 6ab4b7887..76c0ab0f5 100644
--- a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
+++ b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
@@ -33,10 +33,8 @@ and precache them.
 > > > * `sync --content` 2x speedup!
 > > > * `fsck --fast` 1.5x speedup
 > > > * `whereis` 1.5x speedup
+> > > * `copy --to` 2x speedup when remote has all files
 > > > 
-> > > move, copy, and drop probably are also faster, but the work will
-> > > dominate
-> > >
 > > > [[done]]
 
 Another thing that the same cat-file --buffer approach could be used with

diff --git a/Annex/Drop.hs b/Annex/Drop.hs
index af603ac98..57ca15bc3 100644
--- a/Annex/Drop.hs
+++ b/Annex/Drop.hs
@@ -17,6 +17,7 @@ import qualified Command.Drop
 import Command
 import Annex.Wanted
 import Annex.SpecialRemote.Config
+import Annex.Content
 import qualified Database.Keys
 import Git.FilePath
 
@@ -118,7 +119,8 @@ handleDropsFrom locs rs reason fromhere key afile preverified runner = do
 			)
 
 	dropl fs n = checkdrop fs n Nothing $ \numcopies ->
-		Command.Drop.startLocal afile ai numcopies key preverified
+		stopUnless (inAnnex key) $
+			Command.Drop.startLocal afile ai numcopies key preverified
 
 	dropr fs r n  = checkdrop fs n (Just $ Remote.uuid r) $ \numcopies ->
 		Command.Drop.startRemote afile ai numcopies key r
diff --git a/CHANGELOG b/CHANGELOG
index 6dc53d6b5..d7b353ae7 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -8,8 +8,9 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
     some weird inheriting of ssh FDs by sshd. Bug was introduced in
     git-annex version 7.20200202.7.
   * Fix a bug in find --branch in the previous version.
-  * move, copy: Sped up seeking for annexed files to operate on by a factor
-    of nearly 2x.
+  * move, copy: Some performance improvements.
+  * drop: Sped up seeking files to drop by 2x, and also some performance
+    improvements to checking numcopies.
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/Command/Drop.hs b/Command/Drop.hs
index ccac908ad..c81b727c1 100644
--- a/Command/Drop.hs
+++ b/Command/Drop.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2010 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -52,53 +52,55 @@ parseDropFromOption = parseRemoteOption <$> strOption
 	)
 
 seek :: DropOptions -> CommandSeek
-seek o = startConcurrency commandStages $
+seek o = startConcurrency commandStages $ do
+	from <- case dropFrom o of
+		Nothing -> pure Nothing
+		Just f -> getParsed f >>= \remote -> do
+			u <- getUUID
+			if Remote.uuid remote == u
+				then pure Nothing
+				else pure (Just remote)				
+	let seeker = AnnexedFileSeeker
+		{ startAction = start o from
+		, checkContentPresent = case from of
+			Nothing -> Just True
+			Just _ -> Nothing
+		, usesLocationLog = True
+		}
 	case batchOption o of
 		Batch fmt -> batchAnnexedFilesMatching fmt seeker
 		NoBatch -> withKeyOptions (keyOptions o) (autoMode o) seeker
-			(commandAction . startKeys o)
+			(commandAction . startKeys o from)
 			(withFilesInGitAnnex ww seeker)
 			=<< workTreeItems ww (dropFiles o)
   where
 	ww = WarnUnmatchLsFiles
 
-	seeker = AnnexedFileSeeker
-		{ startAction = start o
-		, checkContentPresent = Nothing
-		, usesLocationLog = False
-		}
-
-start :: DropOptions -> RawFilePath -> Key -> CommandStart
-start o file key = start' o key afile ai
+start :: DropOptions -> Maybe Remote -> RawFilePath -> Key -> CommandStart
+start o from file key = start' o from key afile ai
   where
 	afile = AssociatedFile (Just file)
 	ai = mkActionItem (key, afile)
 
-start' :: DropOptions -> Key -> AssociatedFile -> ActionItem -> CommandStart
-start' o key afile ai = do
-	from <- maybe (pure Nothing) (Just <$$> getParsed) (dropFrom o)
+start' :: DropOptions -> Maybe Remote -> Key -> AssociatedFile -> ActionItem -> CommandStart
+start' o from key afile ai = 
 	checkDropAuto (autoMode o) from afile key $ \numcopies ->
-		stopUnless (want from) $
+		stopUnless want $
 			case from of
 				Nothing -> startLocal afile ai numcopies key []
-				Just remote -> do
-					u <- getUUID
-					if Remote.uuid remote == u
-						then startLocal afile ai numcopies key []
-						else startRemote afile ai numcopies key remote
-	  where
-		want from
-			| autoMode o = wantDrop False (Remote.uuid <$> from) (Just key) afile
-			| otherwise = return True
-
-startKeys :: DropOptions -> (Key, ActionItem) -> CommandStart
-startKeys o (key, ai) = start' o key (AssociatedFile Nothing) ai
+				Just remote -> startRemote afile ai numcopies key remote
+  where
+	want
+		| autoMode o = wantDrop False (Remote.uuid <$> from) (Just key) afile
+		| otherwise = return True
+
+startKeys :: DropOptions -> Maybe Remote -> (Key, ActionItem) -> CommandStart
+startKeys o from (key, ai) = start' o from key (AssociatedFile Nothing) ai
 
 startLocal :: AssociatedFile -> ActionItem -> NumCopies -> Key -> [VerifiedCopy] -> CommandStart
 startLocal afile ai numcopies key preverified =
-	stopUnless (inAnnex key) $
-		starting "drop" (OnlyActionOn key ai) $
-			performLocal key afile numcopies preverified
+	starting "drop" (OnlyActionOn key ai) $
+		performLocal key afile numcopies preverified
 
 startRemote :: AssociatedFile -> ActionItem -> NumCopies -> Key -> Remote -> CommandStart
 startRemote afile ai numcopies key remote = 
diff --git a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
index da6962b65..6ab4b7887 100644
--- a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
+++ b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
@@ -33,11 +33,9 @@ and precache them.
 > > > * `sync --content` 2x speedup!
 > > > * `fsck --fast` 1.5x speedup
 > > > * `whereis` 1.5x speedup
-> > > * `copy --to --fast` twenty-five percent or so speedup
-> > > * `copy --to` 2x speedup
-> > > * `copy --from` 2x speedup
-> > >
-> > > For copy benchmarks, note that both repos had all files.
+> > > 
+> > > move, copy, and drop probably are also faster, but the work will
+> > > dominate
 > > >
 > > > [[done]]
 

diff --git a/CHANGELOG b/CHANGELOG
index a51cf9997..6dc53d6b5 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -8,6 +8,8 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
     some weird inheriting of ssh FDs by sshd. Bug was introduced in
     git-annex version 7.20200202.7.
   * Fix a bug in find --branch in the previous version.
+  * move, copy: Sped up seeking for annexed files to operate on by a factor
+    of nearly 2x.
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/Command/Copy.hs b/Command/Copy.hs
index 75ede060e..5a8b185c4 100644
--- a/Command/Copy.hs
+++ b/Command/Copy.hs
@@ -57,8 +57,11 @@ seek o = startConcurrency commandStages $ do
 	
 	seeker = AnnexedFileSeeker
 		{ startAction = start o
-		, checkContentPresent = Nothing
-		, usesLocationLog = False
+		, checkContentPresent = case fromToOptions o of
+			Right (FromRemote _) -> Just False
+			Right (ToRemote _) -> Just True
+			Left ToHere -> Just False
+		, usesLocationLog = True
 		}
 
 {- A copy is just a move that does not delete the source file.
diff --git a/Command/Move.hs b/Command/Move.hs
index bbb4a0dc2..ba3444133 100644
--- a/Command/Move.hs
+++ b/Command/Move.hs
@@ -55,11 +55,6 @@ data RemoveWhen = RemoveSafe | RemoveNever
 
 seek :: MoveOptions -> CommandSeek
 seek o = startConcurrency stages $ do
-	let seeker = AnnexedFileSeeker
-		{ startAction = start (fromToOptions o) (removeWhen o)
-		, checkContentPresent = Nothing
-		, usesLocationLog = False
-		}
 	case batchOption o of
 		NoBatch -> withKeyOptions (keyOptions o) False seeker
 			(commandAction . startKey (fromToOptions o) (removeWhen o))
@@ -67,6 +62,14 @@ seek o = startConcurrency stages $ do
 			=<< workTreeItems ww (moveFiles o)
 		Batch fmt -> batchAnnexedFilesMatching fmt seeker
   where
+	seeker = AnnexedFileSeeker
+		{ startAction = start (fromToOptions o) (removeWhen o)
+		, checkContentPresent = case fromToOptions o of
+			Right (FromRemote _) -> Nothing
+			Right (ToRemote _) -> Just True
+			Left ToHere -> Nothing
+		, usesLocationLog = True
+		}
 	stages = case fromToOptions o of
 		Right (FromRemote _) -> downloadStages
 		Right (ToRemote _) -> commandStages
@@ -103,9 +106,8 @@ describeMoveAction _ = "move"
 toStart :: RemoveWhen -> AssociatedFile -> Key -> ActionItem -> Remote -> CommandStart
 toStart removewhen afile key ai dest = do
 	u <- getUUID
-	ishere <- inAnnex key
-	if not ishere || u == Remote.uuid dest
-		then stop -- not here, so nothing to do
+	if u == Remote.uuid dest
+		then stop
 		else toStart' dest removewhen afile key ai
 
 toStart' :: Remote -> RemoveWhen -> AssociatedFile -> Key -> ActionItem -> CommandStart
@@ -188,11 +190,8 @@ toPerform dest removewhen key afile fastcheck isthere =
 			return False
 
 fromStart :: RemoveWhen -> AssociatedFile -> Key -> ActionItem -> Remote -> CommandStart
-fromStart removewhen afile key ai src = case removewhen of
-	RemoveNever -> stopUnless (not <$> inAnnex key) go
-	RemoveSafe -> go
-  where
-	go = stopUnless (fromOk src key) $
+fromStart removewhen afile key ai src = 
+	stopUnless (fromOk src key) $
 		starting (describeMoveAction removewhen) (OnlyActionOn key ai) $
 			fromPerform src removewhen key afile
 
@@ -247,11 +246,8 @@ fromPerform src removewhen key afile = do
  - When moving, the content is removed from all the reachable remotes that
  - it can safely be removed from. -}
 toHereStart :: RemoveWhen -> AssociatedFile -> Key -> ActionItem -> CommandStart
-toHereStart removewhen afile key ai = case removewhen of
-	RemoveNever -> stopUnless (not <$> inAnnex key) go
-	RemoveSafe -> go
-  where
-	go = startingNoMessage (OnlyActionOn key ai) $ do
+toHereStart removewhen afile key ai = 
+	startingNoMessage (OnlyActionOn key ai) $ do
 		rs <- Remote.keyPossibilities key
 		forM_ rs $ \r ->
 			includeCommandAction $
diff --git a/doc/todo/faster_key_lookup_for_limits.mdwn b/doc/todo/faster_key_lookup_for_limits.mdwn
new file mode 100644
index 000000000..b9cac1164
--- /dev/null
+++ b/doc/todo/faster_key_lookup_for_limits.mdwn
@@ -0,0 +1,9 @@
+As part of the work in [[precache_logs_for_speed_with_cat-file_--buffer]],
+key lookups are now done twice as fast as before.
+
+But, limits that look up keys still do a key lookup, before the key
+is looked up efficiently. Avoiding that would speed up --in etc, probably
+another 1.5x-2x speedup when such limits are used. What that optimisation
+needs is a way to tell if the current limit needs the key or not. If it
+does, then match on it after getting the key (and precaching the location
+log for limits that need that), otherwise before getting the key.
diff --git a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
index 127e7e717..da6962b65 100644
--- a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
+++ b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
@@ -33,10 +33,13 @@ and precache them.
 > > > * `sync --content` 2x speedup!
 > > > * `fsck --fast` 1.5x speedup
 > > > * `whereis` 1.5x speedup
+> > > * `copy --to --fast` twenty-five percent or so speedup
+> > > * `copy --to` 2x speedup
+> > > * `copy --from` 2x speedup
 > > >
-> > > Still todo:
-> > > 
-> > > * move, copy, drop, and mirror were left not using the location log caching yet
+> > > For copy benchmarks, note that both repos had all files.
+> > >
+> > > [[done]]
 
 Another thing that the same cat-file --buffer approach could be used with
 is to cat the annex links. Git.LsFiles.inRepoDetails provides the Sha
@@ -52,10 +55,4 @@ Some calls to lookupKey remain, and the above could
 be used to remove them and make it faster. The ones in Annex.View and
 Command.Unused seem most likely to be able to be converted.
 
-Also, limits that look up keys still do a key lookup, before the key is
-looked up efficiently. (Before these changes, the same key lookup was done
-2x too..) Avoiding that would speed up --in etc, probably another 1.5x-2x
-speedup when such limits are used. What that optimisation needs is a way to
-tell if the current limit needs the key or not. If it does, then match on
-it after getting the key (and precaching the location log for limits that
-need that), otherwise before getting the key.
+See also [[faster_key_lookup_for_limits]]

diff --git a/doc/forum/Exporting_files_to_special_remote__44___even_if_not_in_preferred_contents/comment_2_dc1e718d0043686987abc9526a459dc3._comment b/doc/forum/Exporting_files_to_special_remote__44___even_if_not_in_preferred_contents/comment_2_dc1e718d0043686987abc9526a459dc3._comment
new file mode 100644
index 000000000..17ac43d13
--- /dev/null
+++ b/doc/forum/Exporting_files_to_special_remote__44___even_if_not_in_preferred_contents/comment_2_dc1e718d0043686987abc9526a459dc3._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="erewhon"
+ avatar="http://cdn.libravatar.org/avatar/b9bd5ad7176ebe149d0f051dcfe0a63e"
+ subject="comment 2"
+ date="2020-07-24T16:05:06Z"
+ content="""
+Thanks Ilya,
+
+This seems like a very nice solution. I had not known about metadata tags. I look forward to finding other uses for them as well.
+"""]]

diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_1_ab8017246c0075b731c7d31235672c01._comment b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_1_ab8017246c0075b731c7d31235672c01._comment
new file mode 100644
index 000000000..a8cccc075
--- /dev/null
+++ b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__/comment_1_ab8017246c0075b731c7d31235672c01._comment
@@ -0,0 +1,66 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 1"
+ date="2020-07-24T15:22:36Z"
+ content="""
+- unsetting `IFS` at top of `runshell` \"fixes\" it but IMHO would not be proper since shell completion etc tools relying on passing it into calls of git etc would be effected.
+
+- 
+
+<details><summary>`bash` somehow seems to be also avoiding the segfault:</summary>
+
+```
+$> IFS=$'\013' dash /usr/lib/git-annex.linux/git-annex version
+[2]    1059479 segmentation fault (core dumped)  IFS=$'\013' dash /usr/lib/git-annex.linux/git-annex version
+1 9812 ->139 [1].....................................:Fri 24 Jul 2020 11:09:07 AM EDT:.
+(git)lena:~/proj/misc/git[master]git
+$> IFS=$'\013' posh /usr/lib/git-annex.linux/git-annex version    
+[2]    1059547 segmentation fault (core dumped)  IFS=$'\013' posh /usr/lib/git-annex.linux/git-annex version
+1 9813 ->139 [1].....................................:Fri 24 Jul 2020 11:09:12 AM EDT:.
+(git)lena:~/proj/misc/git[master]git
+$> IFS=$'\013' bash /usr/lib/git-annex.linux/git-annex version    
+git-annex version: 8.20200617+git192-g5849bd634-1~ndall+1
+build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
+dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.0.0 ghc-8.4.4 http-client-0.5.13.1 persistent-sqlite-2.8.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs hook external
+operating system: linux x86_64
+supported repository versions: 8
+upgrade supported from repository versions: 0 1 2 3 4 5 6 7
+```
+</details>
+
+- most likely `runshell` (etc) shims within standalone need to sanitize IFS only when they invoke some tools they use and which rely on IFS, but pass IFS as is into the actual call.  It seems that the following patch does the trick
+
+```diff
+$> diff -Naur /home/yoh/proj/git-annex/standalone/linux/skel/runshell /usr/lib/git-annex.linux/runshell
+--- /home/yoh/proj/git-annex/standalone/linux/skel/runshell	2020-03-08 10:37:15.462465282 -0400
++++ /usr/lib/git-annex.linux/runshell	2020-07-24 11:19:17.707935260 -0400
+@@ -4,6 +4,9 @@
+ 
+ set -e
+ 
++orig_IFS=\"${IFS:-}\"
++unset IFS
++
+ os=\"$(uname -o 2>/dev/null || true)\"
+ base=\"$(dirname \"$0\")\"
+ 
+@@ -238,6 +241,11 @@
+ 	cmd=sh
+ fi
+ 
++if [ -n \"${orig_IFS}\" ]; then
++	IFS=\"${orig_IFS}\"
++	export IFS
++fi
++
+ if [ -z \"$tbase\" ]; then
+ 	if [ \"$useproot\" ]; then
+ 		exec proot \"$cmd\" \"$@\"
+```
+
+and it seems that git shell completion then still works fine
+
+"""]]

diff --git a/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__.mdwn b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__.mdwn
new file mode 100644
index 000000000..bee64d7f7
--- /dev/null
+++ b/doc/bugs/standalone_shim_is_segfaulting_if_IFS__61____36____39____92__013__39__.mdwn
@@ -0,0 +1,26 @@
+### Please describe the problem.
+
+I did manage to reach the rabbit hole bottom in the troubleshooting of my unique inability to use argcomplete for shell completion in datalad: [datalad issue](https://github.com/datalad/datalad/issues/4538) [argcomplete issue](https://github.com/kislyuk/argcomplete/issues/303)
+
+And the bottom looked liked:
+
+```
+$> IFS=$'\013' /usr/lib/git-annex.linux/git-annex version
+[1]    1040489 segmentation fault (core dumped)  IFS=$'\013' /usr/lib/git-annex.linux/git-annex version
+
+$> IFS=$'\013' /usr/lib/git-annex.linux/git version      
+[1]    1040532 segmentation fault (core dumped)  IFS=$'\013' /usr/lib/git-annex.linux/git version
+
+```
+
+whenever stock git is ok
+
+```
+$> IFS=$'\013' /usr/bin/git version  
+git version 2.27.0
+```
+
+most likely it is just a matter of sanitizing this variable in `runshell` or alike.
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

diff --git a/doc/forum/Please_help_me_help_my_photographer_friends.mdwn b/doc/forum/Please_help_me_help_my_photographer_friends.mdwn
new file mode 100644
index 000000000..035819f9b
--- /dev/null
+++ b/doc/forum/Please_help_me_help_my_photographer_friends.mdwn
@@ -0,0 +1,18 @@
+Hi everyone,
+
+first thanks for the great software and community support. I am a big fan of git-annex for the longest time, even participated the kickstarter back then. Throughout this time, I also have helping a few professional photographers to manage their files. I can very much relate to Joeys original story for the assistant, thus felt the potential for it to be very useful for them.
+
+However, even though I use git-annex personally and even tried a few projects with it, I did not manage to create the solution I envisioned for my friends. The use case is more about managing space, backing up and archiving. I will give more details below, but I am starting to lose hope. I tried a couple of approaches ([gateway][1], [rsync.net][2]) to no avail, but I still have the feeling it should be possible. Therefore, I bring forth my idea to put it out of its misery:
+
+  - The goal is to (a) use the glacier archive feature to get rid of closed projects, freeing up space and (b) to have a “cloud backup” of the worktree. The backup ideally would be something I could ``git clone`` in case of a disaster and still would be able to recover archived files from glacier (by moving it out of “archive” folder).
+
+  - Very importantly restriction, these users have no tech background, thus the whole process must be automated and as error free as possible. On the other hand, it does not need to be online, or even use the assistant at all.  It would be fine to have rules like “takes 24h to recover an archive” (because of ``cron`` or whatever). 
+
+One optional feature is finding a use for the tons of external hard drives they hoarded over the years. The biggest hope here is not to have to rely on cheap hardware to store a lifework as soon as possible. Even though I try to help them to catalog projects and maintain multiple copies, it is inevitable that now and then something fails. I was really hopping git-annex would help here.
+
+Thanks in advance and all the best!
+
+f.
+
+[1]: https://git-annex.branchable.com/forum/Is_it_possible_to_setup_a_git-annex_Glacier_gatway__63__/
+[2]: https://git-annex.branchable.com/forum/using_rsync.net_as_backup_and_glacier_as_archive/

diff --git a/doc/bugs/git_annex_sync_remove_all_recently_added_files.mdwn b/doc/bugs/git_annex_sync_remove_all_recently_added_files.mdwn
index b50197be0..e277633da 100644
--- a/doc/bugs/git_annex_sync_remove_all_recently_added_files.mdwn
+++ b/doc/bugs/git_annex_sync_remove_all_recently_added_files.mdwn
@@ -4,15 +4,15 @@
 
 since a bit of time i have a strange probleme. My git annex environement look like this :
 
-A E
-|\ /
-| C--D
-|/ \
-B F.....G
+A E  
+|\ /  
+| C--D  
+|/ \  
+B F.....G  
 
-the dot point because G is an external drive and can be plug anywhere else. let's say my last comit is db1a9a If i add some file on the repo from G and sync point to point like : i'm in G : git annex sync F a new commit is created say 61dacd go F git annex sync D go D git annex sync E ...etc
+the dot point because G is an external drive and can be plug anywhere else. let's say my last comit is **db1a9a** If i add some file on the repo from G and sync point to point like : i'm in G : git annex sync F a new commit is created say **61dacd** go F git annex sync D go D git annex sync E ...etc
 
-at some point my commit 61dacd becomme the parent of a new commit i dono the source ... let's say 4fcdae
+at some point my commit **61dacd** becomme the parent of a new commit i dono the source ... let's say **4fcdae**
 
 in git log i found
 
@@ -20,11 +20,11 @@ commit 4fcdae... (HEAD -> master, B/synced/ma ster, B/master, C/synced/master, C
 
 and that commit just undo all the change done by the previous one, indeed :
 
-git diff 4fcdae db1a9a -> nothing
+git diff **4fcdae** **db1a9a** -> nothing
 
 this... force me to play with branch like every time and anoyed me alot. I don't understand what's happening here. Does any one have any clue/insight on what i can do to fix this/investigate ?
 
-All repo are at the commit db1a9a before i start...
+All repo are at the commit **db1a9a** before i start...
 
 best regards
 
@@ -35,12 +35,12 @@ i'm not sur on how i can reproduce it. It happens time to time.
 
 ### What version of git-annex are you using? On what operating system?
 
-Ubuntu 18.04.4 LTS -> git-annex version: 6.20180227
-Ubuntu 14.04.6 LTS -> git-annex version: 5.20140412ubuntu1
-freebsd 12.1-RELEASE-p3 -> git-annex version: 7.20190626
-debian 10.4 -> git-annex version: 7.20190129
-debian 8.11 -> git-annex version: 5.20141125+oops-1+deb8u2
-Ubuntu 18.04.4 LTS -> git-annex version: 6.20180227
+Ubuntu 18.04.4 LTS -> git-annex version: 6.20180227  
+Ubuntu 14.04.6 LTS -> git-annex version: 5.20140412ubuntu1  
+freebsd 12.1-RELEASE-p3 -> git-annex version: 7.20190626    
+debian 10.4 -> git-annex version: 7.20190129  
+debian 8.11 -> git-annex version: 5.20141125+oops-1+deb8u2  
+Ubuntu 18.04.4 LTS -> git-annex version: 6.20180227  
 
 
 

diff --git a/doc/bugs/git_annex_sync_remove_all_recently_added_files.mdwn b/doc/bugs/git_annex_sync_remove_all_recently_added_files.mdwn
new file mode 100644
index 000000000..b50197be0
--- /dev/null
+++ b/doc/bugs/git_annex_sync_remove_all_recently_added_files.mdwn
@@ -0,0 +1,59 @@
+### Please describe the problem.
+
+
+
+since a bit of time i have a strange probleme. My git annex environement look like this :
+
+A E
+|\ /
+| C--D
+|/ \
+B F.....G
+
+the dot point because G is an external drive and can be plug anywhere else. let's say my last comit is db1a9a If i add some file on the repo from G and sync point to point like : i'm in G : git annex sync F a new commit is created say 61dacd go F git annex sync D go D git annex sync E ...etc
+
+at some point my commit 61dacd becomme the parent of a new commit i dono the source ... let's say 4fcdae
+
+in git log i found
+
+commit 4fcdae... (HEAD -> master, B/synced/ma ster, B/master, C/synced/master, C/master, synced/master)
+
+and that commit just undo all the change done by the previous one, indeed :
+
+git diff 4fcdae db1a9a -> nothing
+
+this... force me to play with branch like every time and anoyed me alot. I don't understand what's happening here. Does any one have any clue/insight on what i can do to fix this/investigate ?
+
+All repo are at the commit db1a9a before i start...
+
+best regards
+
+
+### What steps will reproduce the problem?
+
+i'm not sur on how i can reproduce it. It happens time to time.
+
+### What version of git-annex are you using? On what operating system?
+
+Ubuntu 18.04.4 LTS -> git-annex version: 6.20180227
+Ubuntu 14.04.6 LTS -> git-annex version: 5.20140412ubuntu1
+freebsd 12.1-RELEASE-p3 -> git-annex version: 7.20190626
+debian 10.4 -> git-annex version: 7.20190129
+debian 8.11 -> git-annex version: 5.20141125+oops-1+deb8u2
+Ubuntu 18.04.4 LTS -> git-annex version: 6.20180227
+
+
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+nope allways have this weird bug who come to haunt me time to time.

diff --git a/doc/forum/git_annex_sync_remove_all_recently_added_files/comment_1_6c059f7b2eb9d213bc0865739832b5d2._comment b/doc/forum/git_annex_sync_remove_all_recently_added_files/comment_1_6c059f7b2eb9d213bc0865739832b5d2._comment
new file mode 100644
index 000000000..5a15fc286
--- /dev/null
+++ b/doc/forum/git_annex_sync_remove_all_recently_added_files/comment_1_6c059f7b2eb9d213bc0865739832b5d2._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="karel-de-macil"
+ avatar="http://cdn.libravatar.org/avatar/46688332185c941113a9c9827cb093e9"
+ subject="comment 1"
+ date="2020-07-23T10:17:06Z"
+ content="""
+just see i may-be should have put this in bug :( please forgive me.
+"""]]

diff --git a/doc/forum/git_annex_sync_remove_all_recently_added_files.mdwn b/doc/forum/git_annex_sync_remove_all_recently_added_files.mdwn
new file mode 100644
index 000000000..769811b21
--- /dev/null
+++ b/doc/forum/git_annex_sync_remove_all_recently_added_files.mdwn
@@ -0,0 +1,40 @@
+since a bit of time i have a strange probleme.
+My git annex environement look like this :
+
+A      E  
+|\    /  
+| C--D  
+|/    \  
+B      F.....G  
+
+the dot point because G is an external drive and can be plug anywhere else.
+let's say my last comit is **db1a9a**
+If i add some file on the repo from G and sync point to point like :
+i'm in G : 
+git annex sync F
+a new commit is created say **61dacd**
+go F
+git annex sync D
+go D
+git annex sync E ...etc
+
+at some point my commit 61dacd becomme the parent of a new commit i dono the source ... let's say **4fcdae**
+
+in git log i found 
+
+commit **4fcdae**... (HEAD -> master, B/synced/ma
+ster, B/master, C/synced/master, C/master, synced/master)
+
+and that commit just undo all the change done by the previous one, indeed : 
+
+git diff **4fcdae** **db1a9a**
+-> nothing
+
+this... force me to play with branch like every time and anoyed me alot. I don't understand what's happening here.
+Does any one have any clue/insight on what i can do to fix this/investigate ?
+
+All repo are at the commit db1a9a before i start...
+
+best regards
+
+

diff --git a/Command/Get.hs b/Command/Get.hs
index 1d46704b2..95d1b9007 100644
--- a/Command/Get.hs
+++ b/Command/Get.hs
@@ -70,7 +70,7 @@ startKeys from (key, ai) = checkFailedTransferDirection ai Download $
 
 start' :: Annex Bool -> Maybe Remote -> Key -> AssociatedFile -> ActionItem -> CommandStart
 start' expensivecheck from key afile ai =
-	stopUnless (not <$> inAnnex key) $ stopUnless expensivecheck $
+	stopUnless expensivecheck $
 		case from of
 			Nothing -> go $ perform key afile
 			Just src ->
diff --git a/Command/Mirror.hs b/Command/Mirror.hs
index 022ea82ca..62f5203ba 100644
--- a/Command/Mirror.hs
+++ b/Command/Mirror.hs
@@ -75,7 +75,10 @@ startKey o afile (key, ai) = case fromToOptions o of
 		haskey <- flip Remote.hasKey key =<< getParsed r
 		case haskey of
 			Left _ -> stop
-			Right True -> Command.Get.start' (return True) Nothing key afile ai
+			Right True -> ifM (inAnnex key)
+				( stop
+				, Command.Get.start' (return True) Nothing key afile ai
+				)
 			Right False -> ifM (inAnnex key)
 				( do
 					numcopies <- getnumcopies
diff --git a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
index d0cd632d8..127e7e717 100644
--- a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
+++ b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
@@ -37,10 +37,6 @@ and precache them.
 > > > Still todo:
 > > > 
 > > > * move, copy, drop, and mirror were left not using the location log caching yet
-> > > * get is left with an unncessary inAnnex check so could be sped up
-> > >   a little bit more. Above improvements to batch mode would allow
-> > >   fixing this.
-> > > 
 
 Another thing that the same cat-file --buffer approach could be used with
 is to cat the annex links. Git.LsFiles.inRepoDetails provides the Sha

diff --git a/CmdLine/Action.hs b/CmdLine/Action.hs
index e42c2a8c7..f1d9eda29 100644
--- a/CmdLine/Action.hs
+++ b/CmdLine/Action.hs
@@ -43,9 +43,6 @@ performCommandAction Command { cmdcheck = c, cmdname = name } seek cont = do
 commandActions :: [CommandStart] -> Annex ()
 commandActions = mapM_ commandAction
 
-commandAction' :: (a -> b -> CommandStart) -> a -> b -> Annex ()
-commandAction' start a b = commandAction $ start a b
-
 {- Runs one of the actions needed to perform a command.
  - Individual actions can fail without stopping the whole command,
  - including by throwing non-async exceptions.
diff --git a/CmdLine/Batch.hs b/CmdLine/Batch.hs
index 4218cec08..14c840919 100644
--- a/CmdLine/Batch.hs
+++ b/CmdLine/Batch.hs
@@ -1,6 +1,6 @@
 {- git-annex batch commands
  -
- - Copyright 2015 Joey Hess <id@joeyh.name>
+ - Copyright 2015-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -11,10 +11,13 @@ import Annex.Common
 import Types.Command
 import CmdLine.Action
 import CmdLine.GitAnnex.Options
+import CmdLine.Seek
 import Options.Applicative
 import Limit
 import Types.FileMatcher
 import Annex.BranchState
+import Annex.WorkTree
+import Annex.Content
 
 data BatchMode = Batch BatchFormat | NoBatch
 
@@ -110,12 +113,22 @@ batchStart fmt a = batchInput fmt (Right <$$> liftIO . relPathCwdToFile) $
 
 -- Like batchStart, but checks the file matching options
 -- and skips non-matching files.
-batchFilesMatching :: BatchFormat -> (FilePath -> CommandStart) -> Annex ()
+batchFilesMatching :: BatchFormat -> (RawFilePath -> CommandStart) -> Annex ()
 batchFilesMatching fmt a = do
 	matcher <- getMatcher
 	batchStart fmt $ \f ->
 		let f' = toRawFilePath f
 		in ifM (matcher $ MatchingFile $ FileInfo f' f')
-			( a f
+			( a f'
 			, return Nothing
 			)
+
+batchAnnexedFilesMatching :: BatchFormat -> AnnexedFileSeeker -> Annex ()
+batchAnnexedFilesMatching fmt seeker = batchFilesMatching fmt $
+	whenAnnexed $ \f k -> case checkContentPresent seeker of
+		Just v -> do
+			present <- inAnnex k
+			if (present == v)
+				then startAction seeker f k
+				else return Nothing
+		Nothing -> startAction seeker f k
diff --git a/CmdLine/Seek.hs b/CmdLine/Seek.hs
index f6a74ec25..b28956960 100644
--- a/CmdLine/Seek.hs
+++ b/CmdLine/Seek.hs
@@ -44,12 +44,13 @@ import qualified Annex.BranchState
 import qualified Database.Keys
 import qualified Utility.RawFilePath as R
 import Utility.Tuple
+import CmdLine.Action
 
 import Control.Concurrent.Async
 import System.Posix.Types
 
 data AnnexedFileSeeker = AnnexedFileSeeker
-	{ seekAction :: RawFilePath -> Key -> CommandSeek
+	{ startAction :: RawFilePath -> Key -> CommandStart
 	, checkContentPresent :: Maybe Bool
 	, usesLocationLog :: Bool
 	}
@@ -305,7 +306,8 @@ seekFilteredKeys seeker listfs = do
 		Just (f, content) -> do
 			case parseLinkTargetOrPointerLazy =<< content of
 				Just k -> checkpresence k $
-					seekAction seeker f k
+					commandAction $
+						startAction seeker f k
 				Nothing -> noop
 			finisher oreader
 		Nothing -> return ()
@@ -313,7 +315,7 @@ seekFilteredKeys seeker listfs = do
 	precachefinisher lreader = liftIO lreader >>= \case
 		Just ((logf, f, k), logcontent) -> do
 			maybe noop (Annex.BranchState.setCache logf) logcontent
-			seekAction seeker f k
+			commandAction $ startAction seeker f k
 			precachefinisher lreader
 		Nothing -> return ()
 	
diff --git a/Command/Add.hs b/Command/Add.hs
index 4e33727b2..858fd4821 100644
--- a/Command/Add.hs
+++ b/Command/Add.hs
@@ -80,7 +80,7 @@ seek o = startConcurrency commandStages $ do
 		Batch fmt
 			| updateOnly o ->
 				giveup "--update --batch is not supported"
-			| otherwise -> batchFilesMatching fmt (gofile . toRawFilePath)
+			| otherwise -> batchFilesMatching fmt gofile
 		NoBatch -> do
 			-- Avoid git ls-files complaining about files that
 			-- are not known to git yet, since this will add
diff --git a/Command/Copy.hs b/Command/Copy.hs
index e58573b42..cd3a781c8 100644
--- a/Command/Copy.hs
+++ b/Command/Copy.hs
@@ -45,22 +45,21 @@ instance DeferredParseClass CopyOptions where
 
 seek :: CopyOptions -> CommandSeek
 seek o = startConcurrency commandStages $ do
-	let go = start o
-	let seeker = AnnexedFileSeeker
-		{ seekAction = commandAction' go
-		, checkContentPresent = Nothing
-		, usesLocationLog = False
-		}
 	case batchOption o of
-		Batch fmt -> batchFilesMatching fmt
-			(whenAnnexed go . toRawFilePath)
 		NoBatch -> withKeyOptions
 			(keyOptions o) (autoMode o)
 			(commandAction . Command.Move.startKey (fromToOptions o) Command.Move.RemoveNever)
 			(withFilesInGitAnnex ww seeker)
 			=<< workTreeItems ww (copyFiles o)
+		Batch fmt -> batchAnnexedFilesMatching fmt seeker
   where
 	ww = WarnUnmatchLsFiles
+	
+	seeker = AnnexedFileSeeker
+		{ startAction = start o
+		, checkContentPresent = Nothing
+		, usesLocationLog = False
+		}
 
 {- A copy is just a move that does not delete the source file.
  - However, auto mode avoids unnecessary copies, and avoids getting or
diff --git a/Command/Drop.hs b/Command/Drop.hs
index e8b80554a..9b0f2aa6f 100644
--- a/Command/Drop.hs
+++ b/Command/Drop.hs
@@ -54,18 +54,16 @@ parseDropFromOption = parseRemoteOption <$> strOption
 seek :: DropOptions -> CommandSeek
 seek o = startConcurrency commandStages $
 	case batchOption o of
-		Batch fmt -> batchFilesMatching fmt
-			(whenAnnexed go . toRawFilePath)
+		Batch fmt -> batchAnnexedFilesMatching fmt seeker
 		NoBatch -> withKeyOptions (keyOptions o) (autoMode o)
 			(commandAction . startKeys o)
 			(withFilesInGitAnnex ww seeker)
 			=<< workTreeItems ww (dropFiles o)
   where
-	go = start o
 	ww = WarnUnmatchLsFiles
 
 	seeker = AnnexedFileSeeker
-		{ seekAction = commandAction' go
+		{ startAction = start o
 		, checkContentPresent = Nothing
 		, usesLocationLog = False
 		}
diff --git a/Command/Find.hs b/Command/Find.hs
index 337ad110f..2a6b171fc 100644
--- a/Command/Find.hs
+++ b/Command/Find.hs
@@ -18,7 +18,6 @@ import Types.Key
 import Git.FilePath
 import qualified Utility.Format
 import Utility.DataUnits
-import Annex.Content
 
 cmd :: Command
 cmd = notBareRepo $ withGlobalOptions [annexedMatchingOptions] $ mkCommand $
@@ -54,27 +53,24 @@ parseFormatOption =
 		)
 
 seek :: FindOptions -> CommandSeek
-seek o = case batchOption o of
-	NoBatch -> do
-		islimited <- limited
-		let seeker = AnnexedFileSeeker
-			{ seekAction = commandAction' go
-			-- only files with content present are shown, unless
-			-- the user has requested others via a limit
-			, checkContentPresent = if islimited
-				then Nothing

(Diff truncated)

diff --git a/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_8_6f5e18dec962aba505ed15bed328f11f._comment b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_8_6f5e18dec962aba505ed15bed328f11f._comment
new file mode 100644
index 000000000..d8b1d7dfc
--- /dev/null
+++ b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_8_6f5e18dec962aba505ed15bed328f11f._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="kyle"
+ avatar="http://cdn.libravatar.org/avatar/7d6e85cde1422ad60607c87fa87c63f3"
+ subject="comment 8"
+ date="2020-07-22T16:12:23Z"
+ content="""
+> fixed made it cancel the threads 2 seconds after the process exited
+> if still running. So there will be a bit of a slow down when using
+> that broken ssh version, but it will work
+
+Just to see this in action, in the Xenial VM, I checked out the parent
+of 4c9ad1de4 (1df9e72a7) to get to a state that hangs with the demo
+script.  I then applied the fix (aa492bc65) on top.  The hang is
+avoided, and I see \"git-annex: AsyncCancelled\".
+
+Thanks.
+"""]]

diff --git a/doc/install/Windows/comment_11_6310a850074cf65b1c0b5b49e2e5b582._comment b/doc/install/Windows/comment_11_6310a850074cf65b1c0b5b49e2e5b582._comment
new file mode 100644
index 000000000..133b3a1e6
--- /dev/null
+++ b/doc/install/Windows/comment_11_6310a850074cf65b1c0b5b49e2e5b582._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="mreiden"
+ avatar="http://cdn.libravatar.org/avatar/00e411c947bed82f673935f36f6be689"
+ subject="Autobuild permissions"
+ date="2020-07-22T16:07:20Z"
+ content="""
+It looks like the windows autobuild is updating based on the timestamps and the build-version contents (8.20200720.2-gd9ae6ab0b), but permission is denied on the exe file (https://downloads.kitenet.net/git-annex/autobuild/windows/git-annex-installer.exe).
+"""]]

diff --git a/doc/bugs/SSH-based_git-annex-init_hang_on_older_systems___40__Xenial__44___Jessie__41__/comment_5_8a5b1e1cb4482fb787bde97a6066de59._comment b/doc/bugs/SSH-based_git-annex-init_hang_on_older_systems___40__Xenial__44___Jessie__41__/comment_5_8a5b1e1cb4482fb787bde97a6066de59._comment
new file mode 100644
index 000000000..6206f0e01
--- /dev/null
+++ b/doc/bugs/SSH-based_git-annex-init_hang_on_older_systems___40__Xenial__44___Jessie__41__/comment_5_8a5b1e1cb4482fb787bde97a6066de59._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="kyle"
+ avatar="http://cdn.libravatar.org/avatar/7d6e85cde1422ad60607c87fa87c63f3"
+ subject="comment 5"
+ date="2020-07-22T16:01:03Z"
+ content="""
+Great, thanks for the analysis and for applying the patch.
+"""]]

diff --git a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_1_6dd92d0737b72b927fc6b82636084c54._comment b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_1_6dd92d0737b72b927fc6b82636084c54._comment
new file mode 100644
index 000000000..6004dd0ab
--- /dev/null
+++ b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__/comment_1_6dd92d0737b72b927fc6b82636084c54._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-07-22T15:07:06Z"
+ content="""
+This should only happen if your program is responding to LISTCONFIGS
+with a list that does not include that parameter.
+
+If it responds with UNSUPPORTED-REQUEST, all configs will be allowed.
+
+I'm not familiar with the python library, but I guess it is doing that, and
+probably has a way to specify what configs to list.
+"""]]

diff --git a/Utility/Metered.hs b/Utility/Metered.hs
index 1c35a9a05..dd49bd06d 100644
--- a/Utility/Metered.hs
+++ b/Utility/Metered.hs
@@ -1,6 +1,6 @@
 {- Metered IO and actions
  -
- - Copyright 2012-2018 Joey Hess <id@joeyh.name>
+ - Copyright 2012-2020 Joey Hess <id@joeyh.name>
  -
  - License: BSD-2-clause
  -}
@@ -46,6 +46,7 @@ import Common
 import Utility.Percentage
 import Utility.DataUnits
 import Utility.HumanTime
+import Utility.ThreadScheduler
 
 import qualified Data.ByteString.Lazy as L
 import qualified Data.ByteString as S
@@ -314,10 +315,22 @@ outputFilter cmd params environ outfilter errfilter =
 	catchMaybeIO $ withCreateProcess p go
   where
 	go _ (Just outh) (Just errh) pid = do
-		void $ concurrently 
-			(tryIO (outfilter outh) >> hClose outh)
-			(tryIO (errfilter errh) >> hClose errh)
-		waitForProcess pid
+		outt <- async $ tryIO (outfilter outh) >> hClose outh
+		errt <- async $ tryIO (errfilter errh) >> hClose errh
+		ret <- waitForProcess pid
+		-- Normally, now that the process has exited, the threads
+		-- will finish processing its output and terminate.
+		-- But, just in case the process did something evil like
+		-- forking to the background while inheriting stderr,
+		-- it's possible that the threads will not finish, which
+		-- would result in a deadlock. So, wait a few seconds
+		-- maximum for them to finish and then cancel them.
+		-- (One program that has behaved this way in the past is
+		-- openssh.)
+		race_
+			(wait outt >> wait errt)
+			(threadDelaySeconds (Seconds 2) >> cancel outt >> cancel errt)
+		return ret
 	go _ _ _ _ = error "internal"
 	
 	p = (proc cmd (toCommand params))
diff --git a/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__.mdwn b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__.mdwn
index 1ef2e78b4..aad4c5ae3 100644
--- a/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__.mdwn
+++ b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__.mdwn
@@ -161,3 +161,7 @@ in git-annex.
 
 [[!meta author=kyle]]
 [[!tag projects/datalad]]
+
+> [[fixed|done]] made it cancel the threads 2 seconds after the process
+> exited if still running. So there will be a bit of a slow down when using
+> that broken ssh version, but it will work. --[[Joey]]

diff --git a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
index bb739380b..8dc580b9b 100644
--- a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
+++ b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
@@ -2,9 +2,9 @@ I am investigating how to create special remotes using the library  Lykos153 / A
 
 I get this:
 
-    git annex initremote foo type=external externaltype=precomp encryption=none folder=/z/old/1111   
+    git annex initremote foo type=external externaltype=precomp encryption=none directory=/z/old/1111   
     initremote foo 
-    git-annex: Unexpected parameters: folder
+    git-annex: Unexpected parameters: directory
     failed
     git-annex: initremote: 1 failed
 

diff --git a/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment
index 0aec4253a..2f79028cb 100644
--- a/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment
+++ b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment
@@ -8,5 +8,48 @@ Reproducer script did not work for me.
 But in the meantime, 4c9ad1de4 has gotten into the built that script uses,
 so whatever about that commit fixes or masks the problem, it seems.
 
-(The other stall's reproducer script did still work.)
+But, the other stall's reproducer script did still work. And I was able to
+confirm that git-annex was blocked while ssh was a
+zombie process.. and killing sshd got git-annex unstuck. So apparently sshd
+is inheriting a handle keeping it open... at least sometimes.
+
+It's really weird my first patch didn't work. The only difference between
+reverting commit 1f2e2d15e and my first patch is the use of
+withCreateProcess. So um.. withCreateProcess is somehow blocking on ssh to
+close stderr? Looking at its code, it calls cleanupProcess, which has
+this comment:
+
+	-- Note, it's important that other threads that might be reading/writing
+	-- these handles also get killed off, since otherwise they might be holding
+	-- the handle lock and prevent us from closing, leading to deadlock.
+
+Ok, that begins to make sense.. The threads are not killed off, so
+are accessing the handle, so closing the handle blocks. And this explains
+why it got to ExitSuccess and then hung.
+
+My second patch, by using withAsync, avoided that. But like I said, that
+patch could kill the threads before they have read and processed all the 
+output. For example, if the process output an error and exited,
+that could cause the error message to not get displayed to the user.
+
+So hmm, this is not reproducible, but we don't know why a recent change
+hid it, and the only patch I know of that fixes it is to revert commit 1f2e2d15e,
+which is part of the work being done for [[todo/more_extensive_retries_to_mask_transient_failures]].
+And while that todo is currently stalled and I may not use this work
+to resolve it at all, it seems a pity to revert that commit.
+
+Also, before 1f2e2d15e, it would have started a thread to read from stderr,
+and that thread could keep running after outputFilter returned.
+And nothing would ever stop that thread, which would block forever due to
+ssh's behavior. So, before that commit, I think there was also a bug,
+a FD leak (and small memory leak).
+
+This is getting kind of long, but the only way forward I see out of all
+this, aside from somehow determining how 4c9ad1de4 masks the problem,
+would be to make the stderr waiter thread only be allowed to run
+for a "little while" after the process exits. Similar to how cb74cefde78
+fixed the other hang.
+
+(Or well, git-annex could detect the ssh versions that have this
+behavior and refuse to use them..)
 """]]

diff --git a/CHANGELOG b/CHANGELOG
index 305fd1ca1..14abd2ea4 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -4,6 +4,9 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
     like git-annex add with -J. Bug was introduced as part of a different FD
     leak fix in version 6.20160318.
   * Support building with dlist-1.0
+  * Fix a hang when using git-annex with an old openssh 7.2p2, which had
+    some weird inheriting of ssh FDs by sshd. Bug was introduced in
+    git-annex version 7.20200202.7.
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 
diff --git a/Git/Config.hs b/Git/Config.hs
index 9e2ac2bd8..45cf4aa8f 100644
--- a/Git/Config.hs
+++ b/Git/Config.hs
@@ -22,6 +22,7 @@ import Git.Types
 import qualified Git.Command
 import qualified Git.Construct
 import Utility.UserInfo
+import Utility.ThreadScheduler
 
 {- Returns a single git config setting, or a fallback value if not set. -}
 get :: ConfigKey -> ConfigValue -> Repo -> ConfigValue
@@ -214,13 +215,20 @@ fromPipe r cmd params st = tryNonAsync $ withCreateProcess p go
 		{ std_out = CreatePipe
 		, std_err = CreatePipe
 		}
-	go _ (Just hout) (Just herr) pid = do
-		(val, err) <- concurrently 
-			(S.hGetContents hout)
-			(S.hGetContents herr)
-		forceSuccessProcess p pid
-		r' <- store val st r
-		return (r', val, err)
+	go _ (Just hout) (Just herr) pid =
+		withAsync (S.hGetContents herr) $ \errreader -> do
+			val <- S.hGetContents hout
+			-- In case the process exits while something else,
+			-- like a background process, keeps the stderr handle
+			-- open, don't block forever waiting for stderr.
+			-- A few seconds after finishing reading stdout
+			-- should get any error message.
+			err <- either id id <$> 
+				wait errreader
+					`race` (threadDelaySeconds (Seconds 2) >> return mempty)
+			forceSuccessProcess p pid
+			r' <- store val st r
+			return (r', val, err)
 	go _ _ _ _ = error "internal"
 
 {- Reads git config from a specified file and returns the repo populated
diff --git a/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment
new file mode 100644
index 000000000..0aec4253a
--- /dev/null
+++ b/doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2020-07-21T19:59:40Z"
+ content="""
+Reproducer script did not work for me.
+
+But in the meantime, 4c9ad1de4 has gotten into the built that script uses,
+so whatever about that commit fixes or masks the problem, it seems.
+
+(The other stall's reproducer script did still work.)
+"""]]
diff --git a/doc/bugs/SSH-based_git-annex-init_hang_on_older_systems___40__Xenial__44___Jessie__41__/comment_4_c480f7fad76759742bbdc4837d8c0c36._comment b/doc/bugs/SSH-based_git-annex-init_hang_on_older_systems___40__Xenial__44___Jessie__41__/comment_4_c480f7fad76759742bbdc4837d8c0c36._comment
new file mode 100644
index 000000000..4eacab9ee
--- /dev/null
+++ b/doc/bugs/SSH-based_git-annex-init_hang_on_older_systems___40__Xenial__44___Jessie__41__/comment_4_c480f7fad76759742bbdc4837d8c0c36._comment
@@ -0,0 +1,31 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2020-07-21T20:02:22Z"
+ content="""
+Reproduced this with the script.
+
+	1798550 pts/0    S+     0:00              |   \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git -C clone annex init --debug
+	1798579 pts/0    Sl+    0:00              |       \_ /root/git-annex.linux/exe/git-annex --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git-annex/git-annex init --debug
+	1798596 pts/0    S+     0:00              |           \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git --git-dir=.git --work-tree=. --literal-pathspecs cat-file --batch
+	1798597 pts/0    S+     0:00              |           \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git --git-dir=.git --work-tree=. --literal-pathspecs cat-file --batch-check=%(objectname) %(objecttype) %(objectsize)
+	1798598 pts/0    S+     0:00              |           \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git --git-dir=.git --work-tree=. --literal-pathspecs hash-object -w --stdin-paths --no-filters
+	1798628 pts/0    Z+     0:00              |           \_ [ssh] <defunct>
+	1798267 ?        Ss     0:00              \_ /usr/sbin/sshd
+	1798629 ?        Ss     0:00              |   \_ sshd: root@notty
+
+I killed 1798629 and git-annex stopped hanging. That seems to confirm my
+theory that sshd has inherited the stderr handle that git-annex is waiting
+for input on. That process had these FDs:
+
+	lrwx------ 1 root root 64 Jul 21 16:04 0 -> /dev/null
+	lrwx------ 1 root root 64 Jul 21 16:04 1 -> /dev/null
+	lrwx------ 1 root root 64 Jul 21 16:04 2 -> /dev/null
+	lrwx------ 1 root root 64 Jul 21 16:04 3 -> socket:[15959002]
+	lr-x------ 1 root root 64 Jul 21 16:04 4 -> pipe:[15956866]
+	l-wx------ 1 root root 64 Jul 21 16:04 5 -> pipe:[15956866]
+
+So probably 4 or 5 was connected to git-annex on the other end.
+
+I've applied my patch, to fix this one.
+"""]]

diff --git a/CHANGELOG b/CHANGELOG
index edbb3195f..305fd1ca1 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,5 +1,8 @@
 git-annex (8.20200720.2) UNRELEASED; urgency=medium
 
+  * Fix a lock file descriptor leak that could occur when running commands
+    like git-annex add with -J. Bug was introduced as part of a different FD
+    leak fix in version 6.20160318.
   * Support building with dlist-1.0
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
diff --git a/Utility/LockPool/LockHandle.hs b/Utility/LockPool/LockHandle.hs
index d6172d613..e8701a5b6 100644
--- a/Utility/LockPool/LockHandle.hs
+++ b/Utility/LockPool/LockHandle.hs
@@ -52,7 +52,7 @@ makeLockHandle pool file pa fa = bracketOnError setup cleanup go
   where
 	setup = debugLocks $ atomically (pa pool file)
 	cleanup ph = debugLocks $ P.releaseLock ph
-	go ph = mkLockHandle pool file ph =<< fa file
+	go ph = mkLockHandle ph =<< fa file
 
 tryMakeLockHandle :: P.LockPool -> LockFile -> (P.LockPool -> LockFile -> STM (Maybe P.LockHandle)) -> (LockFile -> IO (Maybe FileLockOps)) -> IO (Maybe LockHandle)
 tryMakeLockHandle pool file pa fa = bracketOnError setup cleanup go
@@ -67,10 +67,10 @@ tryMakeLockHandle pool file pa fa = bracketOnError setup cleanup go
 			Nothing -> do
 				cleanup (Just ph)
 				return Nothing
-			Just fo -> Just <$> mkLockHandle pool file ph fo
+			Just fo -> Just <$> mkLockHandle ph fo
 
-mkLockHandle :: P.LockPool -> LockFile -> P.LockHandle -> FileLockOps -> IO LockHandle
-mkLockHandle pool file ph fo = do
-	atomically $ P.registerCloseLockFile pool file (fDropLock fo)
+mkLockHandle :: P.LockHandle -> FileLockOps -> IO LockHandle
+mkLockHandle ph fo = do
+	atomically $ P.registerCloseLockFile ph (fDropLock fo)
 	return $ LockHandle ph fo
 	
diff --git a/Utility/LockPool/STM.hs b/Utility/LockPool/STM.hs
index 5cb7b8834..86e1bec06 100644
--- a/Utility/LockPool/STM.hs
+++ b/Utility/LockPool/STM.hs
@@ -1,6 +1,6 @@
 {- STM implementation of lock pools.
  -
- - Copyright 2015 Joey Hess <id@joeyh.name>
+ - Copyright 2015-2020 Joey Hess <id@joeyh.name>
  -
  - License: BSD-2-clause
  -}
@@ -36,11 +36,11 @@ data LockMode = LockExclusive | LockShared
 
 -- This TMVar is full when the handle is open, and is emptied when it's
 -- closed.
-type LockHandle = TMVar (LockPool, LockFile)
+type LockHandle = TMVar (LockPool, LockFile, CloseLockFile)
 
 type LockCount = Integer
 
-data LockStatus = LockStatus LockMode LockCount CloseLockFile
+data LockStatus = LockStatus LockMode LockCount
 
 type CloseLockFile = IO ()
 
@@ -70,25 +70,22 @@ tryTakeLock pool file mode = do
 	m <- takeTMVar pool
 	let success v = do
 		putTMVar pool (M.insert file v m)
-		Just <$> newTMVar (pool, file)
+		Just <$> newTMVar (pool, file, noop)
 	case M.lookup file m of
-		Just (LockStatus mode' n closelockfile)
+		Just (LockStatus mode' n)
 			| mode == LockShared && mode' == LockShared ->
-				success $ LockStatus mode (succ n) closelockfile
+				success $ LockStatus mode (succ n)
 			| n > 0 -> do
 				putTMVar pool m
 				return Nothing
-		_ -> success $ LockStatus mode 1 noop
+		_ -> success $ LockStatus mode 1
 
 -- Call after waitTakeLock or tryTakeLock, to register a CloseLockFile
 -- action to run when releasing the lock.
-registerCloseLockFile :: LockPool -> LockFile -> CloseLockFile -> STM ()
-registerCloseLockFile pool file closelockfile = do
-	m <- takeTMVar pool
-	putTMVar pool (M.update go file m)
-  where
-	go (LockStatus mode n closelockfile') = Just $
-		LockStatus mode n (closelockfile' >> closelockfile)
+registerCloseLockFile :: LockHandle -> CloseLockFile -> STM ()
+registerCloseLockFile h closelockfile = do
+	(p, f, c) <- takeTMVar h
+	putTMVar h (p, f, c >> closelockfile)
 
 -- Checks if a lock is being held. If it's held by the current process,
 -- runs the getdefault action; otherwise runs the checker action.
@@ -103,7 +100,7 @@ getLockStatus pool file getdefault checker = do
 	v <- atomically $ do
 		m <- takeTMVar pool
 		let threadlocked = case M.lookup file m of
-			Just (LockStatus _ n _) | n > 0 -> True
+			Just (LockStatus _ n) | n > 0 -> True
 			_ -> False
 		if threadlocked
 			then do
@@ -123,16 +120,16 @@ getLockStatus pool file getdefault checker = do
 releaseLock :: LockHandle -> IO ()
 releaseLock h = go =<< atomically (tryTakeTMVar h)
   where
-	go (Just (pool, file)) = do
-		(m, closelockfile) <- atomically $ do
+	go (Just (pool, file, closelockfile)) = do
+		m <- atomically $ do
 			m <- takeTMVar pool
 			return $ case M.lookup file m of
-				Just (LockStatus mode n closelockfile)
-					| n == 1 -> (M.delete file m, closelockfile)
+				Just (LockStatus mode n)
+					| n == 1 -> (M.delete file m)
 					| otherwise ->
-						(M.insert file (LockStatus mode (pred n) closelockfile) m, noop)
-				Nothing -> (m, noop)
-		closelockfile
+						(M.insert file (LockStatus mode (pred n)) m)
+				Nothing -> m
+		() <- closelockfile
 		atomically $ putTMVar pool m
 	-- The LockHandle was already closed.
 	go Nothing = return ()
diff --git a/doc/bugs/Adding_files_fails_with_--jobs_more_than_2.mdwn b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2.mdwn
index 0fcfdc452..8dbecc710 100644
--- a/doc/bugs/Adding_files_fails_with_--jobs_more_than_2.mdwn
+++ b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2.mdwn
@@ -62,3 +62,5 @@ failed
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 I've been having quite a bit of fun revisiting git-annex and datalad after quite a while, and I'm really excited to see how far things have come. I'm pretty close to adopting these tools in my data science group after a pretty exhaustive search of related technologies like quilt, dvc, and attempts to role my own solution. Using Github + the S3 special remote w/versioning enabled is just about the best solution to dataset tracking I've come across.
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_2_45702c5a87e891a0dde5459b5156c57e._comment b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_2_45702c5a87e891a0dde5459b5156c57e._comment
new file mode 100644
index 000000000..9b89f2eaf
--- /dev/null
+++ b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_2_45702c5a87e891a0dde5459b5156c57e._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""analysis"""
+ date="2020-07-21T18:35:08Z"
+ content="""
+Utility.LockPool.STM.releaseLock seems to be where the problem is.
+
+It waits to close a lock FD if some other thread is using the lock. 
+But, this means that, if enough threads are run that the lock is
+always in use by one thread, it will never close it. Meanwhile, each
+lockShared call opens the lock file anew, accumilating another FD.
+
+3334130368829ad2041006560e578f1f876f68e4 is at fault, indeed.
+
+That commit mentions that it would be better to have two calls to
+lockShared only open the file once, but that it would be difficult to do
+that atomically. Perhaps there is a way to do that... It didn't seem easy
+to do this time either.
+
+Alternatively, registerCloseLockFile currently takes an action that closes
+one lock FD, and just combines that to its existing close action with `>>`.
+So it builds up one big action that closes all the FDs. Instead, make each
+lock handle contain its close action, and have releaseLock only release the
+one it was called with. Implemented this, and it solved it.
+"""]]

diff --git a/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_1_77cfd006625edc78e8c3bf6cbb4e1f09._comment b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_1_77cfd006625edc78e8c3bf6cbb4e1f09._comment
index a7d6a93ce..2a4c83916 100644
--- a/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_1_77cfd006625edc78e8c3bf6cbb4e1f09._comment
+++ b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_1_77cfd006625edc78e8c3bf6cbb4e1f09._comment
@@ -13,7 +13,8 @@ those a lot), and the remaining 37 were event and timer stuff used by the
 RTS.
 
 The number of -J does not change that leak, same number with -J2. -J2 will
-open slightly less other files though.
+open slightly less other files though. Without -J, the FD leak does not
+happen.
 
 Also, git-annex 8.20200330 does not have this problem, it seems to be a
 recent bug.

diff --git a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
index ee75f58b1..bb739380b 100644
--- a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
+++ b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
@@ -2,10 +2,10 @@ I am investigating how to create special remotes using the library  Lykos153 / A
 
 I get this:
 
-   git annex initremote foo type=external externaltype=precomp encryption=none folder=/z/old/1111   
-   initremote foo 
-   git-annex: Unexpected parameters: folder
-   failed
-   git-annex: initremote: 1 failed
+    git annex initremote foo type=external externaltype=precomp encryption=none folder=/z/old/1111   
+    initremote foo 
+    git-annex: Unexpected parameters: folder
+    failed
+    git-annex: initremote: 1 failed
 
 The order of the arguments make no difference. Do I need to somehow register parameters now, that was not required before? Or something else?

diff --git a/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
new file mode 100644
index 000000000..ee75f58b1
--- /dev/null
+++ b/doc/forum/How_to_register_arguments_for_an_external_special_remote__63__.mdwn
@@ -0,0 +1,11 @@
+I am investigating how to create special remotes using the library  Lykos153 / AnnexRemote. I have copied [this file](https://github.com/Lykos153/AnnexRemote/blob/master/examples/git-annex-remote-directory) locally, renamed it, set it up as needed, but can not initialise a remote with it. 
+
+I get this:
+
+   git annex initremote foo type=external externaltype=precomp encryption=none folder=/z/old/1111   
+   initremote foo 
+   git-annex: Unexpected parameters: folder
+   failed
+   git-annex: initremote: 1 failed
+
+The order of the arguments make no difference. Do I need to somehow register parameters now, that was not required before? Or something else?

diff --git a/doc/todo/external_backends/comment_13_6e8452f06da80458bb2cccde05cc9813._comment b/doc/todo/external_backends/comment_13_6e8452f06da80458bb2cccde05cc9813._comment
new file mode 100644
index 000000000..1192c6114
--- /dev/null
+++ b/doc/todo/external_backends/comment_13_6e8452f06da80458bb2cccde05cc9813._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="external backend protocol"
+ date="2020-07-21T17:43:27Z"
+ content="""
+1. What is the advantage of a separate `VERIFYCONTENT` request, vs calling `GENKEY` and comparing the result?
+2. Can the protocol specify that the file passed to `GENKEY` may be a named pipe?  Or, add a `CANPIPE` request where the external backend program tells git-annex that it can take pipes; if the program can't, git-annex can always drain the pipe to a tempfile before passing it to the program.
+3. \"While stderr is connected to the console and so visible to the user, the program should avoid using it\" -- then maybe add `DEBUG` and `INFO` requests as in the [[design/external_special_remote_protocol]]?
+"""]]

diff --git a/doc/forum/build_questions__58___Flag_HttpClientRestricted.mdwn b/doc/forum/build_questions__58___Flag_HttpClientRestricted.mdwn
new file mode 100644
index 000000000..6b0fe8301
--- /dev/null
+++ b/doc/forum/build_questions__58___Flag_HttpClientRestricted.mdwn
@@ -0,0 +1,7 @@
+Some questions about building git-annex with `stack`:
+
+* What are the pros/cons of setting the new flags `HttpClientRestricted` and `GitLfs` to `True`/`False`?
+
+* `extra-deps` in [stack.yaml](https://git.joeyh.name/index.cgi/git-annex.git/tree/stack.yaml) specifies specific package versions, some of which are [older than the latest ones](https://hackage.haskell.org/package/http-client) on Hackage.  I find myself needing to [patch](https://github.com/notestaff/git-annex-feedstock/blob/is-update-to-8.20200720.1-new/recipe/0001-enable-magicmime-dbus.patch) some of these `extra-deps` to later ones; these still seem to meet the [git-annex.cabal constraints](https://git.joeyh.name/index.cgi/git-annex.git/tree/git-annex.cabal#n425), but I wanted to check if there's a specific reason the later versions aren't used in the standard `stack.yaml`.
+
+Thanks!

diff --git a/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_1_77cfd006625edc78e8c3bf6cbb4e1f09._comment b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_1_77cfd006625edc78e8c3bf6cbb4e1f09._comment
new file mode 100644
index 000000000..a7d6a93ce
--- /dev/null
+++ b/doc/bugs/Adding_files_fails_with_--jobs_more_than_2/comment_1_77cfd006625edc78e8c3bf6cbb4e1f09._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-07-21T14:09:48Z"
+ content="""
+It looks to me like git-annex add is leaking a FD to
+.git/annex/othertmp.lck
+
+I ran git-annex add -J16 in a repo with 1000 files, and around half way
+through it had 832 open FDs, and 737 of them were that lock file. 55 more
+were pipes (to and from git cat-file etc; recent work reduced the number of
+those a lot), and the remaining 37 were event and timer stuff used by the
+RTS.
+
+The number of -J does not change that leak, same number with -J2. -J2 will
+open slightly less other files though.
+
+Also, git-annex 8.20200330 does not have this problem, it seems to be a
+recent bug.
+
+[[!commit 3334130368829ad2041006560e578f1f876f68e4]] claimed to fix a leak
+in this same code path.
+"""]]

diff --git a/doc/bugs/smudge_filter_drastically_slows_down_git_diff.mdwn b/doc/bugs/smudge_filter_drastically_slows_down_git_diff.mdwn
index 7f05e76bd..3899d18fd 100644
--- a/doc/bugs/smudge_filter_drastically_slows_down_git_diff.mdwn
+++ b/doc/bugs/smudge_filter_drastically_slows_down_git_diff.mdwn
@@ -114,3 +114,5 @@ Running on openSUSE Tumbleweed.
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 Yes, mostly a happy user ;-)
+
+> [[dup|done]]
diff --git a/doc/bugs/smudge_filter_drastically_slows_down_git_diff/comment_2_98cdd89081d2f31a45b693a04b20f345._comment b/doc/bugs/smudge_filter_drastically_slows_down_git_diff/comment_2_98cdd89081d2f31a45b693a04b20f345._comment
new file mode 100644
index 000000000..9f35e654f
--- /dev/null
+++ b/doc/bugs/smudge_filter_drastically_slows_down_git_diff/comment_2_98cdd89081d2f31a45b693a04b20f345._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-07-21T14:00:05Z"
+ content="""
+This is exactly what that todo is about. I'm going to close this as a dup.
+
+I suppose git diff runs it twice because it's smudging both sides of the diff.
+
+Note that git status does not re-run the smudge filter on the same file
+repeatedly (at least most of the time), so the overhead there is only one
+run per modification of a file. Unfortunately git diff does not benefit from
+such caching. Still, it only runs once per modified file in the repo,
+not on unmodified files.
+"""]]

diff --git a/doc/forum/using_rsync.net_as_backup_and_glacier_as_archive.mdwn b/doc/forum/using_rsync.net_as_backup_and_glacier_as_archive.mdwn
new file mode 100644
index 000000000..7cd68f123
--- /dev/null
+++ b/doc/forum/using_rsync.net_as_backup_and_glacier_as_archive.mdwn
@@ -0,0 +1,13 @@
+Hi,
+
+My current setup is:
+
+  - "work" in my computer, client group
+  - "backup"  rsync.net also in client group
+  - glacier in full archive group
+
+My intention was to have rsync.net to be my live backup, from which I could pull the whole repo in case of disaster. I expected when I moved a file to "archive" in my computer this would be reflected in rsync.net and once it been archived on glacier both files (my computer and rsync) would be dropped.
+
+That did not happen, through the assistant. I watched the file uploaded to glacier but it never dropped my copy. Actually when I stopped the assistant ``git annex drop --auto`` it drops the file. But if I start the assistant again it starts downloading from rsync.net.
+
+Is there a way to accomplish what I'm trying? Maybe if rsync.net was in _trasfer_ group, but then again without being able to run ``git annex sync`` on rsync.net side (which actually supports ``git`` commands) I don't know if would work...

diff --git a/doc/news/version_8.20200330.mdwn b/doc/news/version_8.20200330.mdwn
deleted file mode 100644
index e5ae7b0e5..000000000
--- a/doc/news/version_8.20200330.mdwn
+++ /dev/null
@@ -1,16 +0,0 @@
-git-annex 8.20200330 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * fsck: Fix reversion in 8.20200226 that made it incorrectly warn
-     that hashed keys with an extension should be upgraded.
-   * add --force-small: Fix a bug that, when adding a symbolic link,
-     checked in the content of the file the symlink pointed to.
-     Thanks, Kyle Meyer for the patch.
-   * add --force-small: Fix failure when passed a modified submodule.
-     Thanks, Kyle Meyer for the patch.
-   * When syncing changes back from an adjusted branch to the basis branch,
-     include changes to submodules.
-     Thanks, Kyle Meyer for the patch.
-   * webdav: Made exporttree remotes faster by caching connection to the
-     server.
-   * Fix a minor bug that caused options provided with -c to be passed
-     multiple times to git."""]]
\ No newline at end of file
diff --git a/doc/news/version_8.20200720.1.mdwn b/doc/news/version_8.20200720.1.mdwn
new file mode 100644
index 000000000..11a01b6aa
--- /dev/null
+++ b/doc/news/version_8.20200720.1.mdwn
@@ -0,0 +1,3 @@
+git-annex 8.20200720.1 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Fix a bug in find --batch in the previous version."""]]
\ No newline at end of file

diff --git a/CHANGELOG b/CHANGELOG
index 6b32db0c4..ff72a9895 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,8 +1,8 @@
-git-annex (8.20200720.1) UNRELEASED; urgency=medium
+git-annex (8.20200720.1) upstream; urgency=medium
 
   * Fix a bug in find --batch in the previous version.
 
- -- Joey Hess <id@joeyh.name>  Mon, 20 Jul 2020 19:37:40 -0400
+ -- Joey Hess <id@joeyh.name>  Mon, 20 Jul 2020 19:39:11 -0400
 
 git-annex (8.20200720) upstream; urgency=medium
 
diff --git a/doc/design/external_backend_protocol.mdwn b/doc/design/external_backend_protocol.mdwn
index 305f8fbbb..69f0a6bca 100644
--- a/doc/design/external_backend_protocol.mdwn
+++ b/doc/design/external_backend_protocol.mdwn
@@ -131,7 +131,8 @@ These messages can be sent at any time by either git-annex or the program.
 
 ## considerations for generating keys
 
-See [[internals/key_format]] for how to format a key.
+See [[internals/key_format]] for how to format a key and details about the
+parts of a key.
 
 The backend name should match the name of the program, eg if the program
 is git-annex-backend-XFOO, it should generate a key starting with "XFOO-".
@@ -159,6 +160,15 @@ in the key name, where users are unlikely to notice it. While git has
 several things that make sha1 collision attacks difficult, we don't want
 this chink in the armor.
 
+It's almost always a good idea to include the size field when generating a
+key. The size does not need to be checked when verifying content, as
+git-annex handles that for you. The only time it would make sense to omit
+the size field is if the content of a key is not stable and might have
+different sizes (like some URL keys do).
+
+There's generally no reason to include the mtime field, and it should
+never be verified when verifying content.
+
 ## program names must be unique
 
 It's important that two different programs don't use the same name, because
diff --git a/doc/design/external_backend_protocol/git-annex-backend-XFOO b/doc/design/external_backend_protocol/git-annex-backend-XFOO
index 0282fab4a..6886ab064 100755
--- a/doc/design/external_backend_protocol/git-annex-backend-XFOO
+++ b/doc/design/external_backend_protocol/git-annex-backend-XFOO
@@ -33,8 +33,9 @@ while read line; do
 		GENKEY)
 			contentfile="$2"
 			hash=$(hashfile "$contentfile")
+			sz=$(wc -c "$contentfile" | cut -d ' ' -f 1)
 			if [ -n "$hash" ]; then
-				echo "GENKEY-SUCCESS" "XFOO--$hash"
+				echo "GENKEY-SUCCESS" "XFOO-s$sz--$hash"
 			else
 				echo "GENKEY-FAILURE" "md5sum failed"
 			fi

diff --git a/CHANGELOG b/CHANGELOG
index 9f7555799..6b32db0c4 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,9 @@
+git-annex (8.20200720.1) UNRELEASED; urgency=medium
+
+  * Fix a bug in find --batch in the previous version.
+
+ -- Joey Hess <id@joeyh.name>  Mon, 20 Jul 2020 19:37:40 -0400
+
 git-annex (8.20200720) upstream; urgency=medium
 
   * import: Added --no-content option, which avoids downloading files
diff --git a/Command/Find.hs b/Command/Find.hs
index 9c1f33628..337ad110f 100644
--- a/Command/Find.hs
+++ b/Command/Find.hs
@@ -18,6 +18,7 @@ import Types.Key
 import Git.FilePath
 import qualified Utility.Format
 import Utility.DataUnits
+import Annex.Content
 
 cmd :: Command
 cmd = notBareRepo $ withGlobalOptions [annexedMatchingOptions] $ mkCommand $
@@ -70,9 +71,10 @@ seek o = case batchOption o of
 			(withFilesInGitAnnex ww seeker)
 			=<< workTreeItems ww (findThese o)
 	Batch fmt -> batchFilesMatching fmt
-		(whenAnnexed go . toRawFilePath)
+		(whenAnnexed gobatch . toRawFilePath)
   where
 	go = start o
+	gobatch f k = stopUnless (limited <||> inAnnex k) (go f k)
 	ww = WarnUnmatchLsFiles
 
 start :: FindOptions -> RawFilePath -> Key -> CommandStart
diff --git a/doc/bugs/find_--batch_reports_files_that_have_no_content.mdwn b/doc/bugs/find_--batch_reports_files_that_have_no_content.mdwn
index 8c481401a..8a56afc0d 100644
--- a/doc/bugs/find_--batch_reports_files_that_have_no_content.mdwn
+++ b/doc/bugs/find_--batch_reports_files_that_have_no_content.mdwn
@@ -61,3 +61,8 @@ local repository version: 8
 
 [[!meta author=kyle]]
 [[!tag projects/datalad]]
+
+> [[fixed|done]], and audited for any other similar reversions.
+> 
+> Unfortunately did not see this bug before the release,
+> so I should make a point release for it. --[[Joey]]
diff --git a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
index 0bae6e00b..264617877 100644
--- a/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
+++ b/doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
@@ -37,8 +37,12 @@ and precache them.
 > > > Still todo:
 > > > 
 > > > * move, copy, drop, and mirror were left not using the location log caching yet
+> > > * find has a bit of ugliness around batch mode, and this shows it
+> > >   would be worth making the batch mode take the same AnnexedFileSeeker,
+> > >   to reunify the batch and non-batch code
 > > > * get is left with an unncessary inAnnex check so could be sped up
-> > >   a little bit more
+> > >   a little bit more. Above improvements to batch mode would allow
+> > >   fixing this.
 > > > 
 
 Another thing that the same cat-file --buffer approach could be used with
diff --git a/git-annex.cabal b/git-annex.cabal
index 621862780..e679a347b 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -1,5 +1,5 @@
 Name: git-annex
-Version: 8.20200720
+Version: 8.20200720.1
 Cabal-Version: >= 1.10
 License: AGPL-3
 Maintainer: Joey Hess <id@joeyh.name>

diff --git a/doc/news/version_8.20200226/comment_1_0dc2c663924fe478ce60df02e03fc07a._comment b/doc/news/version_8.20200226/comment_1_0dc2c663924fe478ce60df02e03fc07a._comment
deleted file mode 100644
index 6d8eaab45..000000000
--- a/doc/news/version_8.20200226/comment_1_0dc2c663924fe478ce60df02e03fc07a._comment
+++ /dev/null
@@ -1,23 +0,0 @@
-[[!comment format=mdwn
- username="https://openid.jorsn.eu/"
- avatar="http://cdn.libravatar.org/avatar/250999e054ac1e0c82a45d8b5dcfad69ec611f16216b7c1df36a3df1d715aa37"
- subject="automatic repository upgrades"
- date="2020-03-30T23:24:51Z"
- content="""
-Hi Joey,
-
-I really like git-annex but there is one thing which annoys me and friends of mine: Automatic repo upgrades.
-
-I just installed the newer version as an unstable package from my distro[^1] for a test and noticed afterwards that git-annex v7[^2] doesn't work anymore.
-
-This is something I would like to be warned about in advance; it's nothing I would expect. Imagine a repo shared between people. One of them gets a new version of git-annex installed by the system admin, thus converting the repo accidentally, without even noticing. The others will be happy.
-
-So please document `annex.autoupgraderepository` in the man page and set it to `false` by default.
-
-Stay happy and healthy,
-
-Johannes
-
-[^1]: so didn't read the news first
-[^2]: the stable distro package
-"""]]

diff --git a/doc/news/version_8.20200309.mdwn b/doc/news/version_8.20200309.mdwn
deleted file mode 100644
index d9a87d4e5..000000000
--- a/doc/news/version_8.20200309.mdwn
+++ /dev/null
@@ -1,30 +0,0 @@
-git-annex 8.20200309 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Fix bug that caused unlocked annexed dotfiles to be added to git by the
-     smudge filter when annex.dotfiles was not set.
-   * Upgrade other repos than the current one by running git-annex upgrade
-     inside them, which avoids problems with upgrade code making assumptions
-     that the cwd will be inside the repo being upgraded. In particular,
-     this fixes a problem where upgrading a v7 repo to v8 caused an ugly
-     git error message.
-   * Fix upgrade failure when a file has been deleted from the working tree.
-   * Fix regression 1 month ago that prevented external special remotes from
-     using GETCONFIG to query values like "name".
-   * Improve behavior when a directory git-annex is writing to gets
-     unmounted. Previously it could in some cases re-create the mount point
-     and directory tree, and even write object contents to the wrong disk.
-   * Don't ignore --debug when it is followed by -c.
-   * whereis: If a remote fails to report on urls where a key
-     is located, display a warning, rather than giving up and not displaying
-     any information.
-   * When external special remotes fail but neglect to provide an error
-     message, say what request failed, which is better than displaying an
-     empty error message to the user.
-   * git-annex config: Only allow configs be set that are ones git-annex
-     actually supports reading from repo-global config, to avoid confusion.
-   * Avoid converting .git file in a worktree or submodule to a symlink
-     when the repository is not a git-annex repository.
-   * Linux standalone: Use md5sum to shorten paths in .cache/git-annex/locales
-   * Fix build with ghc 8.8 (MonadFail)
-     Thanks, Peter Simons
-   * stack.yaml: Updated to lts-14.27."""]]
\ No newline at end of file
diff --git a/doc/news/version_8.20200720.mdwn b/doc/news/version_8.20200720.mdwn
new file mode 100644
index 000000000..bb53c5a45
--- /dev/null
+++ b/doc/news/version_8.20200720.mdwn
@@ -0,0 +1,40 @@
+git-annex 8.20200720 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * import: Added --no-content option, which avoids downloading files
+     from a special remote. Currently only supported by the directory
+     special remote.
+   * Honor annex.largefiles when importing a tree from a special remote.
+     (Except for when --no-content is used.)
+   * Fix a deadlock that could occur after git-annex got an unlocked
+     file, causing the command to hang indefinitely. Known to happen on
+     vfat filesystems, possibly others.
+   * Build with the http-client-restricted and git-lfs libraries when
+     available, otherwise use the vendored copy as before.
+   * testremote: Fix over-allocation of resources and bad caching,
+     including starting up a large number of external special remote processes.
+     (Regression introduced in version 8.20200501)
+   * test: Fix some test cases that assumed git's default branch name.
+   * importfeed: Added some additional --template variables:
+     itempubyear, itempubmonth, itempubday, itempubhour,
+     itempubminute, itempubsecond.
+   * Made several special remotes support locking content on them,
+     which allows dropping from other special remotes in some situations
+     where it was not possible before. Supported special remotes:
+     S3 (with versioning=yes), git-lfs, tahoe
+   * Fix reversion that broke passing annex.* and remote.*.annex-*
+     git configs with -c. (Since version 8.20200330.)
+   * Bring back git-annex branch read cache. This speeds up some operations,
+     eg git-annex sync --content --all gets 20% faster.
+   * Fix a recently introduced bug that could cause a "fork: resource exhausted"
+     after getting several thousand files.
+   * Sped up the --all option by 2x to 16x by using git cat-file --buffer.
+     Thanks to Lukey for finding this optimisation.
+   * Sped up seeking for annexed files to operate on by a factor of nearly 2x.
+   * Sped up sync --content by 2x and other commands like fsck --fast and
+     whereis by around 50%, by using git cat-file --buffer.
+   * importfeed: Made checking known urls step around 15% faster.
+   * fsck: Detect if WORM keys contain a carriage return, and recommend
+     upgrading the key. (git-annex could have maybe created such keys back
+     in 2013).
+   * When on an adjust --hide-missing branch, fix handling of files that
+     have been deleted but the deletion is not yet staged."""]]
\ No newline at end of file

diff --git a/doc/design/external_backend_protocol.mdwn b/doc/design/external_backend_protocol.mdwn
index fbfbbabb6..305f8fbbb 100644
--- a/doc/design/external_backend_protocol.mdwn
+++ b/doc/design/external_backend_protocol.mdwn
@@ -131,7 +131,7 @@ These messages can be sent at any time by either git-annex or the program.
 
 ## considerations for generating keys
 
-See [[doc/internals/key_format]] for how to format a key.
+See [[internals/key_format]] for how to format a key.
 
 The backend name should match the name of the program, eg if the program
 is git-annex-backend-XFOO, it should generate a key starting with "XFOO-".

diff --git a/doc/design/external_backend_protocol.mdwn b/doc/design/external_backend_protocol.mdwn
index 2fbdb176c..fbfbbabb6 100644
--- a/doc/design/external_backend_protocol.mdwn
+++ b/doc/design/external_backend_protocol.mdwn
@@ -96,7 +96,7 @@ reply with one of the listed replies.
   longer. A program can change its answer to this question as the state of the
   art advances, and should aim to stay ahead of the state of the art by a
   reasonable amount of time.
-  * ISCRYPTOGRAPHICALLYSECURE-YES`
+  * `ISCRYPTOGRAPHICALLYSECURE-YES`
   * ISCRYPTOGRAPHICALLYSECURE-NO`
 
 ## main messages and replies