Recent changes to this wiki:

git-annex-view fails because of "another git process running"
diff --git a/doc/bugs/view.mdwn b/doc/bugs/view.mdwn
index 7833f52b1..4dcbf82d5 100644
--- a/doc/bugs/view.mdwn
+++ b/doc/bugs/view.mdwn
@@ -20,7 +20,7 @@ git commit -m 'initial commit'
 git annex metadata -s test=test1 test1
 git annex metadata -s test=test2 test2
 git annex view test=test1
-"""
+"""]]
 
 Note: The problem does not appear with git-annex version 7.20190819+git2-g908476a9b-1~ndall+1 but it appears with 8.20201007-g903b2f1.
 
@@ -43,9 +43,10 @@ local repository version: 8
 ```
 
 
-
 ### Please provide any additional information below.
 
+Here is the complete error message:
+
 [[!format sh """
 view (searching...) fatal: Unable to create '/tmp/mytmp/.git/annex/viewindex.lock': File exists.
 

git-annex-view fails
diff --git a/doc/bugs/view.mdwn b/doc/bugs/view.mdwn
new file mode 100644
index 000000000..7833f52b1
--- /dev/null
+++ b/doc/bugs/view.mdwn
@@ -0,0 +1,66 @@
+### Please describe the problem.
+
+git-annex-view fails with an error message that `viewindex.lock` exists and another git process is running in the repository.
+The problem appears even with newly initialized repositories.
+
+### What steps will reproduce the problem?
+
+The steps I followed to reproduce the problem:
+
+[[!format sh """
+cd /tmp
+mkdir testgit
+cd testgit
+git init
+git annex init
+echo 123 > test1
+echo 456 > test2
+git annex add test1 test2
+git commit -m 'initial commit'
+git annex metadata -s test=test1 test1
+git annex metadata -s test=test2 test2
+git annex view test=test1
+"""
+
+Note: The problem does not appear with git-annex version 7.20190819+git2-g908476a9b-1~ndall+1 but it appears with 8.20201007-g903b2f1.
+
+
+### What version of git-annex are you using? On what operating system?
+
+I could reproduce the problem at two different machines, both with Ubuntu 18.04 LTS.
+`git-annex` was installed via conda using the command `conda install -c conda-forge git-annex`
+
+```
+git-annex version: 8.20201007-g903b2f1
+build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
+dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso hook external
+operating system: linux x86_64
+supported repository versions: 8
+upgrade supported from repository versions: 0 1 2 3 4 5 6 7
+local repository version: 8
+```
+
+
+
+### Please provide any additional information below.
+
+[[!format sh """
+view (searching...) fatal: Unable to create '/tmp/mytmp/.git/annex/viewindex.lock': File exists.
+
+Another git process seems to be running in this repository, e.g.
+an editor opened by 'git commit'. Please make sure all processes
+are terminated then try again. If it still fails, a git process
+may have crashed in this repository earlier:
+remove the file manually to continue.
+
+git-annex: failed to read sha from git write-tree
+CallStack (from HasCallStack):
+  error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
+failed
+git-annex: view: 1 failed
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+git-annex is an awesome tool! Really appreciate your work and everything else works great!

diff --git a/doc/forum/git_annex_view__58___viewindex.lock__44___file_exists___40__it_doesn__39__t__41__.mdwn b/doc/forum/git_annex_view__58___viewindex.lock__44___file_exists___40__it_doesn__39__t__41__.mdwn
new file mode 100644
index 000000000..1230f9f40
--- /dev/null
+++ b/doc/forum/git_annex_view__58___viewindex.lock__44___file_exists___40__it_doesn__39__t__41__.mdwn
@@ -0,0 +1,7 @@
+Hi!
+
+I tagged some files with `tag=telegram`, but upon trying to try out views with `git annex view tag=telegram` I get:
+
+https://del.dog/nifumairra.txt
+
+Any ideas? :(  The file viewindex.lock doesn't exist..

diff --git a/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle.mdwn b/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle.mdwn
index 748e36234..8d08810c1 100644
--- a/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle.mdwn
+++ b/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle.mdwn
@@ -18,6 +18,8 @@ mv: cannot move '/home/lyderic/.cache/git-annex/locales.tmp/ec977e22ef909b450a3a
 
 $ mkdir ~/.cache/git-annex/locales
 
+[[done]]
+
 ### What version of git-annex are you using? On what operating system?
 
 git-annex version: 8.20200909-g2d5036e44

Added a comment: This bug has alreay been raised
diff --git a/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle/comment_1_9c2410c55154bc8d9f733377cde58eb9._comment b/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle/comment_1_9c2410c55154bc8d9f733377cde58eb9._comment
new file mode 100644
index 000000000..97cfb25e7
--- /dev/null
+++ b/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle/comment_1_9c2410c55154bc8d9f733377cde58eb9._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="lyderic"
+ avatar="http://cdn.libravatar.org/avatar/d03e900b5e94b54891ee9596460f76c3"
+ subject="This bug has alreay been raised"
+ date="2020-10-29T10:13:50Z"
+ content="""
+I found out that this bug has already been raised and addressed...
+https://git-annex.branchable.com/bugs/git_annex_init_fails/
+"""]]

diff --git a/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle.mdwn b/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle.mdwn
new file mode 100644
index 000000000..748e36234
--- /dev/null
+++ b/doc/bugs/__34__no_such_file_or_directory__34___when_using_bundle.mdwn
@@ -0,0 +1,44 @@
+### Please describe the problem.
+
+The bundle provided at https://git-annex.branchable.com/install/Linux_standalone/ doesn't work
+
+### What steps will reproduce the problem?
+
+1. I downloaded this file: https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-amd64.tar.gz
+
+2. I untared it in /usr/local and renamed the resulting directory to '/usr/local/git-annex'
+
+3. I added '/usr/local/git-annex' to my PATH
+
+4. I ran 'git-annex' and got:
+
+mv: cannot move '/home/lyderic/.cache/git-annex/locales.tmp/ec977e22ef909b450a3a84bd55783865.1075139' to '/home/lyderic/.cache/git-annex/locales/ec977e22ef909b450a3a84bd55783865': No such file or directory
+
+5. I fixed the problem by doing this:
+
+$ mkdir ~/.cache/git-annex/locales
+
+### What version of git-annex are you using? On what operating system?
+
+git-annex version: 8.20200909-g2d5036e44
+OS: Ubuntu 20.04.1
+
+### Please provide any additional information below.
+
+This is a transcript of the commands I ran to show and fix the bug:
+
+[[!format sh """
+~$ sudo tar -xzf Downloads/git-annex-standalone-amd64.tar.gz -C /usr/local
+~$ sudo mv /usr/local/git-annex.linux /usr/local/git-annex
+~$ sudo chown -R 0.0 /usr/local/git-annex
+~$ export PATH=$PATH:/usr/local/git-annex
+~$ git-annex version
+mv: cannot move '/home/lyderic/.cache/git-annex/locales.tmp/ec977e22ef909b450a3a84bd55783865.1075139' to '/home/lyderic/.cache/git-annex/locales/ec977e22ef909b450a3a84bd55783865': No such file or directory
+~$ mkdir ~/.cache/git-annex/locales
+~$ git-annex version | head -1
+git-annex version: 8.20200909-g2d5036e44
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+I have never used it. I am learning it / evaluating it right now. I am very impressed so far!

Added a comment
diff --git a/doc/bugs/git-annex_looses_itself_on_Windows__63__/comment_1_e809445ff0109d8aa98093b024b3b35c._comment b/doc/bugs/git-annex_looses_itself_on_Windows__63__/comment_1_e809445ff0109d8aa98093b024b3b35c._comment
new file mode 100644
index 000000000..c2e0e652e
--- /dev/null
+++ b/doc/bugs/git-annex_looses_itself_on_Windows__63__/comment_1_e809445ff0109d8aa98093b024b3b35c._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 1"
+ date="2020-10-27T19:44:36Z"
+ content="""
+probably nevermind and feel free to close -- something is finicky with the `bash` which I have
+
+```
+(datalad-2) C:\Users\DataLad\AppData\Local\Temp\datalad_temp_tree_hufbz7i1>bash -c \"git-annex version\"
+bash: git-annex: command not found
+```
+
+need to figure that out
+
+```
+(datalad-2) C:\Users\DataLad\AppData\Local\Temp\datalad_temp_tree_hufbz7i1>echo %PATH%
+C:\Users\DataLad\miniconda3\envs\datalad-2;C:\Users\DataLad\miniconda3\envs\datalad-2\Library\mingw-w64\bin;C:\Users\DataLad\miniconda3\envs\datalad-2\Library\usr\bin;C:\Users\DataLad\miniconda3\envs\datalad-2\Library\bin;C:\Users\DataLad\miniconda3\envs\datalad-2\Scripts;C:\Users\DataLad\miniconda3\envs\datalad-2\bin;C:\Users\DataLad\miniconda3\condabin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\Git\cmd;C:\Users\DataLad\AppData\Local\Microsoft\WindowsApps;.
+
+(datalad-2) C:\Users\DataLad\AppData\Local\Temp\datalad_temp_tree_hufbz7i1>bash -c \"echo $PATH\"
+/c/Users/DataLad/miniconda3/envs/datalad-2:/mingw-w64/bin:/usr/bin:/bin:/c/Users/DataLad/miniconda3/envs/datalad-2/Scripts:/c/Users/DataLad/miniconda3/envs/datalad-2/bin:/c/Users/DataLad/miniconda3/condabin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files/Git/cmd:/c/Users/DataLad/AppData/Local/Microsoft/WindowsApps:.
+```
+so it seems to discard `C:\Users\DataLad\miniconda3\envs\datalad-2\Library\bin`, may be I should not \"install\" there ;) (but it also looses `conda` itself in that `bash` \"shell\")
+"""]]

Added a comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_9_552e36f88bc39bc68b65ccba06002082._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_9_552e36f88bc39bc68b65ccba06002082._comment
new file mode 100644
index 000000000..70fa26315
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_9_552e36f88bc39bc68b65ccba06002082._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 9"
+ date="2020-10-27T19:30:24Z"
+ content="""
+Working in the same environment as in [the other issue](https://git-annex.branchable.com/bugs/git-annex_looses_itself_on_Windows__63__/?updated).  It does come with more recent `cp` (coreutils) 8.25 BUT that one blows with (sorry -- this is debug output of datalad, not direct output from git/annex invocation):
+
+<details>
+<summary></summary> 
+
+```shell
+datalad.cmd: DEBUG: Async run ['git', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', 'fi le.dat']
+datalad.cmd: DEBUG: Launching process ['git', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', 'fi le.dat']
+datalad.cmd: DEBUG: Process 2932 started
+datalad.cmd: DEBUG: Waiting for process 2932 to complete
+datalad.cmd: DEBUG: Process 2932 exited with return code 0
+datalad.cmd: DEBUG: Process file list chunk 0 (length 1)
+datalad.annex: DEBUG: Running add resulted in stderr output: cp: preserving permissions for ‘.git\\annex\\objects\\6f0\\fbd\\SHA256E-s7--ed7002b439e9ac845f22357d822bac1444730fbdb6016d3ec9432297b9ec9f73.dat\\SHA256E-s7--ed7002b439e9ac845f22357d822bac1444730fbdb6016d3ec9432297b9ec9f73.dat’: Not supported
+git-annex: add: 1 failed
+
+```
+</details>
+
+so it echoes your \"-a/-p/--preserve-timestamps are more important and also probed at build time.\" and is yet another gotcha of using different build and run environments. I only wonder how is that our github windows workflow environment supports `cp` with preserving permissions while conda's doesn't
+"""]]

report on possibly PATH related issue on windows
diff --git a/doc/bugs/git-annex_looses_itself_on_Windows__63__.mdwn b/doc/bugs/git-annex_looses_itself_on_Windows__63__.mdwn
new file mode 100644
index 000000000..44cb9c972
--- /dev/null
+++ b/doc/bugs/git-annex_looses_itself_on_Windows__63__.mdwn
@@ -0,0 +1,41 @@
+Well, I know that my setup is "experimental" - I am trying to establish consistent `conda` environment, ATM everything (`git` and all the `posix` tools) come from conda (conda-forge channel at large) and then git-annex is installed on top - pretty much simply extracted into conda environment from our git-annex-installer.exe built with magic.
+
+Running (in a windows shell, so not in bash)
+
+```
+(datalad-2) C:\Users\DataLad\AppData\Local\Temp\datalad_temp_tree_hufbz7i1>git add --verbose "fi le2.dat"
+git-annex smudge --clean -- 'fi le2.dat': git-annex: command not found
+error: external filter 'git-annex smudge --clean -- %f' failed 127
+error: external filter 'git-annex smudge --clean -- %f' failed
+add 'fi le2.dat'
+
+```
+
+suggests that `git` does see `git-annex`, but then `git-annex smudge` doesn't see `git-annex` in the PATH?
+
+```
+(datalad-2) C:\Users\DataLad\AppData\Local\Temp\datalad_temp_tree_hufbz7i1>echo %PATH%
+C:\Users\DataLad\miniconda3\envs\datalad-2;C:\Users\DataLad\miniconda3\envs\datalad-2\Library\mingw-w64\bin;C:\Users\DataLad\miniconda3\envs\datalad-2\Library\usr\bin;C:\Users\DataLad\miniconda3\envs\datalad-2\Library\bin;C:\Users\DataLad\miniconda3\envs\datalad-2\Scripts;C:\Users\DataLad\miniconda3\envs\datalad-2\bin;C:\Users\DataLad\miniconda3\condabin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\Git\cmd;C:\Users\DataLad\AppData\Local\Microsoft\WindowsApps;.
+```
+
+and there is `C:\Users\DataLad\miniconda3\envs\datalad-2\Library\bin\git-annex.exe` .
+
+<details>
+<summary>git annex version</summary> 
+
+```shell
+(datalad-2) C:\Users\DataLad\AppData\Local\Temp\datalad_temp_tree_hufbz7i1>git annex version
+git-annex version: 8.20201008-g3be4731
+build flags: Assistant Webapp Pairing TorrentParser MagicMime Feeds Testsuite S3 WebDAV
+dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso hook external
+operating system: mingw32 x86_64
+supported repository versions: 8
+upgrade supported from repository versions: 2 3 4 5 6 7
+local repository version: 8
+```
+</details>
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

comment
diff --git a/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__/comment_2_23cfa4b9abe96e4477f06c5271b706b2._comment b/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__/comment_2_23cfa4b9abe96e4477f06c5271b706b2._comment
new file mode 100644
index 000000000..6465d7efb
--- /dev/null
+++ b/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__/comment_2_23cfa4b9abe96e4477f06c5271b706b2._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-10-27T18:23:30Z"
+ content="""
+A git-annex repo consisting of annex symlinks (locked files) can be cloned
+and used on machines running git-annex versions between 2 and 8. (Any
+version from the past 9 years.) It's the same information stored in git.
+
+If you choose to commit unlocked files to the repo, those files will only
+work when using git-annex 7 or 8. But that's optional and you can also just
+re-lock the files to regain compatability all the way back to version 2.
+
+So, no, you do not need to worry about git-annex sync breaking something.
+
+And, git-annex 8 on one machine can talk to git-annex 7 (or any version
+really) on another machine, no problem.
+
+Now, if you have a repo that you are physically moving around between
+machines, on USB for example, then the machines all need have to have
+a git-annex version installed that's compatible with the annex.version
+of the repo.
+"""]]

Added a comment: Documentation of demand
diff --git a/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_6_3d2f62f0946528c9986135f8db86a237._comment b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_6_3d2f62f0946528c9986135f8db86a237._comment
new file mode 100644
index 000000000..228348cda
--- /dev/null
+++ b/doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_6_3d2f62f0946528c9986135f8db86a237._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="michael.hanke@c60e12358aa3fc6060531bdead1f530ac4d582ec"
+ nickname="michael.hanke"
+ avatar="http://cdn.libravatar.org/avatar/f881df265a423e4f24eff27c623148fd"
+ subject="Documentation of demand"
+ date="2020-10-27T14:59:47Z"
+ content="""
+We are routinely working with datasets in the 100TB range, with individual file sizes of up to 16TB. Some of our systems have ~500MB/s disk read throughput only. The separation of download and checksuming leads to 6+ hours of additional disk reads and delay the availability of a file. Having this feature would be a stellar performance improvement for our use case. Thanks in advance.
+"""]]

Added a comment
diff --git a/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__/comment_1_30fb300ec5dbdb166f2bb979c6f2ee78._comment b/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__/comment_1_30fb300ec5dbdb166f2bb979c6f2ee78._comment
new file mode 100644
index 000000000..fcbb8115e
--- /dev/null
+++ b/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__/comment_1_30fb300ec5dbdb166f2bb979c6f2ee78._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Lukey"
+ avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
+ subject="comment 1"
+ date="2020-10-27T09:06:38Z"
+ content="""
+I just use the (daily) standalone linux build everywhere, not the package from my distro. I extract it to `/opt`, set `GIT_ANNEX_PACKAGE_INSTALL=no` at the top of `git-annex.linux/runshell` and then link git-annex to `/usr/bin/git-annex` with `ln -s /opt/git-annex.linux/git-annex /usr/bin/git-annex`.
+"""]]

diff --git a/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__.mdwn b/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__.mdwn
new file mode 100644
index 000000000..6989dbf55
--- /dev/null
+++ b/doc/forum/How_are_people_handling_the_v8_repo_upgrade__63__.mdwn
@@ -0,0 +1,13 @@
+I noticed today when running `git annex sync` that it was unhappy with one of my remotes:
+
+```
+remote: git-annex: Repository /home/anovak/annex/.git is at unsupported version 7. Automatic upgrade exception! actually-broken.log: getSymbolicLinkStatus: does not exist (No such file or directory)
+```
+
+Digging in, it looks like that machine is running Ubuntu 20.04, which ships Git Annex 8, which *only* supports repo version 8. The rest of my machines are a mix of Ubuntu 18.04 and other Debian derivatives, which all ship older versions of Git Annex, which *can't* support repo version 8.
+
+Is there a way for me to get out of manually installing Git Annex on every machine to upgrade everything to v8 at the same time?
+
+And, before I've upgraded, is there a way to screw up my repo by e.g. running `git annex sync` from the newer Git Annex and pushing commits in the new format over to machines that still only speak the old format?
+
+How are other people handling this transition where there's no repo format that's speakable by all the machines? Just upgrading everything at once and replacing the distro's git annex with a manually installed version?

view: Avoid using ':' from metadata when generating a view
Because it's a special character on Windows ("c:").
Use same technique already used for '/' and '\'.
I didn't record how I generated their encoded forms before, so am sure
there was a better way, but the way I did it now is to look at
ghci> encodeFilePath "∕"
"\226\136\149"
And then the difference from that to "\56546\56456\56469"
is adding 56320 to each, to get up to the escaped code plane.
See comment for why I think handling ':' is ok, but that other illegal
windows filenames won't. Note that, this should be enough to make the
test suite always work. Other windows illegal filenames will fail at
checkout time when it tries to put the illegal filename on the
filesystem.
diff --git a/Annex/View.hs b/Annex/View.hs
index fa9a7b632..725cce3cb 100644
--- a/Annex/View.hs
+++ b/Annex/View.hs
@@ -235,27 +235,34 @@ pseudoSlash = "\56546\56456\56469"
 pseudoBackslash :: String
 pseudoBackslash = "\56546\56469\56498"
 
+-- And this is '﹕' similarly.
+pseudoColon :: String
+pseudoColon = "\56559\56505\56469"
+
 toViewPath :: MetaValue -> FilePath
-toViewPath = escapeslash [] . decodeBS . fromMetaValue
+toViewPath = escapepseudo [] . decodeBS . fromMetaValue
   where
-	escapeslash s ('/':cs) = escapeslash (pseudoSlash:s) cs
-	escapeslash s ('\\':cs) = escapeslash (pseudoBackslash:s) cs
-	escapeslash s ('%':cs) = escapeslash ("%%":s) cs
-	escapeslash s (c1:c2:c3:cs)
-		| [c1,c2,c3] == pseudoSlash = escapeslash ("%":pseudoSlash:s) cs
-		| [c1,c2,c3] == pseudoBackslash = escapeslash ("%":pseudoBackslash:s) cs
-		| otherwise = escapeslash ([c1]:s) (c2:c3:cs)
-	escapeslash s cs = concat (reverse (cs:s))
+	escapepseudo s ('/':cs) = escapepseudo (pseudoSlash:s) cs
+	escapepseudo s ('\\':cs) = escapepseudo (pseudoBackslash:s) cs
+	escapepseudo s (':':cs) = escapepseudo (pseudoColon:s) cs
+	escapepseudo s ('%':cs) = escapepseudo ("%%":s) cs
+	escapepseudo s (c1:c2:c3:cs)
+		| [c1,c2,c3] == pseudoSlash = escapepseudo ("%":pseudoSlash:s) cs
+		| [c1,c2,c3] == pseudoBackslash = escapepseudo ("%":pseudoBackslash:s) cs
+		| [c1,c2,c3] == pseudoColon = escapepseudo ("%":pseudoColon:s) cs
+		| otherwise = escapepseudo ([c1]:s) (c2:c3:cs)
+	escapepseudo s cs = concat (reverse (cs:s))
 
 fromViewPath :: FilePath -> MetaValue
-fromViewPath = toMetaValue . encodeBS . deescapeslash []
+fromViewPath = toMetaValue . encodeBS . deescapepseudo []
   where
-	deescapeslash s ('%':escapedc:cs) = deescapeslash ([escapedc]:s) cs
-	deescapeslash s (c1:c2:c3:cs)
-		| [c1,c2,c3] == pseudoSlash = deescapeslash ("/":s) cs
-		| [c1,c2,c3] == pseudoBackslash = deescapeslash ("\\":s) cs
-		| otherwise = deescapeslash ([c1]:s) (c2:c3:cs)
-	deescapeslash s cs = concat (reverse (cs:s))
+	deescapepseudo s ('%':escapedc:cs) = deescapepseudo ([escapedc]:s) cs
+	deescapepseudo s (c1:c2:c3:cs)
+		| [c1,c2,c3] == pseudoSlash = deescapepseudo ("/":s) cs
+		| [c1,c2,c3] == pseudoBackslash = deescapepseudo ("\\":s) cs
+		| [c1,c2,c3] == pseudoColon = deescapepseudo (":":s) cs
+		| otherwise = deescapepseudo ([c1]:s) (c2:c3:cs)
+	deescapepseudo s cs = concat (reverse (cs:s))
 
 prop_viewPath_roundtrips :: MetaValue -> Bool
 prop_viewPath_roundtrips v = fromViewPath (toViewPath v) == v
diff --git a/CHANGELOG b/CHANGELOG
index 126b9aa6f..3652ec89e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -21,6 +21,8 @@ git-annex (8.20201008) UNRELEASED; urgency=medium
     will work.
     Thanks to John Thorvald Wodder II and Yaroslav Halchenko for their work
     on this.
+  * view: Avoid using ':' from metadata when generating a view, because
+    it's a special character on Windows ("c:")
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Oct 2020 10:48:17 -0400
 
diff --git a/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips.mdwn b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips.mdwn
index 9e073c552..4e804b39f 100644
--- a/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips.mdwn
+++ b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips.mdwn
@@ -30,3 +30,5 @@ Cheers,
 
 [[!meta author=yoh]]
 [[!tag projects/datalad]]
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_3_06dd9606225cf881f7681fc27fb4833d._comment b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_3_06dd9606225cf881f7681fc27fb4833d._comment
new file mode 100644
index 000000000..029540d2b
--- /dev/null
+++ b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_3_06dd9606225cf881f7681fc27fb4833d._comment
@@ -0,0 +1,42 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-10-26T18:41:27Z"
+ content="""
+Hmm, this uses viewedFiles, which generates filenames
+based on the MetaValue. Note use of pathProduct, which uses
+System.FilePath.combine.
+
+So, generating random ascii (including escape sequences)
+bytestrings, and passing them through decodeBS to generate FilePaths, 
+and then operating on those filepaths. What could possibly go wrong.
+
+And aha! I made pathProduct use System.FilePath.Windows.combine
+and was able to reproduce the test suite failure on Linux.
+
+And aha again: 
+
+	MetaValue (CurrentlySet True) "c:"
+
+Which of course breaks it on windows because it wanted to generate
+something like "bar/c:/baz/a" but instead it gets "c:/bar/baz/a"
+
+git-annex does replace '/' and '\' when generating these filenames.
+Not as a security measure (when the view branch is checked out, git's
+security checks apply same as any branch so it piggybacks on those),
+but to let the user build a view and successfully check it out
+when their metadata happens to include such stuff.
+
+However, windows does have enough special filenames and gotchas
+that it simply does not seem to make sense for git-annex to try to work
+around them all in the view code. If a MetaValue happens to end with a
+period, or is "nul", and so the generated filename is illegal on Windows,
+it'll blow up at checkout time, and I am ok with that.
+
+So I think it would make sense to also escape ':', but that's about as far
+as this should go. *Especially* because the filenames it generates need to
+roundtrip back to metadata cleanly, which is what this test case is
+testing. While I can finesse individual characters, it would be quite hard
+to make a filename w/o a trailing dot roundtrip back to one with it, for
+example.
+"""]]

Added a comment
diff --git a/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_2_1945040bcdf209b9a0faecd4a77e0f35._comment b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_2_1945040bcdf209b9a0faecd4a77e0f35._comment
new file mode 100644
index 000000000..6c1ae21b1
--- /dev/null
+++ b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_2_1945040bcdf209b9a0faecd4a77e0f35._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2020-10-26T17:57:12Z"
+ content="""
+you say \"windows port\", I say \"windows as a whole\", e.g. today revelation (or just a come back if I ran into it before but forgot) to me [was inability to have a file/directory named `con`...](https://github.com/datalad/datalad/issues/5097) - no bloody sense on how such design decision has happened and how it dragged all the way into the flagman of the 2020 product.
+"""]]

make windows installer builder include libmagic files when present, but skip otherwise
diff --git a/Build/NullSoftInstaller.hs b/Build/NullSoftInstaller.hs
index d3d1a2243..b21480948 100644
--- a/Build/NullSoftInstaller.hs
+++ b/Build/NullSoftInstaller.hs
@@ -9,11 +9,15 @@
  - for that.
  - 
  - To build the installer, git-annex should already be built to
- - ./git-annex.exe and the necessary utility programs
- - (specifically rsync)
- - already installed in PATH from msys32.
+ - ./git-annex.exe, then run this program.
  -
- - Copyright 2013-2015 Joey Hess <id@joeyh.name>
+ - A build of libmagic will also be included in the installer, if its files
+ - are found in the current directory: 
+ -   ./magic.mgc ./libmagic-1.dll ./libgnurx-0.dll
+ - To build git-annex to usse libmagic, it has to be built with the
+ - magicmime build flag turned on.
+ -
+ - Copyright 2013-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -42,7 +46,8 @@ main = do
 	withTmpDir "nsis-build" $ \tmpdir -> do
 		let gitannex = tmpdir </> gitannexprogram
 		mustSucceed "ln" [File "git-annex.exe", File gitannex]
-		mapM_ (\f -> mustSucceed "ln" [File f, File (tmpdir </> f)]) (magicDLLs ++ magicShare)
+		magicDLLs' <- installwhenpresent magicDLLs
+		magicShare' <- installwhenpresent magicShare
 		let license = tmpdir </> licensefile
 		mustSucceed "sh" [Param "-c", Param $ "zcat standalone/licences.gz > '" ++ license ++ "'"]
 		webappscript <- vbsLauncher tmpdir "git-annex-webapp" "git annex webapp"
@@ -52,7 +57,7 @@ main = do
 		let gitannexcmd = tmpdir </> "git-annex.cmd"
 		writeFile gitannexcmd "git annex %*"
 		writeFile nsifile $ makeInstaller
-			gitannex gitannexcmd license htmlhelp (winPrograms ++ magicDLLs) magicShare
+			gitannex gitannexcmd license htmlhelp (winPrograms ++ magicDLLs') magicShare'
 			[ webappscript, autostartscript ]
 		mustSucceed "makensis" [File nsifile]
 	removeFile nsifile -- left behind if makensis fails
@@ -63,6 +68,15 @@ main = do
 		case r of
 			True -> return ()
 			False -> error $ cmd ++ " failed"
+	installwhenpresent fs = do
+		fs' <- forM fs $ \f -> do
+			present <- doesFileExist f
+			if present
+				then do
+					mustSucceed "ln" [File f, File (tmpdir </> f)]
+					return (Just f)
+				else return Nothing
+		return (catMaybes fs')
 
 {- Generates a .vbs launcher which runs a command without any visible DOS
  - box. It expects to be passed the directory where git-annex is installed. -}
diff --git a/doc/install/Windows.mdwn b/doc/install/Windows.mdwn
index 6c4029d41..6bfce5249 100644
--- a/doc/install/Windows.mdwn
+++ b/doc/install/Windows.mdwn
@@ -26,5 +26,4 @@ tree, run "stack build" to download and
 build all dependencies and git-annex. "stack install" will install git-annex.
 
 (To build the git-annex installer, you also need to install the NullSoft
-installer system. The script `standalone/windows/build.sh` is
-used to make the builds linked to above.)
+installer system, see Build/NullSoftInstaller.hs for details.)

Windows build changed to one done by the datalad-extensions project using Github actions
This is a cleaner build than on Jenkins because the whole environment setup
is handled by the CI config, at least up to the point of "get a random bag
of Windows bytes".
Also, the Jenkins autobuilder has been intermittently failing for a long
time, not due to any problem with git-annex but just a failure to clean up
directories.
Also, this build runs the test suite, and it is (mostly) passing. Test
suite always failed in the jenkins environment.
Also, this build includes libmagic.
Here is the build workflow used by github actions:
https://github.com/datalad/datalad-extensions/blob/master/.github/workflows/build-git-annex-windows.yaml
The libmagic build has its own workflow:
https://github.com/datalad/file-windows/blob/master/.github/workflows/build.yml
(Also cleaned up some windows build cruft I don't use anymore.)
There is no build-version file to link to. I've opened a todo requesting
one: https://github.com/datalad/datalad-extensions/issues/55
diff --git a/CHANGELOG b/CHANGELOG
index 0b470d9c1..983df2f2c 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -15,6 +15,8 @@ git-annex (8.20201008) UNRELEASED; urgency=medium
   * Fixed some problems that prevented this command from working:
     git submodule foreach git annex init
   * testremote: Display exceptions when tests fail, to aid debugging.
+  * Windows build changed to one done by the datalad-extensions project
+    using Github actions.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Oct 2020 10:48:17 -0400
 
diff --git a/Jenkinsfile b/Jenkinsfile
deleted file mode 100644
index c77d5383b..000000000
--- a/Jenkinsfile
+++ /dev/null
@@ -1,55 +0,0 @@
-currentBuild.result = 'SUCCESS'
-def caught_exception = null
-
-final def EMAIL_RECIPIENTS = 'joey+git-annex@joeyh.name'
-
-properties([
-        buildDiscarder(logRotator(artifactNumToKeepStr: '1')),
-        pipelineTriggers([[$class: 'hudson.triggers.SCMTrigger', scmpoll_spec: ''],])  // pollScm('') in 2.22+
-])
-
-try {
-
-    node('windows') {
-
-        dir('git-annex') {
-
-            stage('Checkout') {
-                checkout scm
-            }
-
-            stage('Build') {
-                bat 'c:/cygwin/bin/sh standalone/windows/build.sh'
-            }
-
-            stage('Archive') {
-                archiveArtifacts 'git-annex-installer.exe,dist/build-version'
-            }
-
-            stage('Upload') {
-                withCredentials([usernamePassword(credentialsId: 'rsync-downloads-kitenet-net', passwordVariable: 'RSYNC_PASSWORD', usernameVariable: 'DUMMY')]) {
-                    bat 'c:/cygwin/bin/rsync git-annex-installer.exe winautobuild@downloads.kitenet.net::winautobuild'
-                    bat 'c:/cygwin/bin/rsync dist/build-version winautobuild@downloads.kitenet.net::winautobuild'
-                }
-            }
-
-        }
-
-    }
-
-} catch (exception) {
-
-    caught_exception = exception
-    currentBuild.result = 'FAILURE'
-
-} finally {
-
-    node('master') {
-        step([$class: 'Mailer', notifyEveryUnstableBuild: false, recipients: EMAIL_RECIPIENTS, sendToIndividuals: false])
-    }
-
-    if (caught_exception) {
-        throw caught_exception
-    }
-
-}
diff --git a/build.bat b/build.bat
deleted file mode 100644
index c2ccfac5f..000000000
--- a/build.bat
+++ /dev/null
@@ -1 +0,0 @@
-sh standalone/windows/build-simple.sh
diff --git a/doc/builds.mdwn b/doc/builds.mdwn
index b56caa419..a95d340e0 100644
--- a/doc/builds.mdwn
+++ b/doc/builds.mdwn
@@ -22,7 +22,7 @@
 <iframe width=1024 height=20em scrolling=no frameborder=0 marginheight=0 marginwidth=0 src="https://downloads.kitenet.net/git-annex/autobuild/x86_64-apple-yosemite/build-version">
 </iframe>
 <h2>Windows</h2>
-<iframe width=1024 height=20em scrolling=no frameborder=0 marginheight=0 marginwidth=0 src="https://downloads.kitenet.net/git-annex/autobuild/windows/build-version">
+<iframe width=1024 height=20em scrolling=no frameborder=0 marginheight=0 marginwidth=0 src="fixme">
 </iframe>
 """]]
 
@@ -47,6 +47,10 @@
 <iframe width=1024 scrolling=no frameborder=0 marginheight=0 marginwidth=0 src="https://downloads.kitenet.net/git-annex/autobuild/x86_64-apple-yosemite/">
 </iframe>
 <h2>Debian standalone packages</h2>
-<a href="https://github.com/datalad/datalad-extensions/actions?query=workflow%3A%22Build+git-annex+snapshot%22"><img src="https://github.com/datalad/datalad-extensions/workflows/Build%20git-annex%20snapshot/badge.svg"></a>
+<a href="https://github.com/datalad/datalad-extensions/actions?query=workflow%3A%22Build+git-annex+snapshot%22">
+<img src="https://github.com/datalad/datalad-extensions/workflows/Build%20git-annex%20snapshot/badge.svg">
+</a>
 <h2>Windows</h2>
-<a href="http://vps.zaytsev.net:8080/">here (limited access)</a>
+<a href="https://github.com/datalad/datalad-extensions/actions?query=workflow%3A%22Build+git-annex+snapshot+on+Windows%22">
+<img src="https://github.com/datalad/datalad-extensions/workflows/Build%20git-annex%20snapshot%20on%20Windows/badge.svg">
+</a>
diff --git a/doc/install/Windows.mdwn b/doc/install/Windows.mdwn
index 08d8e63f5..6c4029d41 100644
--- a/doc/install/Windows.mdwn
+++ b/doc/install/Windows.mdwn
@@ -9,17 +9,11 @@ git-annex. The assistant and webapp are also usable. There are some known
 problems and parts that don't work. See [[todo/windows_support]] for
 current status.
 
-To verify that the build of git-annex works in your Windows system, you are
-encouraged to run the test suite before using git-annex on real data. After
-installation, run `git annex test`. There will be a lot of output; the
-important thing is that it should end with "All tests passed".
-
 ## autobuilds
 
-An hourly autobuild is also available, thanks to Yury V. Zaytsev and
-Dartmouth College.
+An autobuild is also available, thanks to the Datalad project.
 
-* [download](https://downloads.kitenet.net/git-annex/autobuild/windows/)
+* [download](http://datasets.datalad.org/datalad/packages/windows/)
 
 ## building it yourself
 
diff --git a/standalone/windows/build-simple.sh b/standalone/windows/build-simple.sh
deleted file mode 100755
index bee40e335..000000000
--- a/standalone/windows/build-simple.sh
+++ /dev/null
@@ -1,16 +0,0 @@
-#!/bin/sh
-# Script to build git-annex on windows. Run by build.bat
-
-set -e
-set -x
-
-PATH="/c/Program Files/Git/cmd:/c/Program Files/NSIS:$PATH"
-
-stack setup
-stack build --dependencies-only
-
-# Build git-annex
-stack build
-
-# Build the installer
-stack runghc --package nsis Build/NullSoftInstaller.hs
diff --git a/standalone/windows/build.sh b/standalone/windows/build.sh
deleted file mode 100755
index 0a772e4a8..000000000
--- a/standalone/windows/build.sh
+++ /dev/null
@@ -1,71 +0,0 @@
-#!/bin/sh
-# 
-# This script is run by the jenkins autobuilder, in a mingw environment,
-# to build git-annex for Windows.
-
-set -x
-set -e
-
-PATH="/cygdrive/c/git/cmd:/cygdrive/c/Program Files (x86)/NSIS/Bin:/cygdrive/c/Program Files (x86)/NSIS:/usr/local/bin:/usr/bin:/cygdrive/c/Users/jenkins/AppData/Roaming/local/bin:$PATH"
-
-# Run a command with the cygwin environment available.
-# However, programs not from cygwin are preferred.
-withcyg () {
-	PATH="$PATH:/c/cygwin/bin" "$@"
-}
-
-# Prefer programs from cygwin.
-withcygpreferred () {
-	PATH="/c/cygwin/bin:$PATH" "$@"
-}
-
-# This tells git-annex where to upgrade itself from.
-UPGRADE_LOCATION=http://downloads.kitenet.net/git-annex/windows/current/git-annex-installer.exe
-export UPGRADE_LOCATION
-
-# This can be used to force git-annex to build supporting a particular
-# version of git, instead of the version installed at build time.
-#FORCE_GIT_VERSION=1.9.5
-#export FORCE_GIT_VERSION
-
-# Don't allow build artifact from a past successful build to be extracted
-# if we fail.
-rm -f git-annex-installer.exe
-rm -f git-annex.exe
-rm -rf dist
-
-# Upgrade stack
-stack --version
-#stack upgrade --force-download
-#stack --version
-

(Diff truncated)
comment
diff --git a/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_1_3a2be0a1cd74c7dba3b07e1300ce2a30._comment b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_1_3a2be0a1cd74c7dba3b07e1300ce2a30._comment
new file mode 100644
index 000000000..a2dfd1a58
--- /dev/null
+++ b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips/comment_1_3a2be0a1cd74c7dba3b07e1300ce2a30._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-10-26T16:03:40Z"
+ content="""
+I can reproduce it in a windows VM running
+`git-annex test --quickcheck-replay=742853`
+
+These quickcheck tests test random input so not flaky exactly.
+
+Does not happen with that seed on linux, so it probably involves something
+encoding specific. An area where the windows port is known to have
+extensive problems.
+
+([[!commit 1b8026b2cbc8df0274082c5f08a8b4f8ca47c5c9]] was similar, 
+although that was MetaField and this appears to be MetaValue.)
+"""]]

close
diff --git a/doc/bugs/storeKey_when_already_present_failures_on_Windows.mdwn b/doc/bugs/storeKey_when_already_present_failures_on_Windows.mdwn
index 96325f40f..233b2cf6d 100644
--- a/doc/bugs/storeKey_when_already_present_failures_on_Windows.mdwn
+++ b/doc/bugs/storeKey_when_already_present_failures_on_Windows.mdwn
@@ -15,3 +15,5 @@ When I ran `annex test` after installing that installer in a local Win 10 VM, I
 
 [[!meta author=yoh]]
 [[!tag projects/datalad]]
+
+> apparently [[fixed|done]]; followup if it reappears --[[Joey]]

Added a comment: thanks
diff --git a/doc/devblog/day_633__ten_years/comment_2_48b09fc7e66cd9d9fb7c0f78f21c7ac5._comment b/doc/devblog/day_633__ten_years/comment_2_48b09fc7e66cd9d9fb7c0f78f21c7ac5._comment
new file mode 100644
index 000000000..554065ee4
--- /dev/null
+++ b/doc/devblog/day_633__ten_years/comment_2_48b09fc7e66cd9d9fb7c0f78f21c7ac5._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="thanks"
+ date="2020-10-26T15:37:57Z"
+ content="""
+\"Thanks\" doesn't quite cut it, but still worth saying :)  Thanks for the huge amount of work and thought you've put into this over the years.  git-annex has grown into an impressive \"swiss army knife\" adaptable to all sorts of uses.  It's certainly one of the most best FLOSS projects I've seen.  With the proliferation of various cloud services for storing data, it's becoming even more useful for keeping track of things without losing sanity.  Thanks @joeyh!
+"""]]

document version for --force-large/--force-small
also fix the wrong name in the changelog
diff --git a/CHANGELOG b/CHANGELOG
index c0b8a0756..0b470d9c1 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -474,7 +474,7 @@ git-annex (7.20200204) upstream; urgency=medium
 
 git-annex (7.20200202.7) upstream; urgency=medium
 
-  * add: --force-annex/--force-git options make it easier to override
+  * add: --force-large/--force-small options make it easier to override
     annex.largefiles configuration (and potentially safer as it avoids
     bugs like the smudge bug fixed in the last release).
   * reinject --known: Fix bug that prevented it from working in a bare repo.
diff --git a/doc/tips/largefiles.mdwn b/doc/tips/largefiles.mdwn
index 7764325af..5a5906bfb 100644
--- a/doc/tips/largefiles.mdwn
+++ b/doc/tips/largefiles.mdwn
@@ -104,6 +104,8 @@ This first removes the file from git's index cache, and then adds it back
 using git-annex. You can modify the file before the `git-annex add` step,
 perhaps replacing it with new larger content that necessitates git-annex.
 
+The --force-large option needs git-annex version 7.20200202.7 or newer.
+
 ## converting annexed to git
 
 When you have a file that is currently stored in the annex, and you want to
@@ -117,3 +119,5 @@ convert that to be stored in git, here's how to accomplish that:
 You can modify the file after unlocking it and before adding it to
 git. And this is probably a good idea if it was really a big file,
 so that you can replace its content with something smaller.
+
+The --force-small option needs git-annex version 7.20200202.7 or newer.

improve wording
diff --git a/doc/git-annex-add.mdwn b/doc/git-annex-add.mdwn
index fadfd59a1..efcf9e567 100644
--- a/doc/git-annex-add.mdwn
+++ b/doc/git-annex-add.mdwn
@@ -50,8 +50,7 @@ annexed content, and other symlinks.
 * `--force-small`
 
   Treat all files as small files, ignoring annex.largefiles and annex.dotfiles
-  configuration, and add to git, also ignoring annex.addsmallfiles
-  configuration.
+  and annex.addsmallfiles configuration, and add to git.
 
 * `--backend`
 

initial report on a failing test
diff --git a/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips.mdwn b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips.mdwn
new file mode 100644
index 000000000..9e073c552
--- /dev/null
+++ b/doc/bugs/1_test_failure_under_conda_on_Windows_10__58___prop__95__view__95__roundtrips.mdwn
@@ -0,0 +1,32 @@
+### Please describe the problem.
+
+
+
+### What steps will reproduce the problem?
+
+
+I am  plowing through on making git-annex available within conda-forge "natively" for Windows.  For now I just took the recently built installer, the one now available from [datasets.datalad.org](http://datasets.datalad.org/datalad/packages/windows/) and built on datalad-extensions github setup.  I just extracted git-annex component from the installer and placed them within conda hierarchy (installed `posix` package with all the needed basic tools.  Overall -- looks great, but:
+
+[[!format sh """
+    prop_view_roundtrips:                                 FAIL (0.30s)
+      *** Failed! Falsified (after 524 tests and 1 shrink):
+      "a"
+      MetaData (fromList [(MetaField "8",fromList [MetaValue (CurrentlySet False) "",MetaValue (CurrentlySet True) "\nD\EM",MetaValue (CurrentlySet True) "GO`!)",MetaValue (CurrentlySet False) "k\FS\CAN"]),(MetaField "dU",fromList [MetaValue (CurrentlySet True) "",MetaValue (CurrentlySet False) "\NUL44Vfm[\t",MetaValue (CurrentlySet True) "\nLMEgYc",MetaValue (CurrentlySet True) "\SO[",MetaValue (CurrentlySet True) "\FS\DC4\DLE\"3",MetaValue (CurrentlySet True) ";\f0&Wc\GS{^",MetaValue (CurrentlySet True) "D",MetaValue (CurrentlySet True) "c:"]),(MetaField "sV",fromList [MetaValue (CurrentlySet True) "",MetaValue (CurrentlySet False) "\STX8#w",MetaValue (CurrentlySet False) "\ny",MetaValue (CurrentlySet False) "\DC4qOq",MetaValue (CurrentlySet True) "\FSbqjq",MetaValue (CurrentlySet True) "T_bx%[lN",MetaValue (CurrentlySet True) "W0`",MetaValue (CurrentlySet True) "~ ueY"]),(MetaField "V",fromList [MetaValue (CurrentlySet False) "",MetaValue (CurrentlySet False) "\t\DC1~`\SOHv\DC1",MetaValue (CurrentlySet True) "\DLE3",MetaValue (CurrentlySet True) "/MZh$",MetaValue (CurrentlySet False) "0",MetaValue (CurrentlySet False) "MEulc",MetaValue (CurrentlySet True) "P5D",MetaValue (CurrentlySet True) "i|S,",MetaValue (CurrentlySet True) "x|C"])])
+      True
+      Use --quickcheck-replay=742853 to reproduce.
+"""]]
+
+unfortunately I cannot tell from that output what could be the problem.  Please let me know if hard to figure it out and I should provide access to such environment (ATM needs effort, so I do not want to spend time on that unless "no other way")
+
+And it seems it might be a flaky test -- I started another run, it is still running but I this test did not fail 
+
+```
+$ grep prop_view_roundtrips git-annex-test-miniconda*.log
+git-annex-test-miniconda-2.log:    prop_view_roundtrips:                                 OK (2.51s)
+git-annex-test-miniconda.log:    prop_view_roundtrips:                                 FAIL (0.30s)
+```
+
+Cheers,
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

diff --git a/doc/forum/include__61__subdir__47____42___in_prefererred_content_expression_not_working.mdwn b/doc/forum/include__61__subdir__47____42___in_prefererred_content_expression_not_working.mdwn
new file mode 100644
index 000000000..275a052b4
--- /dev/null
+++ b/doc/forum/include__61__subdir__47____42___in_prefererred_content_expression_not_working.mdwn
@@ -0,0 +1,31 @@
+Hi there,
+
+If I use a preferred content expression like the following:
+
+    include=subdir/*
+
+it does not work in commands like `git annex get --auto`.
+
+But the following works:
+
+    include=*
+
+Here is a short bash script that can be used to recreate this issue:
+
+    mkdir annexA && cd annexA
+    git init && git annex init
+    git remote add annexB ../annexB
+
+    mkdir ../annexB && cd ../annexB
+    git init && git annex init
+    git annex wanted . "include=subdir/*"
+    git remote add annexA ../annexA
+
+    cd ../annexA && mkdir subdir
+    echo "lorem ipsum" > subdir/test.txt
+    git annex add . && git annex sync
+
+    cd ../annexB && git annex sync
+    git annex get --auto
+
+I'm using **Git 2.28.0.windows.1** and **git-annex 8.20200815-g335aae266**.

Added a comment: where to trace the Windows build errors?
diff --git a/doc/bugs/windows_dosn__39__t_build_again__63____33__/comment_1_df3359fee007094877286161e192d8af._comment b/doc/bugs/windows_dosn__39__t_build_again__63____33__/comment_1_df3359fee007094877286161e192d8af._comment
new file mode 100644
index 000000000..94b0e99e0
--- /dev/null
+++ b/doc/bugs/windows_dosn__39__t_build_again__63____33__/comment_1_df3359fee007094877286161e192d8af._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="nix.zahlen@1211ac6c964ba2d68b70655f747bef1383032e77"
+ nickname="nix.zahlen"
+ avatar="http://cdn.libravatar.org/avatar/66cc45a05749fe8d4ca36d8c6071da51"
+ subject="where to trace the Windows build errors?"
+ date="2020-10-25T15:23:01Z"
+ content="""
+Thanks for you comment / fix...
+
+So where would you like to trace the Windows build errors if not here?
+"""]]

Added a comment
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_5_167ade7bcd58354280df31442b586024._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_5_167ade7bcd58354280df31442b586024._comment
new file mode 100644
index 000000000..7656acb43
--- /dev/null
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_5_167ade7bcd58354280df31442b586024._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="achilleas.k@14be77d42a1252fab5ec9dbf4e5ea03c5833e8c8"
+ avatar="http://cdn.libravatar.org/avatar/ed6c67c4d8e6c6850930e16eaf85a771"
+ subject="comment 5"
+ date="2020-10-24T14:39:38Z"
+ content="""
+Awesome!  Thanks for the quick response and fix Joey!
+"""]]

Added a comment
diff --git a/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_5_096b3af691987ef20281c87519a31935._comment b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_5_096b3af691987ef20281c87519a31935._comment
new file mode 100644
index 000000000..8d9c6f5e3
--- /dev/null
+++ b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_5_096b3af691987ef20281c87519a31935._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 5"
+ date="2020-10-24T12:26:01Z"
+ content="""
+woohoo [last run](https://github.com/datalad/datalad-extensions/runs/1300942377?check_suite_focus=true) had ` All 704 tests passed (698.50s)` !!! so it might have fixed it.  I will submit a PR removing `|| true` to see if that sticks, and also will test locally
+"""]]

Added a comment
diff --git a/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_4_2f701f39286c3a75a88407dcc6947081._comment b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_4_2f701f39286c3a75a88407dcc6947081._comment
new file mode 100644
index 000000000..a7ebabca4
--- /dev/null
+++ b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_4_2f701f39286c3a75a88407dcc6947081._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 4"
+ date="2020-10-24T01:21:09Z"
+ content="""
+Cool! We will see on the next cron run, I will check tomorrow
+"""]]

remove dead links to youtube-dl
Presumably most users will get it from their linux distribution, or
whatever. Don't want to need to keep updating links if they're going to
rot like this.
diff --git a/doc/tips/downloading_podcasts.mdwn b/doc/tips/downloading_podcasts.mdwn
index 1b77abf6b..7bb7f67d2 100644
--- a/doc/tips/downloading_podcasts.mdwn
+++ b/doc/tips/downloading_podcasts.mdwn
@@ -74,7 +74,7 @@ and transferring to your laptop on demand.
 ## youtube channels
 
 You can also use `git annex importfeed` on youtube channels.
-It will use [youtube-dl](https://rg3.github.io/youtube-dl/) to automatically
+It will use youtube-dl to automatically
 download the videos.
 
 To download a youtube channel, you need to find the feed associated with that
diff --git a/doc/tips/using_the_web_as_a_special_remote.mdwn b/doc/tips/using_the_web_as_a_special_remote.mdwn
index e5844addb..e4065cafe 100644
--- a/doc/tips/using_the_web_as_a_special_remote.mdwn
+++ b/doc/tips/using_the_web_as_a_special_remote.mdwn
@@ -71,8 +71,7 @@ number takes that many paths from the end.
 <a name=videos></a>
 
 There's support for downloading videos from sites like YouTube, Vimeo,
-and many more. This relies on [youtube-dl](https://rg3.github.io/youtube-dl/)
-to download the videos.
+and many more. This relies on youtube-dl to download the videos.
 
 When you have youtube-dl installed, you can just 
 `git annex addurl http://youtube.com/foo` and it will detect that

use removeDirGeneric here too for consistency
And because it might be more robust on windows.
diff --git a/Remote/Directory.hs b/Remote/Directory.hs
index 377e6ce80..a4a2c6ccd 100644
--- a/Remote/Directory.hs
+++ b/Remote/Directory.hs
@@ -195,8 +195,7 @@ store d chunkconfig k b p = liftIO $ do
  - down. -}
 finalizeStoreGeneric :: FilePath -> FilePath -> FilePath -> IO ()
 finalizeStoreGeneric d tmp dest = do
-	void $ tryIO $ allowWrite dest -- may already exist
-	void $ tryIO $ removeDirectoryRecursive dest -- or not exist
+	removeDirGeneric d dest
 	createDirectoryUnder d (parentDir dest)
 	renameDirectory tmp dest
 	-- may fail on some filesystems
diff --git a/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_3_c570246b0484ed3694efff62feff968c._comment b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_3_c570246b0484ed3694efff62feff968c._comment
new file mode 100644
index 000000000..42ca756f5
--- /dev/null
+++ b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_3_c570246b0484ed3694efff62feff968c._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-10-23T20:10:27Z"
+ content="""
+Turns out Remote.Directory was using removeDirGeneric everywhere it needed
+to remove a directory, except for in what I believe is failing here.
+And removeDirGeneric already has a workaround for some windows quirk in it.
+
+So I changed that to use it too. I don't know if that will fix the problem.
+"""]]

testremote: Display exceptions when tests fail, to aid debugging
diff --git a/CHANGELOG b/CHANGELOG
index 9d64142ea..c0b8a0756 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -14,6 +14,7 @@ git-annex (8.20201008) UNRELEASED; urgency=medium
     chunks using the configured chunk size.
   * Fixed some problems that prevented this command from working:
     git submodule foreach git annex init
+  * testremote: Display exceptions when tests fail, to aid debugging.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Oct 2020 10:48:17 -0400
 
diff --git a/Command/TestRemote.hs b/Command/TestRemote.hs
index e09b6708c..a07ab67b3 100644
--- a/Command/TestRemote.hs
+++ b/Command/TestRemote.hs
@@ -235,15 +235,15 @@ mkTestTrees runannex mkrs mkunavailr mkexportr mkks = concat $
 test :: RunAnnex -> Annex (Maybe Remote) -> Annex Key -> [TestTree]
 test runannex mkr mkk =
 	[ check "removeKey when not present" $ \r k ->
-		whenwritable r $ isRight <$> tryNonAsync (remove r k)
+		whenwritable r $ runBool (remove r k)
 	, check ("present " ++ show False) $ \r k ->
 		whenwritable r $ present r k False
 	, check "storeKey" $ \r k ->
-		whenwritable r $ isRight <$> tryNonAsync (store r k)
+		whenwritable r $ runBool (store r k)
 	, check ("present " ++ show True) $ \r k ->
 		whenwritable r $ present r k True
 	, check "storeKey when already present" $ \r k ->
-		whenwritable r $ isRight <$> tryNonAsync (store r k)
+		whenwritable r $ runBool (store r k)
 	, check ("present " ++ show True) $ \r k -> present r k True
 	, check "retrieveKeyFile" $ \r k -> do
 		lockContentForRemoval k noop removeAnnex
@@ -273,7 +273,7 @@ test runannex mkr mkk =
 		get r k
 	, check "fsck downloaded object" fsck
 	, check "removeKey when present" $ \r k -> 
-		whenwritable r $ isRight <$> tryNonAsync (remove r k)
+		whenwritable r $ runBool (remove r k)
 	, check ("present " ++ show False) $ \r k -> 
 		whenwritable r $ present r k False
 	]
@@ -306,31 +306,31 @@ testExportTree runannex mkr mkk1 mkk2 =
 	[ check "check present export when not present" $ \ea k1 _k2 ->
 		not <$> checkpresentexport ea k1
 	, check "remove export when not present" $ \ea k1 _k2 -> 
-		isRight <$> tryNonAsync (removeexport ea k1)
+		runBool (removeexport ea k1)
 	, check "store export" $ \ea k1 _k2 ->
-		isRight <$> tryNonAsync (storeexport ea k1)
+		runBool (storeexport ea k1)
 	, check "check present export after store" $ \ea k1 _k2 ->
 		checkpresentexport ea k1
 	, check "store export when already present" $ \ea k1 _k2 ->
-		isRight <$> tryNonAsync (storeexport ea k1)
+		runBool (storeexport ea k1)
 	, check "retrieve export" $ \ea k1 _k2 -> 
 		retrieveexport ea k1
 	, check "store new content to export" $ \ea _k1 k2 ->
-		isRight <$> tryNonAsync (storeexport ea k2)
+		runBool (storeexport ea k2)
 	, check "check present export after store of new content" $ \ea _k1 k2 ->
 		checkpresentexport ea k2
 	, check "retrieve export new content" $ \ea _k1 k2 ->
 		retrieveexport ea k2
 	, check "remove export" $ \ea _k1 k2 -> 
-		isRight <$> tryNonAsync (removeexport ea k2)
+		runBool (removeexport ea k2)
 	, check "check present export after remove" $ \ea _k1 k2 ->
 		not <$> checkpresentexport ea k2
 	, check "retrieve export fails after removal" $ \ea _k1 k2 ->
 		not <$> retrieveexport ea k2
 	, check "remove export directory" $ \ea _k1 _k2 ->
-		isRight <$> tryNonAsync (removeexportdirectory ea)
+		runBool (removeexportdirectory ea)
 	, check "remove export directory that is already removed" $ \ea _k1 _k2 ->
-		isRight <$> tryNonAsync (removeexportdirectory ea)
+		runBool (removeexportdirectory ea)
 	-- renames are not tested because remotes do not need to support them
 	]
   where
@@ -448,3 +448,9 @@ getReadonlyKey r f = lookupKey (toRawFilePath f) >>= \case
 		unlessM ((Remote.uuid r `elem`) <$> loggedLocations k) $
 			giveup $ f ++ " is not stored in the remote being tested, cannot test it"
 		return k
+
+runBool :: Monad m => m () -> m Bool
+runBool a = do
+	a
+	return True
+
diff --git a/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_2_07f7881be6b0d29cdf64a7839b669b3e._comment b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_2_07f7881be6b0d29cdf64a7839b669b3e._comment
new file mode 100644
index 000000000..7c14b1e87
--- /dev/null
+++ b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_2_07f7881be6b0d29cdf64a7839b669b3e._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-10-23T19:27:57Z"
+ content="""
+I've made it display whatever exception caused the test to fail, which
+should help pin down a bit more what the problem is.
+"""]]

deal with .git pointer file in Git.CurrentRepo
This fixes the bug.
Note, it's only done when GIT_DIR is set. When it's not set,
Git.Construct already handled it. This is why it was only noticed with this
git submodule command.
This commit was sponsored by Brett Eisenberg on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index 213d07c42..9d64142ea 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -12,6 +12,8 @@ git-annex (8.20201008) UNRELEASED; urgency=medium
   * When a special remote has chunking enabled, but no chunk sizes are
     recorded (or the recorded ones are not found), speculatively try
     chunks using the configured chunk size.
+  * Fixed some problems that prevented this command from working:
+    git submodule foreach git annex init
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Oct 2020 10:48:17 -0400
 
diff --git a/Git/Construct.hs b/Git/Construct.hs
index 19c89958f..39ac04df1 100644
--- a/Git/Construct.hs
+++ b/Git/Construct.hs
@@ -21,6 +21,7 @@ module Git.Construct (
 	repoAbsPath,
 	checkForRepo,
 	newFrom,
+	adjustGitDirFile,
 ) where
 
 #ifndef mingw32_HOST_OS
@@ -73,7 +74,7 @@ fromAbsPath dir
 				, ret (takeDirectory canondir)
 				)
 		| otherwise = ifM (doesDirectoryExist dir)
-			( gitDirFile dir >>= maybe (ret dir) (pure . newFrom)
+			( checkGitDirFile dir >>= maybe (ret dir) (pure . newFrom)
 			-- git falls back to dir.git when dir doesn't
 			-- exist, as long as dir didn't end with a
 			-- path separator
@@ -198,7 +199,7 @@ expandTilde = expandt True
 checkForRepo :: FilePath -> IO (Maybe RepoLocation)
 checkForRepo dir = 
 	check isRepo $
-		check (gitDirFile dir) $
+		check (checkGitDirFile dir) $
 			check isBareRepo $
 				return Nothing
   where
@@ -219,24 +220,36 @@ checkForRepo dir =
 		<&&> doesDirectoryExist (dir </> "objects")
 	gitSignature file = doesFileExist $ dir </> file
 
+-- Check for a .git file.
+checkGitDirFile :: FilePath -> IO (Maybe RepoLocation)
+checkGitDirFile dir = adjustGitDirFile' $ Local 
+	{ gitdir = toRawFilePath (dir </> ".git")
+	, worktree = Just (toRawFilePath dir)
+	}
+
 -- git-submodule, git-worktree, and --separate-git-dir
 -- make .git be a file pointing to the real git directory.
 -- Detect that, and return a RepoLocation with gitdir pointing 
 -- to the real git directory.
-gitDirFile :: FilePath -> IO (Maybe RepoLocation)
-gitDirFile dir = do
-	c <- firstLine <$>
-		catchDefaultIO "" (readFile $ dir </> ".git")
-	return $ if gitdirprefix `isPrefixOf` c
-		then Just $ Local 
-			{ gitdir = toRawFilePath $ absPathFrom dir $
-				drop (length gitdirprefix) c
-			, worktree = Just (toRawFilePath dir)
-			}
-		else Nothing
+adjustGitDirFile :: RepoLocation -> IO RepoLocation
+adjustGitDirFile loc = fromMaybe loc <$> adjustGitDirFile' loc
+
+adjustGitDirFile' :: RepoLocation -> IO (Maybe RepoLocation)
+adjustGitDirFile' loc = do
+	let gd = fromRawFilePath (gitdir loc)
+	c <- firstLine <$> catchDefaultIO "" (readFile gd)
+	if gitdirprefix `isPrefixOf` c
+		then do
+			top <- takeDirectory <$> absPath gd
+			return $ Just $ loc
+				{ gitdir = toRawFilePath $ absPathFrom top $
+					drop (length gitdirprefix) c
+				}
+		else return Nothing
  where
 	gitdirprefix = "gitdir: "
 
+
 newFrom :: RepoLocation -> Repo
 newFrom l = Repo
 	{ location = l
diff --git a/Git/CurrentRepo.hs b/Git/CurrentRepo.hs
index 054a81e0b..508e71cc9 100644
--- a/Git/CurrentRepo.hs
+++ b/Git/CurrentRepo.hs
@@ -67,15 +67,14 @@ get = do
 	configure (Just d) _ = do
 		absd <- absPath d
 		curr <- getCurrentDirectory
-		r <- Git.Config.read $ newFrom $
-			Local
-				{ gitdir = toRawFilePath absd
-				, worktree = Just (toRawFilePath curr)
-				}
+		loc <- adjustGitDirFile $ Local
+			{ gitdir = toRawFilePath absd
+			, worktree = Just (toRawFilePath curr)
+			}
+		r <- Git.Config.read $ newFrom loc
 		return $ if Git.Config.isBare r
 			then r { location = (location r) { worktree = Nothing } }
 			else r
-
 	configure Nothing Nothing = giveup "Not in a git repository."
 
 	addworktree w r = changelocation r $ Local
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__.mdwn b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__.mdwn
index c79dcae13..873a233c4 100644
--- a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__.mdwn
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__.mdwn
@@ -28,3 +28,5 @@ As a workaround, manually changing directory into each submodule and running `gi
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 I use it regularly with great success :)
+
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_23f3dac4a22cf7202a0bb849ee499766._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_23f3dac4a22cf7202a0bb849ee499766._comment
index a57d48445..27a3c5bd0 100644
--- a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_23f3dac4a22cf7202a0bb849ee499766._comment
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_23f3dac4a22cf7202a0bb849ee499766._comment
@@ -25,13 +25,11 @@ Where does that path that does not exist come from?
 .git/modules/sub/config has "core.worktree = ../../../sub"
 
 git clone sets that. And it's a path from ../.git/modules/sub/
-back to the work tree. git-annex interprets it as well, a regular path
-from the cwd, like it usually is. It's worth noting that setting
-`GIT_WORK_TREE` to the same value makes git behave differently!
-Nothing in git's docs seem to explain this mess.
+back to the work tree. git-annex interprets it as relative to `GIT_DIR`,
+which git submodule sets to ".git".
 
 So git-annex needs to check if .git is a gitdir: reference, and
-then interpret core.worktree relative to it.
+then interpret core.worktree relative to what it points to.
 
 Or, alternatively, fixupUnusualRepos could learn to tell if the *current*
 command is git-annex init, and go ahead with the fixes. Which would avoid

comment
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_4_e772639c09c1089700b2c2c170ea458e._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_4_e772639c09c1089700b2c2c170ea458e._comment
new file mode 100644
index 000000000..423a42244
--- /dev/null
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_4_e772639c09c1089700b2c2c170ea458e._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2020-10-23T18:19:03Z"
+ content="""
+Tried to implement that idea of detecting if the repo is in the process of
+initializing, but automatic initialization makes that impractical to
+implement, I think.
+"""]]

analysis
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_23f3dac4a22cf7202a0bb849ee499766._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_23f3dac4a22cf7202a0bb849ee499766._comment
new file mode 100644
index 000000000..a57d48445
--- /dev/null
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_23f3dac4a22cf7202a0bb849ee499766._comment
@@ -0,0 +1,39 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-10-23T16:21:12Z"
+ content="""
+Let's see, sub/.git is a file not a directory, and that's what it's trying
+to chdir to, when running git.
+
+fixupUnusualRepos usually converts that file to a symlink to a directory,
+but when the annex has not been initialized yet, it avoids doing so, because
+users would be unhappy if an accidential git-annex run in a submodule not
+initialized for git-annex started messing with it.
+
+git-annex does not normally chdir when running git, and the only place that
+does is Config.read'. Which only does it to make git read the right config,
+for when it's reading some remote's config. So, using --git-dir there will
+have the same effect, and avoid the problem.
+
+Which it did, but then there's the new problem not much further along:
+
+	git-annex: /home/joey/tmp/gx-KHSxawT/sub: changeWorkingDirectory: does not exist (No such file or directory)
+
+This is from Git.CurrentRepo.get, which does a setCurrentDirectory.
+Where does that path that does not exist come from?
+.git/modules/sub/config has "core.worktree = ../../../sub"
+
+git clone sets that. And it's a path from ../.git/modules/sub/
+back to the work tree. git-annex interprets it as well, a regular path
+from the cwd, like it usually is. It's worth noting that setting
+`GIT_WORK_TREE` to the same value makes git behave differently!
+Nothing in git's docs seem to explain this mess.
+
+So git-annex needs to check if .git is a gitdir: reference, and
+then interpret core.worktree relative to it.
+
+Or, alternatively, fixupUnusualRepos could learn to tell if the *current*
+command is git-annex init, and go ahead with the fixes. Which would avoid
+needing to chase more of this weird stuff.
+"""]]

close
diff --git a/doc/bugs/git_annex_init_fails.mdwn b/doc/bugs/git_annex_init_fails.mdwn
index 5efa3262c..67307064d 100644
--- a/doc/bugs/git_annex_init_fails.mdwn
+++ b/doc/bugs/git_annex_init_fails.mdwn
@@ -32,3 +32,5 @@ CentOS8 / git-annex-standalone-8.20200909-1.x86_64
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 No, first time install
+
+> [[fixed|done]] --[[Joey]]

analysis
diff --git a/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_1_7e089c436af03ffd8ad00dba45156a14._comment b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_1_7e089c436af03ffd8ad00dba45156a14._comment
new file mode 100644
index 000000000..66c94579f
--- /dev/null
+++ b/doc/bugs/storeKey_when_already_present_failures_on_Windows/comment_1_7e089c436af03ffd8ad00dba45156a14._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-10-23T16:50:31Z"
+ content="""
+So this is the testremote of the directory special remote, and the failing
+test tries to store content into the remote which already contains that
+content, an unusual edge case since git-annex normally knows when a remote
+has content and avoids resending it.
+
+Remote.Directory.finalizeStoreGeneric removes the old directory, and moves
+the new one into place. This is not the first time I've seen windows fail
+in such a situation, it seems that directory removals are not atomic, or
+don't really fully finish by the time the call returns, or something else
+like antivirus has a file open in the directory and so prevents deletion.
+Or something like that.
+
+This kind of thing is why I am not enthused about wasting any more time
+on supporting Windows. `rm -rf foo && mv bar foo` should not cause a bug report
+that requires me to dig out a VM and put in complex and expensive
+workarounds. The tar pit has no bottom.
+
+(Also, FWIW, git-annex test on the jenkins autobuilder always exploded from
+the very beginning, so was always || true, which is why the windows install
+page encourages windows users to test git-annex themselves.)
+"""]]

Added a comment
diff --git a/doc/bugs/git_annex_init_fails/comment_3_479898de9a6635d413b9a6b15461ba7c._comment b/doc/bugs/git_annex_init_fails/comment_3_479898de9a6635d413b9a6b15461ba7c._comment
new file mode 100644
index 000000000..b56e78e92
--- /dev/null
+++ b/doc/bugs/git_annex_init_fails/comment_3_479898de9a6635d413b9a6b15461ba7c._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="rshalaev@3e2130a1e3cb0aaff7dd80aba7548ad9be0ea2d4"
+ nickname="rshalaev"
+ avatar="http://cdn.libravatar.org/avatar/d7f181d338cbcef7418faa01f0441e86"
+ subject="comment 3"
+ date="2020-10-23T16:51:04Z"
+ content="""
+Joey, thank you for the quick response! I applied your suggestion to create `locales` directory - and git annex is now working. This is a very exciting project - look forward to diving in to it. Thanks again! ~Ruslan
+"""]]

Fix bug that prevented linux standalone bundle from working on a fresh install
Bug was introduced in version 8.20201007, lost a necessary mkdir.
This commit was sponsored by Noam Kremen on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index 0d585c1ae..213d07c42 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,5 +1,7 @@
 git-annex (8.20201008) UNRELEASED; urgency=medium
 
+  * Fix bug that prevented linux standalone bundle from working on a fresh
+    install.
   * Fix build on Windows with network-3.
   * Fix a memory leak introduced in the last release.
   * add, import: Fix a reversion in 7.20191009 that broke handling
diff --git a/doc/bugs/git_annex_init_fails/comment_2_7c287bc03f8aa7fe40afe71f8f377593._comment b/doc/bugs/git_annex_init_fails/comment_2_7c287bc03f8aa7fe40afe71f8f377593._comment
new file mode 100644
index 000000000..284a1d48f
--- /dev/null
+++ b/doc/bugs/git_annex_init_fails/comment_2_7c287bc03f8aa7fe40afe71f8f377593._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-10-23T16:15:42Z"
+ content="""
+I've fixed this for the next release. It was simply omitting creating the 
+/home/user/.cache/git-annex/locales/ directory, so as a workaround you can
+create that directory.
+"""]]
diff --git a/standalone/linux/skel/runshell b/standalone/linux/skel/runshell
index 4daf57dae..4413be985 100755
--- a/standalone/linux/skel/runshell
+++ b/standalone/linux/skel/runshell
@@ -151,6 +151,10 @@ if [ -z "${LOCPATH+set}" ] && [ -z "$GIT_ANNEX_PACKAGE_INSTALL" ]; then
 	done
 	
 	if [ ! -d "$LOCPATH" ]; then
+		if ! mkdir -p "$HOME/.cache/git-annex/locales"; then
+			echo "Unable to write to $HOME/.cache/git-annex/locales; can't continue!" >&2
+			exit 1
+		fi
 		locpathtmp="$HOME/.cache/git-annex/locales.tmp/$locpathbase.$$"
 		if ! mkdir -p "$locpathtmp"; then
 			echo "Unable to write to $locpathtmp; can't continue!" >&2

Added a comment
diff --git a/doc/devblog/day_633__ten_years/comment_1_4842e53398d9ee668053ec6d4da426dd._comment b/doc/devblog/day_633__ten_years/comment_1_4842e53398d9ee668053ec6d4da426dd._comment
new file mode 100644
index 000000000..3bc85ff9a
--- /dev/null
+++ b/doc/devblog/day_633__ten_years/comment_1_4842e53398d9ee668053ec6d4da426dd._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="amindfv@97236fbaab6048ce6805b2737b27dd7f1cd51da4"
+ nickname="amindfv"
+ avatar="http://cdn.libravatar.org/avatar/a82f30bc12d79fee2d142021af8e9aa0"
+ subject="comment 1"
+ date="2020-10-23T05:37:40Z"
+ content="""
+Many congratulations and many thanks! Git annex has been huge for me and I know it has been for lots and lots of other people too. Here's to another happy and healthy decade for you and for Git Annex, and then another decade, and then another :)
+"""]]

Added a comment
diff --git a/doc/bugs/git_annex_init_fails/comment_1_2ee620c8ccb2ca411f303edda29bf62a._comment b/doc/bugs/git_annex_init_fails/comment_1_2ee620c8ccb2ca411f303edda29bf62a._comment
new file mode 100644
index 000000000..72bc67218
--- /dev/null
+++ b/doc/bugs/git_annex_init_fails/comment_1_2ee620c8ccb2ca411f303edda29bf62a._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="rshalaev@3e2130a1e3cb0aaff7dd80aba7548ad9be0ea2d4"
+ nickname="rshalaev"
+ avatar="http://cdn.libravatar.org/avatar/d7f181d338cbcef7418faa01f0441e86"
+ subject="comment 1"
+ date="2020-10-23T02:00:08Z"
+ content="""
+Line endings got removed in the original post, so for clarification, after installing git-annex, I'm following this steps:
+
+1. `mkdir ~/annex`
+2. `cd ~/annex`
+3. `git init`
+4. `git annex init`
+
+command that fails is `git annex init` (error message is `mv: cannot move '/home/user/.cache/git-annex/locales.tmp/b03b775f010002f73bc923b1b46a73c7.4662' to '/home/user/.cache/git-annex/locales/b03b775f010002f73bc923b1b46a73c7': No such file or directory`)
+"""]]

diff --git a/doc/bugs/git_annex_init_fails.mdwn b/doc/bugs/git_annex_init_fails.mdwn
new file mode 100644
index 000000000..5efa3262c
--- /dev/null
+++ b/doc/bugs/git_annex_init_fails.mdwn
@@ -0,0 +1,34 @@
+### Please describe the problem.
+
+git annex fails init step:
+
+i.e.:
+$ mkdir ~/annex
+$ cd ~/annex
+$ git init
+$ git annex init < this step fails:
+
+mv: cannot move '/home/user/.cache/git-annex/locales.tmp/b03b775f010002f73bc923b1b46a73c7.4328' to '/home/user/.cache/git-annex/locales/b03b775f010002f73bc923b1b46a73c7': No such file or directory
+
+
+### What steps will reproduce the problem?
+
+install git annex on CentOS8 , try to perform git annex init
+
+### What version of git-annex are you using? On what operating system?
+
+CentOS8 / git-annex-standalone-8.20200909-1.x86_64
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+No, first time install

initial report on a failing test on windows about storeKey when already present
diff --git a/doc/bugs/storeKey_when_already_present_failures_on_Windows.mdwn b/doc/bugs/storeKey_when_already_present_failures_on_Windows.mdwn
new file mode 100644
index 000000000..96325f40f
--- /dev/null
+++ b/doc/bugs/storeKey_when_already_present_failures_on_Windows.mdwn
@@ -0,0 +1,17 @@
+### Please describe the problem.
+Now we have github action which builds git-annex with libmagic support.  Unfortunately we had to "allow" `git annex test` to fail to proceed for now. Although it looks not that bad -- there is only one "test" which fails in various scenarios (to the total of `12 out of 704 tests failed (616.78s)`)
+
+```
+2020-10-22T18:50:07.2946875Z         storeKey when already present:                    FAIL (0.02s)
+2020-10-22T18:50:07.2948174Z           .\Command\TestRemote.hs:290:
+2020-10-22T18:50:07.2949772Z           failed
+```
+
+
+[full log copy](http://www.onerussian.com/tmp/git-annex-windows-failures-8.20201008-g577af1b.log)
+[link to original workflow run](https://github.com/datalad/datalad-extensions/runs/1294367575?check_suite_focus=true)
+
+When I ran `annex test` after installing that installer in a local Win 10 VM, I have got a lesser count (`8 out of 704 tests failed (1197.82s)`)  [full log](http://www.onerussian.com/tmp/git-annex-windows-failures-8.20201008-g577af1b-local.log)
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

this protocol is not draft for some time
diff --git a/doc/design/external_backend_protocol.mdwn b/doc/design/external_backend_protocol.mdwn
index a4a8a22a4..8168099dd 100644
--- a/doc/design/external_backend_protocol.mdwn
+++ b/doc/design/external_backend_protocol.mdwn
@@ -1,5 +1,3 @@
-**Draft**
-
 Communication between git-annex and a program implementing an external
 [[backend|backends]] uses this protocol.
 

turns out this was fixed in 2014
diff --git a/Remote/Helper/Chunked.hs b/Remote/Helper/Chunked.hs
index 2ef836018..0cd316a4b 100644
--- a/Remote/Helper/Chunked.hs
+++ b/Remote/Helper/Chunked.hs
@@ -120,6 +120,10 @@ storeChunks
 	-> Annex ()
 storeChunks u chunkconfig encryptor k f p storer checker = 
 	case chunkconfig of
+		-- Only stable keys are safe to store chunked,
+		-- because an unstable key can have multiple different
+		-- objects, and mixing up chunks from them would be
+		-- possible without this check.
 		(UnpaddedChunks chunksize) -> ifM (isStableKey k)
 			( do
 				h <- liftIO $ openBinaryFile f ReadMode
diff --git a/doc/bugs/possible_data_loss_when_unsized_key_stored_chunked.mdwn b/doc/bugs/possible_data_loss_when_unsized_key_stored_chunked.mdwn
index 118339817..677dd4904 100644
--- a/doc/bugs/possible_data_loss_when_unsized_key_stored_chunked.mdwn
+++ b/doc/bugs/possible_data_loss_when_unsized_key_stored_chunked.mdwn
@@ -39,3 +39,10 @@ since it can fail so badly with them, and they're kind of a side thing?
 (Could continue retrieving, for whatever is stored hopefully w/o being
 corrupted already.)
 --[[Joey]]
+
+> This would also affect any key that is not stable.
+> And oh, it stopped using chunks to store non-stable keys in 2014.
+> 
+> So, this can't really happen with url keys, because they're not stable.
+> Ok, not a bug then, because if the key is stable, there can only be one
+> object for it, by definition. Whew! [[done]] --[[Joey]]

urk
diff --git a/doc/bugs/possible_data_loss_when_unsized_key_stored_chunked.mdwn b/doc/bugs/possible_data_loss_when_unsized_key_stored_chunked.mdwn
new file mode 100644
index 000000000..118339817
--- /dev/null
+++ b/doc/bugs/possible_data_loss_when_unsized_key_stored_chunked.mdwn
@@ -0,0 +1,41 @@
+When a key has no known size (from addurl --relaxed eg), I think data loss
+could occur in this situation:
+
+* repo A has an object for the key with size X
+* repo B has an object for the same key with size Y (!= X)
+* repo A transfers to the special remote
+* then B transfers to the special remote
+* B transfers one more chunk than A, because of the different size
+* B actually "resumes" after the last chunk A uploaded. So now the remote
+  contains A's chunks, followed by B's extra chunk.
+* A and B sync up, which merges the chunk logs. Since that log
+  uses "key:chunksize" as the log key, and the two logs have two different
+  ones, one will win or come first in the merged log. Suppose it's
+  the entry for B. So, the log then will be interpreted as the number of
+  chunks being B's.
+* Now when the object is retrieved from the special remote, it will
+  retrieve and concacenate A's chunks, followed by B's extra chunk.
+
+So this is corruption at least, it can be recovered from, but to do so
+you have to know the original length of A's object. Note that most keys
+with unknown size also have no checksum to use to verify them, so it would
+be easy for this to happen and not be caught.
+
+(Alternatively, after B transfers, it can sync with A, drop, and get
+the content back from the special remote. Same result by another route,
+and without needing any particular git-annex branch merge behavior to
+happen so easier to reproduce. (I have not tried either yet.))
+
+A simulantaneous upload by A and B might cause unrecoverable data loss
+if they eg alternate chunks. Unsure if that can really happen.
+
+If A starts to transfer, sends some chunks, but is interrupted, and B
+then transfers, resuming after the last chunk A stored, that would be data
+loss.
+
+It might be best to just disable storing in chunks for keys of unknown size,
+since it can fail so badly with them, and they're kind of a side thing?
+
+(Could continue retrieving, for whatever is stored hopefully w/o being
+corrupted already.)
+--[[Joey]]

Added a comment: windows build with magic
diff --git a/doc/todo/provide_windows_build_with_MagicMime/comment_7_d16e4fb3d0be2acd23d158272171b842._comment b/doc/todo/provide_windows_build_with_MagicMime/comment_7_d16e4fb3d0be2acd23d158272171b842._comment
new file mode 100644
index 000000000..df8797e74
--- /dev/null
+++ b/doc/todo/provide_windows_build_with_MagicMime/comment_7_d16e4fb3d0be2acd23d158272171b842._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="windows build with magic"
+ date="2020-10-22T19:32:55Z"
+ content="""
+FWIW, the mission of automated building of windows installer for annex with libmagic support is accomplished.  
+
+- It is 64bit build, works on Windows 10 **but in my experience does not work on Windows 7**. [info/pain to share if interested](https://github.com/datalad/datalad-extensions/pull/40#issuecomment-713853822)
+- Not sure if artifacts are publicly available [from github](https://github.com/datalad/datalad-extensions/runs/1294145386?check_suite_focus=true), but current copy could be fetched from [my personal website](http://www.onerussian.com/tmp/git-annex-windows-installer_20201022.zip).  We are [thinking](https://github.com/datalad/datalad-extensions/issues/46) about organizing public consistent distribution for them
+- libmagic is cross-build on Linux, based on [file-windows](https://github.com/nscaife/file-windows) with some fixups (?) in [our clone](https://github.com/datalad/file-windows)
+"""]]

Added a comment
diff --git a/doc/bugs/Error_cloning_repository_on_Windows/comment_4_70595ac6d747eb0cabffb2844fc036a4._comment b/doc/bugs/Error_cloning_repository_on_Windows/comment_4_70595ac6d747eb0cabffb2844fc036a4._comment
new file mode 100644
index 000000000..15826e5aa
--- /dev/null
+++ b/doc/bugs/Error_cloning_repository_on_Windows/comment_4_70595ac6d747eb0cabffb2844fc036a4._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="jwodder"
+ avatar="http://cdn.libravatar.org/avatar/b06e01332c949b895c681cc92934f36a"
+ subject="comment 4"
+ date="2020-10-22T18:13:40Z"
+ content="""
+Unfortunately, it turns out that empty extensions aren't the only problem.  Trying to check out the latest commit (577af1b) on Windows 7 now gives:
+
+```
+error: unable to create file doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_1_8a6cd8dc6e7c2cdefd60a07356be1d27._comment: Filename too long
+error: unable to create file doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_fd3a287ee6e73946889a4dd406fae0fc._comment: Filename too long
+error: unable to create file doc/bugs/crippled_fs___40__pidlock__41___leads_to_git-annex__58___SQLite3_returned_ErrorIO_while_attempting_to_perform_prepare___34__PRAGMA_journal__95__mode__61__WAL__59____34____58___disk_I__47__O_error/comment_1_aaea812a8f2b04d9b17d1ba9c6fad39f._comment: Filename too long
+error: unable to create file doc/bugs/crippled_fs___40__pidlock__41___leads_to_git-annex__58___SQLite3_returned_ErrorIO_while_attempting_to_perform_prepare___34__PRAGMA_journal__95__mode__61__WAL__59____34____58___disk_I__47__O_error/comment_2_c55f2d95f7f08e6f6439f36790e5a538._comment: Filename too long
+error: unable to create file doc/bugs/crippled_fs___40__pidlock__41___leads_to_git-annex__58___SQLite3_returned_ErrorIO_while_attempting_to_perform_prepare___34__PRAGMA_journal__95__mode__61__WAL__59____34____58___disk_I__47__O_error/comment_3_b49a3df3b13f564e588824910a07bc43._comment: Filename too long
+error: unable to create file doc/bugs/crippled_fs___40__pidlock__41___leads_to_git-annex__58___SQLite3_returned_ErrorIO_while_attempting_to_perform_prepare___34__PRAGMA_journal__95__mode__61__WAL__59____34____58___disk_I__47__O_error/comment_4_a9e6e37dbaa1664d825bdbd64de4fbdd._comment: Filename too long
+```
+"""]]

rename to avoid windows stupididy about legal filenames
diff --git a/doc/bugs/Error_cloning_repository_on_Windows/comment_3_2e291373a2f3cd3e90e2c0bbad2cda7d._comment b/doc/bugs/Error_cloning_repository_on_Windows/comment_3_2e291373a2f3cd3e90e2c0bbad2cda7d._comment
new file mode 100644
index 000000000..dd5a2ab76
--- /dev/null
+++ b/doc/bugs/Error_cloning_repository_on_Windows/comment_3_2e291373a2f3cd3e90e2c0bbad2cda7d._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-10-22T17:57:49Z"
+ content="""
+So I did, fixed now.
+"""]]
diff --git a/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files..mdwn b/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files.mdwn
similarity index 100%
rename from doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files..mdwn
rename to doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files.mdwn
diff --git a/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files./comment_1_04201aa4345db10dbb5fb4637946d9c1._comment b/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files/comment_1_04201aa4345db10dbb5fb4637946d9c1._comment
similarity index 100%
rename from doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files./comment_1_04201aa4345db10dbb5fb4637946d9c1._comment
rename to doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files/comment_1_04201aa4345db10dbb5fb4637946d9c1._comment
diff --git a/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files./comment_2_18501567083a4cdb1beaab401b9e7d4f._comment b/doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files/comment_2_18501567083a4cdb1beaab401b9e7d4f._comment
similarity index 100%
rename from doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files./comment_2_18501567083a4cdb1beaab401b9e7d4f._comment
rename to doc/bugs/Very_not_graceful_exit_on_cwd_deletion_while_git_annex_adding_files/comment_2_18501567083a4cdb1beaab401b9e7d4f._comment

add
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_8_dd50ae03bc9a71e04528ac793e79a8b2._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_8_dd50ae03bc9a71e04528ac793e79a8b2._comment
new file mode 100644
index 000000000..53218d213
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_8_dd50ae03bc9a71e04528ac793e79a8b2._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2020-10-22T17:41:49Z"
+ content="""
+If whatever is putting itself at the front of PATH, I think it would
+also risk breaking any part of git that expects to run another part of it
+by path. Including any git commands that use the bundled cp or other
+unix commands.
+
+Perhaps the problem is that you're mixing two different package systems?
+If there's going to be a datalad package, there should also be a git-annex
+package, and then it can be built to use the right cp options for its
+distibution.
+"""]]

add
diff --git a/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_3_ff3bcc2cce6e0cbbeef10a187541f5d3._comment b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_3_ff3bcc2cce6e0cbbeef10a187541f5d3._comment
new file mode 100644
index 000000000..dff49bbd6
--- /dev/null
+++ b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_3_ff3bcc2cce6e0cbbeef10a187541f5d3._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-10-22T17:36:12Z"
+ content="""
+Ok, made update the chunk log as needed while checking if chunks are
+present. So this is done.
+"""]]

update chunk log after speculated chunks are verified to be present
Only done in checkPresentChunks, although retrieveChunks could also do
it. Does not seem necessary though, because git-annex never retrives
content without first checking if it's present AFAICR. And really this will
only be needed when using fsck. Puttting it here, rather than in fsck
avoids breaking an abstraction boundary, and is nice and inexpensive.
diff --git a/Remote/Helper/Chunked.hs b/Remote/Helper/Chunked.hs
index 22030fb39..2ef836018 100644
--- a/Remote/Helper/Chunked.hs
+++ b/Remote/Helper/Chunked.hs
@@ -206,7 +206,7 @@ seekResume h encryptor chunkkeys checker = do
  -}
 removeChunks :: Remover -> UUID -> ChunkConfig -> EncKey -> Key -> Annex ()
 removeChunks remover u chunkconfig encryptor k = do
-	ls <- chunkKeys u chunkconfig k
+	ls <- map chunkKeyList <$> chunkKeys u chunkconfig k
 	mapM_ (remover . encryptor) (concat ls)
 	let chunksizes = catMaybes $ map (fromKey keyChunkSize <=< headMaybe) ls
 	forM_ chunksizes $ chunksRemoved u k . FixedSizeChunks . fromIntegral
@@ -245,7 +245,8 @@ retrieveChunks retriever u chunkconfig encryptor basek dest basep sink
 			(\e -> go (Just e) =<< chunkKeysOnly u chunkconfig basek)
 	| otherwise = go Nothing =<< chunkKeys u chunkconfig basek
   where
-	go pe ls = do
+	go pe cks = do
+		let ls = map chunkKeyList cks
 		currsize <- liftIO $ catchMaybeIO $ getFileSize dest
 		let ls' = maybe ls (setupResume ls) currsize
 		if any null ls'
@@ -359,14 +360,18 @@ checkPresentChunks checker u chunkconfig encryptor basek
   where
 	checklists Nothing [] = return False
 	checklists (Just deferrederror) [] = throwM deferrederror
-	checklists d (l:ls)
+	checklists d (ck:cks)
 		| not (null l) = do
 			v <- checkchunks l
 			case v of
-				Left e -> checklists (Just e) ls
-				Right True -> return True
-				Right False -> checklists Nothing ls
-		| otherwise = checklists d ls
+				Left e -> checklists (Just e) cks
+				Right True -> do
+					ensureChunksAreLogged u basek ck
+					return True
+				Right False -> checklists Nothing cks
+		| otherwise = checklists d cks
+	  where
+		l = chunkKeyList ck
 	
 	checkchunks :: [Key] -> Annex (Either SomeException Bool)
 	checkchunks [] = return (Right True)
@@ -379,6 +384,14 @@ checkPresentChunks checker u chunkconfig encryptor basek
 
 	check = tryNonAsync . checker . encryptor
 
+data ChunkKeys
+	= ChunkKeys [Key]
+	| SpeculativeChunkKeys (ChunkMethod, ChunkCount) [Key]
+
+chunkKeyList :: ChunkKeys -> [Key]
+chunkKeyList (ChunkKeys l) = l
+chunkKeyList (SpeculativeChunkKeys _ l) = l
+
 {- A key can be stored in a remote unchunked, or as a list of chunked keys.
  - This can be the case whether or not the remote is currently configured
  - to use chunking.
@@ -395,22 +408,22 @@ checkPresentChunks checker u chunkconfig encryptor basek
  - recover from data loss, where the chunk log didn't make it out,
  - though only as long as the ChunkConfig is unchanged.
  -}
-chunkKeys :: UUID -> ChunkConfig -> Key -> Annex [[Key]]
+chunkKeys :: UUID -> ChunkConfig -> Key -> Annex [ChunkKeys]
 chunkKeys = chunkKeys' False
 
 {- Same as chunkKeys, but excluding the unchunked key. -}
-chunkKeysOnly :: UUID -> ChunkConfig -> Key -> Annex [[Key]]
+chunkKeysOnly :: UUID -> ChunkConfig -> Key -> Annex [ChunkKeys]
 chunkKeysOnly = chunkKeys' True
 
-chunkKeys' :: Bool -> UUID -> ChunkConfig -> Key -> Annex [[Key]]
+chunkKeys' :: Bool -> UUID -> ChunkConfig -> Key -> Annex [ChunkKeys]
 chunkKeys' onlychunks u chunkconfig k = do
 	recorded <- getCurrentChunks u k
-	let recordedl  = map (toChunkList k) recorded
+	let recordedl = map (ChunkKeys . toChunkList k) recorded
 	return $ addspeculative recorded $ if onlychunks
 		then recordedl
 		else if noChunks chunkconfig
-			then [k] : recordedl
-			else recordedl ++ [[k]]
+			then ChunkKeys [k] : recordedl
+			else recordedl ++ [ChunkKeys [k]]
   where
 	addspeculative recorded l = case chunkconfig of
 		NoChunks -> l
@@ -422,10 +435,19 @@ chunkKeys' onlychunks u chunkconfig k = do
  				    v = (FixedSizeChunks chunksz, chunkcount)
 				in if v `elem` recorded
 					then l
-					else l ++ [toChunkList k v]
+					else l ++ [SpeculativeChunkKeys v (toChunkList k v)]
 		LegacyChunks _ -> l
 
 toChunkList :: Key -> (ChunkMethod, ChunkCount) -> [Key]
 toChunkList k (FixedSizeChunks chunksize, chunkcount) =
 	takeChunkKeyStream chunkcount $ chunkKeyStream k chunksize
 toChunkList _ (UnknownChunks _, _) = []
+
+{- When chunkKeys provided a speculative chunk list, and that has been
+ - verified to be present, use this to log it in the chunk log. This way,
+ - a later change to the chunk size of the remote won't prevent accessing
+ - the chunks. -}
+ensureChunksAreLogged :: UUID -> Key -> ChunkKeys -> Annex ()
+ensureChunksAreLogged u k (SpeculativeChunkKeys (chunkmethod, chunkcount) _) = 
+	chunksStored u k chunkmethod chunkcount
+ensureChunksAreLogged _ _ (ChunkKeys _) = return ()
diff --git a/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks.mdwn b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks.mdwn
index 9f4fc444b..4bb67de23 100644
--- a/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks.mdwn
+++ b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks.mdwn
@@ -24,3 +24,5 @@ but `annex fsck --key KEY --from remote --fast`, since it doesn't have an exact
 
 [[!meta author=yoh]]
 [[!tag projects/dandi]]
+
+> [[done]] --[[Joey]]

speculatively use remote's configured chunk size as a fallback
When a special remote has chunking enabled, but no chunk sizes are
recorded (or the recorded ones are not found), speculatively try chunks
using the configured chunk size.
This makes eg, git-annex fsck --from remote be able to fix up the
location log of a file that the git-annex branch does not indicate is
stored on the remote.
Note that fsck does *not* fix up the chunk log to indicate the chunk
size. So, changing the chunk config of the remote after that will still
prevent accessing the chunks stored on it. Maybe fsck should, but I
wanted to start with this and see if it's needed.
diff --git a/CHANGELOG b/CHANGELOG
index 28c787cc1..0d585c1ae 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,9 @@ git-annex (8.20201008) UNRELEASED; urgency=medium
   * move: Improve resuming a move that was interrupted after the object
     was transferred, in cases where numcopies checks prevented the resumed
     move from dropping the object from the source repository.
+  * When a special remote has chunking enabled, but no chunk sizes are
+    recorded (or the recorded ones are not found), speculatively try
+    chunks using the configured chunk size.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Oct 2020 10:48:17 -0400
 
diff --git a/Remote/Helper/Chunked.hs b/Remote/Helper/Chunked.hs
index b90e69f82..22030fb39 100644
--- a/Remote/Helper/Chunked.hs
+++ b/Remote/Helper/Chunked.hs
@@ -242,7 +242,7 @@ retrieveChunks retriever u chunkconfig encryptor basek dest basep sink
 		-- looking in the git-annex branch for chunk counts
 		-- that are likely not there.
 		getunchunked `catchNonAsync`
-			(\e -> go (Just e) =<< chunkKeysOnly u basek)
+			(\e -> go (Just e) =<< chunkKeysOnly u chunkconfig basek)
 	| otherwise = go Nothing =<< chunkKeys u chunkconfig basek
   where
 	go pe ls = do
@@ -350,10 +350,11 @@ checkPresentChunks checker u chunkconfig encryptor basek
 		-- looking in the git-annex branch for chunk counts
 		-- that are likely not there.
 		v <- check basek
+		let getchunkkeys = chunkKeysOnly u chunkconfig basek
 		case v of
 			Right True -> return True
-			Left e -> checklists (Just e) =<< chunkKeysOnly u basek
-			_ -> checklists Nothing =<< chunkKeysOnly u basek
+			Left e -> checklists (Just e) =<< getchunkkeys
+			_ -> checklists Nothing =<< getchunkkeys
 	| otherwise = checklists Nothing =<< chunkKeys u chunkconfig basek
   where
 	checklists Nothing [] = return False
@@ -388,16 +389,41 @@ checkPresentChunks checker u chunkconfig encryptor basek
  - This finds all possible lists of keys that might be on the remote that
  - can be combined to get back the requested key, in order from most to
  - least likely to exist.
+ -
+ - Speculatively tries chunks using the ChunkConfig last of all
+ - (when that's not the same as the recorded chunks). This can help
+ - recover from data loss, where the chunk log didn't make it out,
+ - though only as long as the ChunkConfig is unchanged.
  -}
 chunkKeys :: UUID -> ChunkConfig -> Key -> Annex [[Key]]
-chunkKeys u chunkconfig k = do
-	l <- chunkKeysOnly u k
-	return $ if noChunks chunkconfig
-		then [k] : l
-		else l ++ [[k]]
-
-chunkKeysOnly :: UUID -> Key -> Annex [[Key]]
-chunkKeysOnly u k = map (toChunkList k) <$> getCurrentChunks u k
+chunkKeys = chunkKeys' False
+
+{- Same as chunkKeys, but excluding the unchunked key. -}
+chunkKeysOnly :: UUID -> ChunkConfig -> Key -> Annex [[Key]]
+chunkKeysOnly = chunkKeys' True
+
+chunkKeys' :: Bool -> UUID -> ChunkConfig -> Key -> Annex [[Key]]
+chunkKeys' onlychunks u chunkconfig k = do
+	recorded <- getCurrentChunks u k
+	let recordedl  = map (toChunkList k) recorded
+	return $ addspeculative recorded $ if onlychunks
+		then recordedl
+		else if noChunks chunkconfig
+			then [k] : recordedl
+			else recordedl ++ [[k]]
+  where
+	addspeculative recorded l = case chunkconfig of
+		NoChunks -> l
+		UnpaddedChunks chunksz -> case fromKey keySize k of
+			Nothing -> l
+			Just keysz -> 
+				let (d, m) = keysz `divMod` fromIntegral chunksz
+				    chunkcount = d + if m == 0 then 0 else 1
+ 				    v = (FixedSizeChunks chunksz, chunkcount)
+				in if v `elem` recorded
+					then l
+					else l ++ [toChunkList k v]
+		LegacyChunks _ -> l
 
 toChunkList :: Key -> (ChunkMethod, ChunkCount) -> [Key]
 toChunkList k (FixedSizeChunks chunksize, chunkcount) =
diff --git a/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_1_5d11bb90105b4d0cf14f85e6d8795023._comment b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_1_5d11bb90105b4d0cf14f85e6d8795023._comment
new file mode 100644
index 000000000..5fc15354d
--- /dev/null
+++ b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_1_5d11bb90105b4d0cf14f85e6d8795023._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-10-22T16:09:17Z"
+ content="""
+Note that what you are trying to do will only work if the special remote
+is not encrypted.
+
+As well as your use case, which seems very unusual, I think one other use
+case would be if a clone uploaded to the special remote, but never synced
+out its git-annex branch before being lost, and fsck --from
+remote is being run in another clone to reconstruct it. Currently it
+won't try chunks as none are recorded.
+
+Speculatively trying the current remote's chunk config would handle the
+majority of cases, though wouldn't help if the other clone had adjusted the
+special remote's chunk size too.
+
+There's some overhead, but it can check it last, and not check it if
+it's in the list of known chunks, so the overhead would only usually 
+be paid if the content git-annex expected to be present had gone missing,
+which I think is rare enough to not care about.
+
+(Also, this can only be done when the size of the key is known, so not
+eg addurl --relaxed keys.)
+"""]]
diff --git a/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_2_f48e4d3571a8fc2b45fbaad7a9f8fb60._comment b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_2_f48e4d3571a8fc2b45fbaad7a9f8fb60._comment
new file mode 100644
index 000000000..5924108ca
--- /dev/null
+++ b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks/comment_2_f48e4d3571a8fc2b45fbaad7a9f8fb60._comment
@@ -0,0 +1,29 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-10-22T17:00:25Z"
+ content="""
+Implemented that. But..
+
+As implemented, there's nothing to make the chunk size get stored in the
+chunk log for a key, after it accesses its content using the configured
+chunk size.
+
+So, changing the chunk= of the remote can prevent accessing content that
+was accessible before. Of course, avoiding that is why chunk sizes are
+logged in the first place.
+
+Seems like maybe fsck --from should fix the chunk log? I think
+fsck would always need to be used, to fix up the location log, before any
+other commands rely on the data being in the special remote, so it seems
+fine to only fix the chunk log there.
+
+But, also a bit unclear how fsck would find out when it needs to do this.
+It only needs to when the remote's configured chunk size is not
+listed in the chunk log. But that's also common after changing the chunk
+size of a remote. So it would have to mess around with checking the
+presence of chunk keys itself, which would be extra work and also ugly
+to implement.
+
+I'm leaving this todo^Wbug open for now due to this.
+"""]]

close
diff --git a/doc/bugs/windows_dosn__39__t_build_again__63____33__.mdwn b/doc/bugs/windows_dosn__39__t_build_again__63____33__.mdwn
index b1338ba8d..c46fffd3f 100644
--- a/doc/bugs/windows_dosn__39__t_build_again__63____33__.mdwn
+++ b/doc/bugs/windows_dosn__39__t_build_again__63____33__.mdwn
@@ -20,3 +20,7 @@ Seems like the Windows version is not updated?
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 - use it to ensure some of my private data / files are save on 2x different media's :)
 
+> The windows autobuilder is currenntly broken and has to be periodically
+> rebooted or something to be "fixed". So updated builds are sporadic. 
+> This is not due to a bug in git-annex, so I don't really want to track
+> it here. [[done]] --[[Joey]]

Added a comment: cannot find this mentioned Windows build :|
diff --git a/doc/news/version_8.20201007/comment_1_0612ed13b7b27721e41ea3fb65b0e265._comment b/doc/news/version_8.20201007/comment_1_0612ed13b7b27721e41ea3fb65b0e265._comment
new file mode 100644
index 000000000..15c818918
--- /dev/null
+++ b/doc/news/version_8.20201007/comment_1_0612ed13b7b27721e41ea3fb65b0e265._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="nix.zahlen@1211ac6c964ba2d68b70655f747bef1383032e77"
+ nickname="nix.zahlen"
+ avatar="http://cdn.libravatar.org/avatar/66cc45a05749fe8d4ca36d8c6071da51"
+ subject="cannot find this mentioned Windows build :|"
+ date="2020-10-21T20:46:37Z"
+ content="""
+seems like the offered Windows installer is still on \"distributionVersion = \"8.20200815\"\"
+"""]]

diff --git a/doc/bugs/windows_dosn__39__t_build_again__63____33__.mdwn b/doc/bugs/windows_dosn__39__t_build_again__63____33__.mdwn
new file mode 100644
index 000000000..b1338ba8d
--- /dev/null
+++ b/doc/bugs/windows_dosn__39__t_build_again__63____33__.mdwn
@@ -0,0 +1,22 @@
+### Please describe the problem.
+Seems like the Windows version is not updated?
+
+1. https://downloads.kitenet.net/git-annex/windows/current/git-annex-installer.exe.info ==> distributionVersion = "8.20200815"
+2. https://git-annex.branchable.com/news/version_8.20201007/#news-version-8.20201007.default ==> 8.20201007 ==> "Fix a build failure on Windows" ==> so where is that version / installer?
+
+### What version of git-annex are you using? On what operating system?
+- git-annex version: 8.20200815-g335aae266
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+- use it to ensure some of my private data / files are save on 2x different media's :)
+

Fix a broken link
diff --git a/doc/devblog/day_633__ten_years.mdwn b/doc/devblog/day_633__ten_years.mdwn
index 673af5cc5..a74e96a69 100644
--- a/doc/devblog/day_633__ten_years.mdwn
+++ b/doc/devblog/day_633__ten_years.mdwn
@@ -4,7 +4,7 @@ had the commands add, get, drop, unannex, init, fix, and fromkey.
 There were 2600 lines of code in all, which has increased 30-fold.
 
 Later that week, 0.02 added the move command. I've been working recently
-on fixing a [[tricky_problem|doc/todo/interruped_move_resume]] with
+on fixing a [[tricky_problem|todo/interruped_move_resume]] with
 resuming interrupted moves. The move command has been surprisingly
 subtle to get just right, since it turns out to not be as simple as a get
 followed by a drop -- numcopies checks that would prevent a drop sometimes

Added a comment
diff --git a/doc/forum/PDF-related_error_when_running_git-annex-add_or_git-annex-status/comment_1_0fefd3bb7646eedfa6bfcd1b5a05e181._comment b/doc/forum/PDF-related_error_when_running_git-annex-add_or_git-annex-status/comment_1_0fefd3bb7646eedfa6bfcd1b5a05e181._comment
new file mode 100644
index 000000000..99e583797
--- /dev/null
+++ b/doc/forum/PDF-related_error_when_running_git-annex-add_or_git-annex-status/comment_1_0fefd3bb7646eedfa6bfcd1b5a05e181._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Lukey"
+ avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
+ subject="comment 1"
+ date="2020-10-21T19:10:01Z"
+ content="""
+Git on windows configures a diff driver by default, which is probably throwing these errors (For whathever reason... *sigh*). It's the `diff.<driver>.command` setting that you have to unset.
+"""]]

diff --git a/doc/forum/PDF-related_error_when_running_git-annex-add_or_git-annex-status.mdwn b/doc/forum/PDF-related_error_when_running_git-annex-add_or_git-annex-status.mdwn
new file mode 100644
index 000000000..020016851
--- /dev/null
+++ b/doc/forum/PDF-related_error_when_running_git-annex-add_or_git-annex-status.mdwn
@@ -0,0 +1,15 @@
+When I run either "git annex add" or "git annex status" on a repository with PDF files, I'm getting the following errors:
+
+    Syntax Warning: May not be a PDF file (continuing anyway)
+    Syntax Error: Couldn't read xref table
+    Syntax Warning: PDF file is damaged - attempting to reconstruct xref table...
+    Syntax Error: Couldn't find trailer dictionary
+    Syntax Error: Couldn't read xref table
+
+I checked the PDFs and I can open them.
+
+What is triggerring these errors? Frequently these errors freezes the git-annex process and I have no choice but to kill one by one all the spawned git processes.
+
+Again, I'm in Windows 7, using Git 2.28.0.windows.1 and git-annex 8.20200815-g335aae266 on a newly-created repository in an NTFS partition.
+
+

devblog
diff --git a/doc/devblog/day_633__ten_years.mdwn b/doc/devblog/day_633__ten_years.mdwn
new file mode 100644
index 000000000..673af5cc5
--- /dev/null
+++ b/doc/devblog/day_633__ten_years.mdwn
@@ -0,0 +1,12 @@
+The first prerelease of git-annex was made ten years ago.
+Taking a look back at 0.01, which I think only I ever used, it
+had the commands add, get, drop, unannex, init, fix, and fromkey.
+There were 2600 lines of code in all, which has increased 30-fold.
+
+Later that week, 0.02 added the move command. I've been working recently
+on fixing a [[tricky_problem|doc/todo/interruped_move_resume]] with
+resuming interrupted moves. The move command has been surprisingly
+subtle to get just right, since it turns out to not be as simple as a get
+followed by a drop -- numcopies checks that would prevent a drop sometimes
+need to be relaxed to allow a move. Maybe I've finally gotten it perfect
+now. Probably not.

move: Improve resuming a move that was interrupted after the object was transferred
In cases where numcopies checks prevented the resumed move from dropping
the object from the source repository, it now relies on a log of recent
moves to replicate the behavior of the interrupted command.
Performance: Probably noticable impact, since it has to add to the log,
check the log, and remove from the log. Seems worth it to avoid this
annoying edge case. The log functions are pretty well optimised to avoid
unncessary work.
An performance improvement to make later would be to avoid cleanup doing
anything if it's not written to the log file, and has confirmed that the
log file does not contain the log line.
This commit was sponsored by Jake Vosloo on Patreon.
diff --git a/Annex/Locations.hs b/Annex/Locations.hs
index fdff25b61..dfdce58d7 100644
--- a/Annex/Locations.hs
+++ b/Annex/Locations.hs
@@ -47,6 +47,8 @@ module Annex.Locations (
 	gitAnnexFsckResultsLog,
 	gitAnnexSmudgeLog,
 	gitAnnexSmudgeLock,
+	gitAnnexMoveLog,
+	gitAnnexMoveLock,
 	gitAnnexExportDir,
 	gitAnnexExportDbDir,
 	gitAnnexExportLock,
@@ -377,6 +379,14 @@ gitAnnexSmudgeLog r = fromRawFilePath $ gitAnnexDir r P.</> "smudge.log"
 gitAnnexSmudgeLock :: Git.Repo -> FilePath
 gitAnnexSmudgeLock r = fromRawFilePath $ gitAnnexDir r P.</> "smudge.lck"
 
+{- .git/annex/move.log is used to log moves that are in progress,
+ - to better support resuming an interrupted move. -}
+gitAnnexMoveLog :: Git.Repo -> FilePath
+gitAnnexMoveLog r = fromRawFilePath $ gitAnnexDir r P.</> "move.log"
+
+gitAnnexMoveLock :: Git.Repo -> FilePath
+gitAnnexMoveLock r = fromRawFilePath $ gitAnnexDir r P.</> "move.lck"
+
 {- .git/annex/export/ is used to store information about
  - exports to special remotes. -}
 gitAnnexExportDir :: Git.Repo -> FilePath
diff --git a/Assistant.hs b/Assistant.hs
index 01fdb80fe..6db2acf46 100644
--- a/Assistant.hs
+++ b/Assistant.hs
@@ -74,7 +74,7 @@ startDaemon assistant foreground startdelay cannotrun listenhost startbrowser =
 	Annex.changeState $ \s -> s { Annex.daemon = True }
 	enableInteractiveBranchAccess
 	pidfile <- fromRepo gitAnnexPidFile
-	logfile <- fromRepo gitAnnexLogFile
+	logfile <- fromRepo gitAnnexDaemonLogFile
 	liftIO $ debugM desc $ "logging to " ++ logfile
 	createAnnexDirectory (parentDir pidfile)
 #ifndef mingw32_HOST_OS
@@ -127,7 +127,7 @@ startDaemon assistant foreground startdelay cannotrun listenhost startbrowser =
 	start daemonize webappwaiter = withThreadState $ \st -> do
 		checkCanWatch
 		dstatus <- startDaemonStatus
-		logfile <- fromRepo gitAnnexLogFile
+		logfile <- fromRepo gitAnnexDaemonLogFile
 		liftIO $ debugM desc $ "logging to " ++ logfile
 		liftIO $ daemonize $
 			flip runAssistant (go webappwaiter) 
diff --git a/CHANGELOG b/CHANGELOG
index 70c4c406b..28c787cc1 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -4,6 +4,9 @@ git-annex (8.20201008) UNRELEASED; urgency=medium
   * Fix a memory leak introduced in the last release.
   * add, import: Fix a reversion in 7.20191009 that broke handling
     of --largerthan and --smallerthan.
+  * move: Improve resuming a move that was interrupted after the object
+    was transferred, in cases where numcopies checks prevented the resumed
+    move from dropping the object from the source repository.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Oct 2020 10:48:17 -0400
 
diff --git a/Command/Move.hs b/Command/Move.hs
index 35cbb9801..cfab0725a 100644
--- a/Command/Move.hs
+++ b/Command/Move.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2010-2018 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -16,9 +16,12 @@ import Annex.UUID
 import Annex.Transfer
 import Logs.Presence
 import Logs.Trust
+import Logs.File
 import Annex.NumCopies
 
 import System.Log.Logger (debugM)
+import qualified Data.ByteString.Char8 as B8
+import qualified Data.ByteString.Lazy as L
 
 cmd :: Command
 cmd = withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption, annexedMatchingOptions] $
@@ -130,34 +133,36 @@ expectedPresent dest key = do
 	return $ dest `elem` remotes
 
 toPerform :: Remote -> RemoveWhen -> Key -> AssociatedFile -> Bool -> Either String Bool -> CommandPerform
-toPerform dest removewhen key afile fastcheck isthere =
+toPerform dest removewhen key afile fastcheck isthere = do
+	srcuuid <- getUUID
 	case isthere of
 		Left err -> do
 			showNote err
 			stop
-		Right False -> do
+		Right False -> logMove srcuuid destuuid False key $ \deststartedwithcopy -> do
 			showAction $ "to " ++ Remote.name dest
 			ok <- notifyTransfer Upload afile $
 				upload (Remote.uuid dest) key afile stdRetry $
 					Remote.action . Remote.storeKey dest key afile
 			if ok
-				then finish False $
+				then finish deststartedwithcopy $
 					Remote.logStatus dest key InfoPresent
 				else do
 					when fastcheck $
 						warning "This could have failed because --fast is enabled."
 					stop
-		Right True -> finish True $
-			unlessM (expectedPresent dest key) $
-				Remote.logStatus dest key InfoPresent
+		Right True -> logMove srcuuid destuuid False key $ \deststartedwithcopy ->
+			finish deststartedwithcopy $
+				unlessM (expectedPresent dest key) $
+					Remote.logStatus dest key InfoPresent
   where
+	destuuid = Remote.uuid dest
 	finish deststartedwithcopy setpresentremote = case removewhen of
 		RemoveNever -> do
 			setpresentremote
 			next $ return True
 		RemoveSafe -> lockContentForRemoval key lockfailed $ \contentlock -> do
 			srcuuid <- getUUID
-			let destuuid = Remote.uuid dest
 			willDropMakeItWorse srcuuid destuuid deststartedwithcopy key afile >>= \case
 				DropAllowed -> drophere setpresentremote contentlock "moved"
 				DropCheckNumCopies -> do
@@ -211,19 +216,21 @@ fromOk src key = do
 fromPerform :: Remote -> RemoveWhen -> Key -> AssociatedFile -> CommandPerform
 fromPerform src removewhen key afile = do
 	showAction $ "from " ++ Remote.name src
-	ifM (inAnnex key)
-		( dispatch removewhen True True
-		, dispatch removewhen False =<< go
-		)
+	present <- inAnnex key
+	destuuid <- getUUID
+	logMove srcuuid destuuid present key $ \deststartedwithcopy ->
+		if present
+			then dispatch removewhen deststartedwithcopy True
+			else dispatch removewhen deststartedwithcopy =<< get
   where
-	go = notifyTransfer Download afile $ 
+	get = notifyTransfer Download afile $ 
 		download (Remote.uuid src) key afile stdRetry $ \p ->
 			getViaTmp (Remote.retrievalSecurityPolicy src) (RemoteVerify src) key $ \t ->
 				Remote.verifiedAction $ Remote.retrieveKeyFile src key afile t p
+	
 	dispatch _ _ False = stop -- failed
 	dispatch RemoveNever _ True = next $ return True -- copy complete
 	dispatch RemoveSafe deststartedwithcopy True = lockContentShared key $ \_lck -> do
-		let srcuuid = Remote.uuid src
 		destuuid <- getUUID
 		willDropMakeItWorse srcuuid destuuid deststartedwithcopy key afile >>= \case
 			DropAllowed -> dropremote "moved"
@@ -233,7 +240,11 @@ fromPerform src removewhen key afile = do
 				verifyEnoughCopiesToDrop "" key Nothing numcopies [Remote.uuid src] verified
 					tocheck (dropremote . showproof) faileddropremote
 			DropWorse -> faileddropremote		
+	
+	srcuuid = Remote.uuid src
+	
 	showproof proof = "proof: " ++ show proof
+	
 	dropremote reason = do
 		liftIO $ debugM "move" $ unwords
 			[ "Dropping from remote"
@@ -242,6 +253,7 @@ fromPerform src removewhen key afile = do
 			]
 		ok <- Remote.action (Remote.removeKey src key)
 		next $ Command.Drop.cleanupRemote key src ok
+	
 	faileddropremote = do
 		showLongNote "(Use --force to override this check, or adjust numcopies.)"
 		showLongNote $ "Content not dropped from " ++ Remote.name src ++ "."
@@ -287,8 +299,8 @@ toHereStart removewhen afile key ai si =
  - This function checks all that. It needs to know if the destination
  - repository already had a copy of the file before the move began.
  -}
-willDropMakeItWorse :: UUID -> UUID -> Bool -> Key -> AssociatedFile -> Annex DropCheck
-willDropMakeItWorse srcuuid destuuid deststartedwithcopy key afile =
+willDropMakeItWorse :: UUID -> UUID -> DestStartedWithCopy -> Key -> AssociatedFile -> Annex DropCheck
+willDropMakeItWorse srcuuid destuuid (DestStartedWithCopy deststartedwithcopy) key afile =
 	ifM (Command.Drop.checkRequiredContent srcuuid key afile)
 		( if deststartedwithcopy
 			then unlessforced DropCheckNumCopies
@@ -309,3 +321,53 @@ willDropMakeItWorse srcuuid destuuid deststartedwithcopy key afile =
 		return (desttrust > UnTrusted || desttrust >= srctrust)
 
 data DropCheck = DropWorse | DropAllowed | DropCheckNumCopies
+
+newtype DestStartedWithCopy = DestStartedWithCopy Bool
+
+{- Runs an action that performs a move, and logs the move, allowing an

(Diff truncated)
removed
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_2_7de900536720d31f81afcf0a9bca91ae._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_2_7de900536720d31f81afcf0a9bca91ae._comment
deleted file mode 100644
index 44cfa2388..000000000
--- a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_2_7de900536720d31f81afcf0a9bca91ae._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="achilleas.k@14be77d42a1252fab5ec9dbf4e5ea03c5833e8c8"
- avatar="http://cdn.libravatar.org/avatar/ed6c67c4d8e6c6850930e16eaf85a771"
- subject="comment 2"
- date="2020-10-21T07:51:13Z"
- content="""
-Thanks for the minimal reproducible example!
-"""]]

Added a comment
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_fd3a287ee6e73946889a4dd406fae0fc._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_fd3a287ee6e73946889a4dd406fae0fc._comment
new file mode 100644
index 000000000..0b12ba684
--- /dev/null
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_3_fd3a287ee6e73946889a4dd406fae0fc._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="achilleas.k@14be77d42a1252fab5ec9dbf4e5ea03c5833e8c8"
+ avatar="http://cdn.libravatar.org/avatar/ed6c67c4d8e6c6850930e16eaf85a771"
+ subject="comment 3"
+ date="2020-10-21T07:51:37Z"
+ content="""
+Great!  Thanks for checking and for the minimal reproducible example!
+"""]]

Added a comment
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_2_7de900536720d31f81afcf0a9bca91ae._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_2_7de900536720d31f81afcf0a9bca91ae._comment
new file mode 100644
index 000000000..44cfa2388
--- /dev/null
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_2_7de900536720d31f81afcf0a9bca91ae._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="achilleas.k@14be77d42a1252fab5ec9dbf4e5ea03c5833e8c8"
+ avatar="http://cdn.libravatar.org/avatar/ed6c67c4d8e6c6850930e16eaf85a771"
+ subject="comment 2"
+ date="2020-10-21T07:51:13Z"
+ content="""
+Thanks for the minimal reproducible example!
+"""]]

report/idea on minting chunked keys to try from special remote
diff --git a/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks.mdwn b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks.mdwn
new file mode 100644
index 000000000..9f4fc444b
--- /dev/null
+++ b/doc/bugs/fsck_--key_without___34__chunking__34___information_in_git-annex_does_not_try_chunks.mdwn
@@ -0,0 +1,26 @@
+### Please describe the problem.
+
+Probably it is more of a todo than a bug.
+
+### What steps will reproduce the problem?
+
+This is a use-case where I am trying to establish a special remote to be shared by multiple unrelated repositories.
+
+So I had original repo1 in which I
+
+- created an external special remote with chunking, it got UUID1
+- uploaded some data (all got chunked)
+
+created repo2 in which I
+
+- initialized special remote with identical settings and provided `uuid=UUID1`
+- decided to test if annex would be able to get a key from the shared special remote
+
+but `annex fsck --key KEY --from remote --fast`, since it doesn't have an exact chunking list, just provides special remote backend with original full key only, which is obviously not found, and it reports failure.   But I wondered -- couldn't `git-annex` just use chunking size and "mint" possible chunked-keys to test on the special remote since it has all the information?  After all chunk keys AFAIK are deterministically minted and pretty much are just "augmented" original key with `-S<chunksize>-C<chunkindex>` added to the key.
+
+### What version of git-annex are you using? On what operating system?
+
+8.20200908+git175-g95d02d6e2-1~ndall+1
+
+[[!meta author=yoh]]
+[[!tag projects/dandi]]

Added a comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_7_644826ed86849eb4babf37bea3937a7c._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_7_644826ed86849eb4babf37bea3937a7c._comment
new file mode 100644
index 000000000..b15f51beb
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_7_644826ed86849eb4babf37bea3937a7c._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 7"
+ date="2020-10-20T16:39:15Z"
+ content="""
+oh, just now realized that `git-annex.exe` is in **exactly** the same `C:\Program Files (x86)\Git\usr\bin` as `cp` etc -- so specifying specific path of where `git-annex.exe` installed (I bet the process would know which dir it is coming from) to `cp` and all those tools for a windows build could potentially make those installations more robust overall.
+"""]]

Added a comment
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_1_8a6cd8dc6e7c2cdefd60a07356be1d27._comment b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_1_8a6cd8dc6e7c2cdefd60a07356be1d27._comment
new file mode 100644
index 000000000..6e21268bb
--- /dev/null
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__/comment_1_8a6cd8dc6e7c2cdefd60a07356be1d27._comment
@@ -0,0 +1,37 @@
+[[!comment format=mdwn
+ username="kyle"
+ avatar="http://cdn.libravatar.org/avatar/7d6e85cde1422ad60607c87fa87c63f3"
+ subject="comment 1"
+ date="2020-10-20T15:59:26Z"
+ content="""
+I can trigger this issue on my end too (tried with
+7.20190503-g4da50456a and 8.20201008-gad86a25c5).  In case it's
+helpful, here is a minimal reproducer:
+
+[[!format sh \"\"\"
+set -ue
+
+cd \"$(mktemp -d \"${TMPDIR:-/tmp}\"/gx-XXXXXXX)\"
+
+git annex version
+pwd
+
+git init src
+(
+    cd src
+    git init sub
+    git -C sub annex init
+    git -C sub commit --allow-empty -m sub-c0
+    git submodule add ./sub
+    git commit -m sub
+)
+
+git clone --recursive src dest
+(
+    cd dest
+    git annex init
+    git submodule foreach 'git annex init'
+)
+\"\"\"]]
+
+"""]]

Added a comment
diff --git a/doc/forum/Matching_options_not_working_in_windows_7/comment_3_d834a6cb0ae1bd47e5280b10ecbaa72d._comment b/doc/forum/Matching_options_not_working_in_windows_7/comment_3_d834a6cb0ae1bd47e5280b10ecbaa72d._comment
new file mode 100644
index 000000000..39e329445
--- /dev/null
+++ b/doc/forum/Matching_options_not_working_in_windows_7/comment_3_d834a6cb0ae1bd47e5280b10ecbaa72d._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="ericm"
+ avatar="http://cdn.libravatar.org/avatar/67ca64c5a99fb142fc2e3916333881ca"
+ subject="comment 3"
+ date="2020-10-20T13:45:09Z"
+ content="""
+This worked for me:
+
+    git -c annex.largefiles=largerthan=200mb -c annex.addsmallfiles=false annex add --backend=WORM .
+
+Then this adds the remaining files:
+
+    git annex add .
+
+Thanks for the help!
+"""]]

New bug report
diff --git a/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__.mdwn b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__.mdwn
new file mode 100644
index 000000000..c79dcae13
--- /dev/null
+++ b/doc/bugs/__34__git_submodule_foreach_git_annex_init__34___fails_with___34__git__58___createProcess__58___runInteractiveProcess__58___chdir__58___inappropriate_type___40__Not_a_directory__41____34__.mdwn
@@ -0,0 +1,30 @@
+### Please describe the problem.
+
+In a repository with submodules, if I want to initialise git-annex on all submodules, I expect that `git submodule foreach git annex init` should work.  Other `git-annex` commands seem to work with the `submodule foreach` command just fine.
+
+
+### What steps will reproduce the problem?
+
+ 1. Clone a repository with submodules (recursively): `git clone --recursive <remote>`
+ 2. Initialise the main repository's annex: `git annex init`
+ 3. Try to initialise git-annex in all submodules: `git submodule foreach git annex init`
+
+This produces the following error:
+```
+git-annex: git: createProcess: runInteractiveProcess: chdir: inappropriate type (Not a directory)
+fatal: run_command returned non-zero status for scripts
+.
+```
+
+As a workaround, manually changing directory into each submodule and running `git annex init` works, but there are cases where having a general script that can iterate submodules is convenient.
+
+
+### What version of git-annex are you using? On what operating system?
+
+- git-annex version: 8.20201007-gcf33be21ac
+- OS: Arch Linux
+
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+I use it regularly with great success :)

Added a comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_6_db5de257e8cfb468bf31d46065b7d251._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_6_db5de257e8cfb468bf31d46065b7d251._comment
new file mode 100644
index 000000000..3ef5e305e
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_6_db5de257e8cfb468bf31d46065b7d251._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 6"
+ date="2020-10-20T12:33:29Z"
+ content="""
+> --reflink seems unlikely to do anything useful on windows even if the option is supported, so it could just be defaulted to not supported on windows. 
+
+yeap, sounds good to me ATM.
+
+> -a/-p/--preserve-timestamps are more important and also probed at build time.
+
+If I read NEWS correctly, it was added in 4.1.10, so old enough (was I even born then?) ;) 
+"""]]

Added a comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_5_94d4cc9baf46fbc09c21abd08b441b0c._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_5_94d4cc9baf46fbc09c21abd08b441b0c._comment
new file mode 100644
index 000000000..37bfbc60b
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_5_94d4cc9baf46fbc09c21abd08b441b0c._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 5"
+ date="2020-10-20T12:29:19Z"
+ content="""
+It isn't exactly \"how git-annex installed\" but rather, what else is installed besides git-annex.  Recommended way to install datalad on OSX and Windows is via [anaconda](https://www.anaconda.com/) or [miniconda](https://docs.conda.io/en/latest/miniconda.html) from conda-forge channel.  See  So it places itself into `PATH` ahead of the rest, and it is the one which comes ATM with outdated `cp`.
+[It seems](https://github.com/conda-forge/coreutils-feedstock/issues/8) that `cp` might come also with `git` package there, and I have initiated (but didn't progress) [effort](https://github.com/conda-forge/coreutils-feedstock/issues/8) to build newer coreutils for Windows for conda-forge. (there is also some other m2sys channel with somewhat newer coreutils build, but mixing channels that much is begging for more of other issues).  So, eventually, I hope that there would be updated `cp` available by default with `datalad` installation, but the point is that unless git-annex uses the specific `cp` it knows about, it could be some other `cp` in its way.
+"""]]

add, import: Fix a reversion in 7.20191009 that broke handling of --largerthan and --smallerthan
This commit was sponsored by Jochen Bartl on Patreon.
diff --git a/Annex/FileMatcher.hs b/Annex/FileMatcher.hs
index e46b75da6..9aa3ca18c 100644
--- a/Annex/FileMatcher.hs
+++ b/Annex/FileMatcher.hs
@@ -25,6 +25,7 @@ module Annex.FileMatcher (
 	AddUnlockedMatcher,
 	addUnlockedMatcher,
 	checkAddUnlockedMatcher,
+	LimitBy(..),
 	module Types.FileMatcher
 ) where
 
diff --git a/CHANGELOG b/CHANGELOG
index cdffcba3b..70c4c406b 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -2,6 +2,8 @@ git-annex (8.20201008) UNRELEASED; urgency=medium
 
   * Fix build on Windows with network-3.
   * Fix a memory leak introduced in the last release.
+  * add, import: Fix a reversion in 7.20191009 that broke handling
+    of --largerthan and --smallerthan.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Oct 2020 10:48:17 -0400
 
diff --git a/CmdLine/GitAnnex/Options.hs b/CmdLine/GitAnnex/Options.hs
index 22f07045c..87660e50b 100644
--- a/CmdLine/GitAnnex/Options.hs
+++ b/CmdLine/GitAnnex/Options.hs
@@ -223,7 +223,7 @@ parseKey = maybe (Fail.fail "invalid key") return . deserializeKey
 annexedMatchingOptions :: [GlobalOption]
 annexedMatchingOptions = concat
 	[ keyMatchingOptions'
-	, fileMatchingOptions'
+	, fileMatchingOptions' Limit.LimitAnnexFiles
 	, combiningOptions
 	, timeLimitOption
 	]
@@ -315,11 +315,11 @@ keyMatchingOptions' =
 	]
 
 -- Options to match files which may not yet be annexed.
-fileMatchingOptions :: [GlobalOption]
-fileMatchingOptions = fileMatchingOptions' ++ combiningOptions ++ timeLimitOption
+fileMatchingOptions :: Limit.LimitBy -> [GlobalOption]
+fileMatchingOptions lb = fileMatchingOptions' lb ++ combiningOptions ++ timeLimitOption
 
-fileMatchingOptions' :: [GlobalOption]
-fileMatchingOptions' =
+fileMatchingOptions' :: Limit.LimitBy -> [GlobalOption]
+fileMatchingOptions' lb =
 	[ globalSetter Limit.addExclude $ strOption
 		( long "exclude" <> short 'x' <> metavar paramGlob
 		<> help "skip files matching the glob pattern"
@@ -330,12 +330,12 @@ fileMatchingOptions' =
 		<> help "limit to files matching the glob pattern"
 		<> hidden
 		)
-	, globalSetter Limit.addLargerThan $ strOption
+	, globalSetter (Limit.addLargerThan lb) $ strOption
 		( long "largerthan" <> metavar paramSize
 		<> help "match files larger than a size"
 		<> hidden
 		)
-	, globalSetter Limit.addSmallerThan $ strOption
+	, globalSetter (Limit.addSmallerThan lb) $ strOption
 		( long "smallerthan" <> metavar paramSize
 		<> help "match files smaller than a size"
 		<> hidden
diff --git a/Command/Add.hs b/Command/Add.hs
index b19aaea54..6aecce8d0 100644
--- a/Command/Add.hs
+++ b/Command/Add.hs
@@ -29,9 +29,16 @@ import qualified Utility.RawFilePath as R
 
 cmd :: Command
 cmd = notBareRepo $ 
-	withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption, fileMatchingOptions] $
+	withGlobalOptions opts $
 		command "add" SectionCommon "add files to annex"
 			paramPaths (seek <$$> optParser)
+  where
+	opts =
+		[ jobsOption
+		, jsonOptions
+		, jsonProgressOption
+		, fileMatchingOptions LimitDiskFiles
+		]
 
 data AddOptions = AddOptions
 	{ addThese :: CmdParams
diff --git a/Command/Import.hs b/Command/Import.hs
index 98bc94cf2..18f75432e 100644
--- a/Command/Import.hs
+++ b/Command/Import.hs
@@ -39,11 +39,21 @@ import Control.Concurrent.STM
 
 cmd :: Command
 cmd = notBareRepo $
-	withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption, fileMatchingOptions] $
+	withGlobalOptions opts $
 		command "import" SectionCommon 
 			"add a tree of files to the repository"
 			(paramPaths ++ "|BRANCH[:SUBDIR]")
 			(seek <$$> optParser)
+  where
+	opts =
+		[ jobsOption
+		, jsonOptions
+		, jsonProgressOption
+		-- These options are only used when importing from a
+		-- directory, not from a special remote. So it's ok
+		-- to use LimitDiskFiles.
+		, fileMatchingOptions LimitDiskFiles
+		]
 
 data ImportOptions 
 	= LocalImportOptions
diff --git a/Limit.hs b/Limit.hs
index 8f958efd6..9caeb44e0 100644
--- a/Limit.hs
+++ b/Limit.hs
@@ -441,11 +441,11 @@ limitSecureHash = MatchFiles
 	}
 
 {- Adds a limit to skip files that are too large or too small -}
-addLargerThan :: String -> Annex ()
-addLargerThan = addLimit . limitSize LimitAnnexFiles (>)
+addLargerThan :: LimitBy -> String -> Annex ()
+addLargerThan lb = addLimit . limitSize lb (>)
 
-addSmallerThan :: String -> Annex ()
-addSmallerThan = addLimit . limitSize LimitAnnexFiles (<)
+addSmallerThan :: LimitBy -> String -> Annex ()
+addSmallerThan lb = addLimit . limitSize lb (<)
 
 limitSize :: LimitBy -> (Maybe Integer -> Maybe Integer -> Bool) -> MkLimit Annex
 limitSize lb vs s = case readSize dataUnits s of
diff --git a/doc/bugs/add_--largerthan_reversion.mdwn b/doc/bugs/add_--largerthan_reversion.mdwn
index 850efdfbe..c94205688 100644
--- a/doc/bugs/add_--largerthan_reversion.mdwn
+++ b/doc/bugs/add_--largerthan_reversion.mdwn
@@ -14,7 +14,4 @@ That commit was otherwise right, eg `git-annex get --largerthan` should
 look at the size of the annexed file, not of the file on disk, which could
 be a small pointer file.
 
-Rather than being global options, --largerthan and --smallerthan, 
-could added by each command, so the command can specify how the size
-should be determined. Finding a way to do that w/o needing to add
-boilerplate to many commands would be best.
+> [[fixed|done]] --[[Joey]]

bug
diff --git a/doc/bugs/add_--largerthan_reversion.mdwn b/doc/bugs/add_--largerthan_reversion.mdwn
new file mode 100644
index 000000000..850efdfbe
--- /dev/null
+++ b/doc/bugs/add_--largerthan_reversion.mdwn
@@ -0,0 +1,20 @@
+`git annex add --largerthan=200mb` of a new file
+does not add a file that is large enough.
+
+This is a reversion, introduced last year in
+[[!commit 3066bdb1fb60e80f40b5badc150fb6eb51a922bb]].
+
+`git annex import /dir --largerthan=100mb` is also affected, and has an
+ugly fail mode as it tries to look up an annexed file outside the repo:
+
+	joey@darkstar:~/tmp/t>git annex import --largerthan=100mb ../d
+	fatal: './../d/foo' is outside repository at '/home/joey/tmp/t'
+
+That commit was otherwise right, eg `git-annex get --largerthan` should
+look at the size of the annexed file, not of the file on disk, which could
+be a small pointer file.
+
+Rather than being global options, --largerthan and --smallerthan, 
+could added by each command, so the command can specify how the size
+should be determined. Finding a way to do that w/o needing to add
+boilerplate to many commands would be best.
diff --git a/doc/forum/Matching_options_not_working_in_windows_7/comment_2_b0319c1214c26f64058eb9c38ef6fb0d._comment b/doc/forum/Matching_options_not_working_in_windows_7/comment_2_b0319c1214c26f64058eb9c38ef6fb0d._comment
new file mode 100644
index 000000000..47f7f691a
--- /dev/null
+++ b/doc/forum/Matching_options_not_working_in_windows_7/comment_2_b0319c1214c26f64058eb9c38ef6fb0d._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2020-10-19T19:12:25Z"
+ content="""
+Opened a bug, [[bugs/add_--largerthan_reversion]]
+"""]]

comment
diff --git a/doc/forum/Matching_options_not_working_in_windows_7/comment_1_616f6c6d26088207ce9c044f4b61eec5._comment b/doc/forum/Matching_options_not_working_in_windows_7/comment_1_616f6c6d26088207ce9c044f4b61eec5._comment
new file mode 100644
index 000000000..61358d859
--- /dev/null
+++ b/doc/forum/Matching_options_not_working_in_windows_7/comment_1_616f6c6d26088207ce9c044f4b61eec5._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-10-19T18:59:30Z"
+ content="""
+Lucky for you, I tried it on linux, and it also fails there.
+
+I think the problem is, --largerthan and --smallerthan only work on annexed
+files, not on files you want to add to the annex. This is probably a bug.
+
+What does work is to configure annex.largefiles and disable
+annex.addsmallfiles:
+
+git config annex.largefiles 'largerthan=200mb'
+git config annex.addsmallfiles false
+"""]]

design done
diff --git a/doc/todo/interruped_move_resume.mdwn b/doc/todo/interruped_move_resume.mdwn
index dae5f8c15..41cd23206 100644
--- a/doc/todo/interruped_move_resume.mdwn
+++ b/doc/todo/interruped_move_resume.mdwn
@@ -10,6 +10,9 @@ numcopies is 2, so it can't drop the copy from the remote.
 
 This happens to me often enough to be annoying.
 
+I think it can also happen with move --to, although I can't remember seeing
+that.
+
 Perhaps some local state could avoid this problem?
 
 --[[Joey]]
@@ -48,3 +51,13 @@ Perhaps some local state could avoid this problem?
 > > Still, if the move is interrupted and never resumed, after a sync
 > > with the remote, the only copy appears missing, which does seem
 > > potentially confusing.
+
+> Local state could be a file listing keys that have had a move started
+> but not finished. When doing the same move, it should be allowed to
+> succeed even if numcopies would prevent it. More accurately, it
+> should disregard the local copy when checking numcopies for a move
+> --from. And for a move --to, it should disregard the remote copy.
+> May need 2 separate lists for the two kinds of moves.
+> 
+> > This is complex to implement, but it avoids the gotchas in the earlier
+> > ideas, so I think is best. --[[Joey]]

update
diff --git a/doc/profiling/comment_8_951c1c047ef414268295626218474677._comment b/doc/profiling/comment_8_951c1c047ef414268295626218474677._comment
new file mode 100644
index 000000000..39e0e446d
--- /dev/null
+++ b/doc/profiling/comment_8_951c1c047ef414268295626218474677._comment
@@ -0,0 +1,29 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2020-10-19T18:27:56Z"
+ content="""
+Update after some recent optimisations involving seekFilteredKeys.
+
+	        Mon Oct 19 14:36 2020 Time and Allocation Profiling Report  (Final)
+	
+	           git-annex +RTS -p -RTS find
+	
+	        total time  =        1.32 secs   (1316 ticks @ 1000 us, 1 processor)
+	        total alloc = 583,359,800 bytes  (excludes profiling overheads)
+	
+	COST CENTRE           MODULE                              SRC                                                   %time %alloc
+	
+	keyFile               Annex.Locations                     Annex/Locations.hs:(602,1)-(612,30)                    13.2   32.7
+	readObjectContent     Git.CatFile                         Git/CatFile.hs:(122,1)-(131,42)                        10.9    2.3
+	copyAndFreeze         Data.ByteArray.Methods              Data/ByteArray/Methods.hs:(237,1)-(240,21)              5.2    0.8
+	withPtr               Basement.Block.Base                 Basement/Block/Base.hs:(402,1)-(411,31)                 5.2    4.1
+	parseLinkTarget       Annex.Link                          Annex/Link.hs:(263,1)-(271,25)                          4.5   12.3
+	fileKey               Annex.Locations                     Annex/Locations.hs:(618,1)-(628,41)                     4.5    8.1
+	decimal               Data.Attoparsec.ByteString.Char8    Data/Attoparsec/ByteString/Char8.hs:(447,1)-(448,49)    4.3    4.5
+	doesPathExist         Utility.RawFilePath                 Utility/RawFilePath.hs:32:1-25                          3.9    0.6
+	ifM                   Utility.Monad                       Utility/Monad.hs:(54,1)-(56,44)                         3.0    3.2
+	catObjectStream       Git.CatFile                         Git/CatFile.hs:(322,1)-(330,45)                         2.6    2.5
+	</>                   System.FilePath.Posix.ByteString    System/FilePath/Posix/../Internal.hs:841:1-15           2.4    3.4
+	respParser            Git.CatFile                         Git/CatFile.hs:(175,1)-(182,39)                         2.4    1.3
+"""]]
diff --git a/doc/profiling/comment_9_7e1a06690b52d241b2d57be0864d6f26._comment b/doc/profiling/comment_9_7e1a06690b52d241b2d57be0864d6f26._comment
new file mode 100644
index 000000000..3212ef1fd
--- /dev/null
+++ b/doc/profiling/comment_9_7e1a06690b52d241b2d57be0864d6f26._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 9"""
+ date="2020-10-19T18:35:02Z"
+ content="""
+Also, /usr/bin/time git-annex find:
+
+	1.70user 0.27system 0:01.55elapsed 126%CPU (0avgtext+0avgdata 97352maxresident)k
+	0inputs+0outputs (0major+9303minor)pagefaults 0swaps
+
+The maxresident seems high, but a stack profile does not show a memory
+leak, or such a large amount of memory use at all. Currently, I
+think that memory is being preallocated by the ghc runtime,
+or something like that. (See [[todo/memory_use_increase]].)
+
+ghc 8.8.4  
+Should keep an eye on this with newer ghc versions.
+"""]]
diff --git a/doc/todo/memory_use_increase.mdwn b/doc/todo/memory_use_increase.mdwn
index 5a2d93477..f161d1505 100644
--- a/doc/todo/memory_use_increase.mdwn
+++ b/doc/todo/memory_use_increase.mdwn
@@ -30,3 +30,5 @@ Version 8.20200720 (cat-file --buffer)
 	
 Smoking gun. And probably reasonable. But, why exactly does that optimisation
 change the memory use in this way?
+
+[[done]]
diff --git a/doc/todo/memory_use_increase/comment_10_362b878949e691e45fac0ff1aa240688._comment b/doc/todo/memory_use_increase/comment_10_362b878949e691e45fac0ff1aa240688._comment
new file mode 100644
index 000000000..f1525e182
--- /dev/null
+++ b/doc/todo/memory_use_increase/comment_10_362b878949e691e45fac0ff1aa240688._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 10"""
+ date="2020-10-19T17:33:49Z"
+ content="""
+[[/profiling]] has a history of `+RTS -p` profiles in the same repo.
+Comparing against the 10 month old one there, current git-annex find
+runs in same time, and actually allocates slightly less memory, 583357880
+bytes down from 608475328. That's memory churn, not max memory usage,
+so doesn't rule out a memory leak. But if there is one, it's memory that
+was allocated before, so it would need to be a laziness bug I think.
+And the profiles are not showing another such leak.
+
+My feeling is, what's left now is all due to a change to haskell runtime's
+memory management, or a library. So not worth keeping this open for since I
+can't do anything about it except for keep an eye on it.
+"""]]

diff --git a/doc/forum/Matching_options_not_working_in_windows_7.mdwn b/doc/forum/Matching_options_not_working_in_windows_7.mdwn
new file mode 100644
index 000000000..14db45ae4
--- /dev/null
+++ b/doc/forum/Matching_options_not_working_in_windows_7.mdwn
@@ -0,0 +1,18 @@
+Hi there,
+
+I'm trying to set up a new git-annex repo. I want to use the WORM backend for files larger than 200MB.
+So I use the following command when adding files:
+
+    git annex add --largerthan=200MB --backend=WORM .
+
+But this does not seem to work. No files are added. I tried several variants of the command like:
+
+    git annex add --largerthan 200MB --backend=WORM .
+    git annex add --largerthan "200 MB" --backend=WORM .
+
+Do not work also. Am I missing anything?
+
+I'm using **Git 2.28.0.windows.1** and **git-annex 8.20200815-g335aae266** on a newly-created repository in an NTFS partition.
+I used the following command to initialize the repo:
+
+    git init && git annex init

more thoughts
diff --git a/doc/todo/interruped_move_resume.mdwn b/doc/todo/interruped_move_resume.mdwn
index 85ab03cd8..dae5f8c15 100644
--- a/doc/todo/interruped_move_resume.mdwn
+++ b/doc/todo/interruped_move_resume.mdwn
@@ -19,9 +19,32 @@ Perhaps some local state could avoid this problem?
 > it could resume the interrupted transfer, and numcopies would work the
 > same as it did when the move started. 
 > 
-> Or, move to annex/objects/ but delay updating the location tracking to
-> say it's in the local repo until after the drop.
+> > After an interrupted move, whereis would say the content is present,
+> > but eg an annex link to it would be broken. That seems surprising,
+> > and if the user doesn't think to resume the move, fsck would have to be
+> > made to deal with it. I don't much like this approach, it seems to
+> > change an invariant that usually existance of copy on disk is ground
+> > truth, and location tracking tries to reflect it. With this, location
+> > tracking would be correct, but only because the content is in an
+> > unusual place on disk that it can be recovered from.
 > 
-> Either way, a problem is that the only copy would appear missing until
-> the move was re-run. The latter approach at least lets fsck clean up
-> from that situation, but it could still be surprising.
+> Or: Move to annex/objects/ w/o updating local location log.
+> Then do the drop, updating the remote's location log as now.
+> Then update local location log.
+> > 
+> > If interrupted, and then the move is resumed, it will see
+> > there's a local copy, and drop again from the remote. Either that
+> > finishes the interrupted drop, or the drop already happened and it's a
+> > noop. Either way, the local location log then gets updated.
+> > That should clean things up.
+> > 
+> > But, if a sync is done with the remote first, and then the move
+> > is resumed, it will no longer think the remote has a copy. This is
+> > where the only copy can appear missing (in whereis). So a fsck
+> > will be needed to recover. Or, move could be made to recover from
+> > this too, noticing the local copy and updating the location log to
+> > reflect it.
+> > 
+> > Still, if the move is interrupted and never resumed, after a sync
+> > with the remote, the only copy appears missing, which does seem
+> > potentially confusing.

comment
diff --git a/doc/bugs/cygwin/comment_1_88b6c7d1f50d0f17443ee74b5886649c._comment b/doc/bugs/cygwin/comment_1_88b6c7d1f50d0f17443ee74b5886649c._comment
new file mode 100644
index 000000000..70c4e3f67
--- /dev/null
+++ b/doc/bugs/cygwin/comment_1_88b6c7d1f50d0f17443ee74b5886649c._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2020-10-19T16:54:15Z"
+ content="""
+> not found .
+> git-annex.exe: pre-commit: 1 failed
+
+I guess whatever this is, running `git annex pre-commit`
+would probably reproduce it, and then you could add the --debug flag and it
+would show what program it's trying to run when this happens.
+
+I don't know why it would not display the name of a program it's tried to
+run. But I guess it's probably a program, since that seems to be the most
+likely thing that would be broken by running git-annex in this unusual way.
+"""]]

comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_4_c2d4485b2a49ada5ab5266e30df91b0b._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_4_c2d4485b2a49ada5ab5266e30df91b0b._comment
new file mode 100644
index 000000000..da50534ae
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_4_c2d4485b2a49ada5ab5266e30df91b0b._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2020-10-19T16:39:28Z"
+ content="""
+I installed git for windows in a fresh windows 10
+VM. Then I installed git-annex from the git-annex for windows installer. 
+Then I ran git-annex add, which ran cp --reflink successfully.
+
+But you didn't say how you installed git-annex on windows, did you?
+I guess, since you're building it yourself, you put it into path somewhere
+else. So it seems to me, it's the up to you to make its path include the
+same cp you built it against. 
+
+Or just build the windows installer too, and install your git-annex build
+using that.
+"""]]

comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_3_fea65a7f0218ccf27446ee7ce6fc017e._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_3_fea65a7f0218ccf27446ee7ce6fc017e._comment
new file mode 100644
index 000000000..ad1f270bc
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_3_fea65a7f0218ccf27446ee7ce6fc017e._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2020-10-19T16:19:49Z"
+ content="""
+I'm not going to make git-annex probe everything at run time. The
+user can find more ways to break their environment than I can probe for,
+and sheer code complexity, and runtime complexity, and debug complexity, 
+and run time would all be affected.
+
+The way git-annex is installed into git for windows's path is supposed to both
+make "git annex" work and also put it in the same directory that git for
+windows installs curl and cp. IIRC on windows a program automatically
+looks in its directory along with the path, so that makes them available
+to git-annex. There are probably configurations that make some other path
+be used in preference, or perhaps something changed in git for windows's
+locations of these things, although the path you show sounds like the same
+one I'd expect git-annex.exe to be in. I do know that used to work.
+
+--reflink seems unlikely to do anything useful on windows
+even if the option is supported, so it could just be defaulted to not
+supported on windows. -a/-p/--preserve-timestamps are more important and
+also probed at build time.
+"""]]

close
diff --git a/doc/todo/report_PID___40__and_may_be_failure_details__63____41___in___34__process_done__34___debug_messages.mdwn b/doc/todo/report_PID___40__and_may_be_failure_details__63____41___in___34__process_done__34___debug_messages.mdwn
index 0b49ec131..4ef9396ef 100644
--- a/doc/todo/report_PID___40__and_may_be_failure_details__63____41___in___34__process_done__34___debug_messages.mdwn
+++ b/doc/todo/report_PID___40__and_may_be_failure_details__63____41___in___34__process_done__34___debug_messages.mdwn
@@ -45,3 +45,6 @@ so I wondered if those `process done ExitFailure ERR` could be made a bit more i
 
 [[!meta author=yoh]]
 [[!tag projects/datalad]]
+
+> pid added, so [[done]] .. please open something else if there's a
+> specific place you need more detail about a nonzero exit. --[[Joey]]

Added a comment: copying directly from directory special remote to the cloud
diff --git a/doc/todo/transitive_transfers/comment_7_6417ec94a5723776b6688ccbb0453ffc._comment b/doc/todo/transitive_transfers/comment_7_6417ec94a5723776b6688ccbb0453ffc._comment
new file mode 100644
index 000000000..3fe5bc28b
--- /dev/null
+++ b/doc/todo/transitive_transfers/comment_7_6417ec94a5723776b6688ccbb0453ffc._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="copying directly from directory special remote to the cloud"
+ date="2020-10-18T21:00:58Z"
+ content="""
+One more use case: running git-annex inside a VirtualBox VM (Linux guest on Windows host), with the repo inside the VM, operating on files on the host through [shared folders](https://help.ubuntu.com/community/VirtualBox/SharedFolders).  Shared folder is too large to copy inside the VM.  With the recent addition of [[importing from directory special remote without downloading|https://git-annex.branchable.com/todo/importing_from_special_remote_without_downloading/]] (thanks @joeyh for implementing that!), can import the files into the repo, bypassing [[issues with VirtualBox shared folder filesystem|bugs/removeLink_failed_when_initializing_a_repo_in_a_VirtualBox_shared_folder]].  But can't upload the files to a cloud special remote, without first [[`git-annex-get`|git-annex-get]]ting them into the VM, and I unfortunately don't have space for two local copies of each file.  So a direct transfer from a directory special remote to a cloud special remote would help a lot.  
+
+The alternative, of course, is to run git-annex directly on Windows, but I've run into speed issues with that, and using WSL means losing ability to run VirtualBox, so running git-annex from a guest VM and operating on host files through the shared folder filesystem -- broken as that is -- seems like the best option right now.
+"""]]

Added a comment: building the standalone distribution on CentOS
diff --git a/doc/install/Linux_standalone/comment_5_9046b80114fc03179c971bfb3e4d666d._comment b/doc/install/Linux_standalone/comment_5_9046b80114fc03179c971bfb3e4d666d._comment
new file mode 100644
index 000000000..4241cb23c
--- /dev/null
+++ b/doc/install/Linux_standalone/comment_5_9046b80114fc03179c971bfb3e4d666d._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="building the standalone distribution on CentOS"
+ date="2020-10-16T14:58:42Z"
+ content="""
+\"This build process is done on a Debian system, and it needs to use dpkg to examine the host system\" -- would it be hard to make this work on CentOS?  (For building the standalone tarball as part of building the [conda-forge package](https://github.com/conda-forge/git-annex-feedstock/), where the build process uses [this docker image](https://github.com/conda-forge/docker-images/blob/master/linux-anvil-comp7/Dockerfile).)
+"""]]

Added a comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_2_4a704f47a709a9de35358d6d2fb9fe20._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_2_4a704f47a709a9de35358d6d2fb9fe20._comment
new file mode 100644
index 000000000..cddb1f8f3
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_2_4a704f47a709a9de35358d6d2fb9fe20._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2020-10-16T01:50:39Z"
+ content="""
+in datalad we took an approach that \"if bundled git-annex is used, use by default git which comes with it\".  If assumption here for Windows is that \"cp which comes with git should be used\", then git-annex needs to use a specific PATH to use that one, and not some other which might be found in the PATH.  The situation is a bit messy though since `git.exe` is at `/c/Program Files (x86)/Git/cmd/git.exe` (pasting from mingw'ed bash) while `cp` is under `/c/Program Files (x86)/Git/usr/bin/cp.exe` so not exactly the same directory, but I guess consistent for all Windows'es.
+"""]]

Added a comment
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_1_44965e93c65148f06bf4fe4cad95993c._comment b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_1_44965e93c65148f06bf4fe4cad95993c._comment
new file mode 100644
index 000000000..00090cafd
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/comment_1_44965e93c65148f06bf4fe4cad95993c._comment
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 1"
+ date="2020-10-16T01:29:22Z"
+ content="""
+a mental note... building git-annex in a windows vm ATM, and saw
+
+```
+> stack ghc --no-haddock --package nsis Build/NullSoftInstaller.hs
+...
+[36 of 36] Compiling StackSetupShim   ( C:\\sr\setup-exe-src\setup-shim-Z6RU0evB.hs, C:\\tmp\git-annex\.stack-work\dist\
+29cc6475\setup\StackSetupShim.o )
+Linking C:\\tmp\\git-annex\\.stack-work\\dist\\29cc6475\\setup\\setup.exe ...
+  checking UPGRADE_LOCATION... http://downloads.kitenet.net/git-annex/windows/current/git-annex-installer.exe
+  checking git... yes
+  checking git version... 2.23.0.windows.1
+  checking cp -a... yes
+  checking cp -p... yes
+  checking cp --preserve=timestamps... yes
+  checking cp --reflink=auto... yes
+...
+  checking ssh connection caching... no
+Configuring git-annex-8.20201007...
+```
+
+so either the check is wrong or there are multiple `cp` around (may be some came with `stack`?) ... I found total over 10 different cp.exe on drive (didn't check yet how many in PATH). But I just wonder, if check is done at build time, but `cp` is not distributed along, the check should be done at run time instead.
+"""]]

Fix typo
diff --git a/doc/tips/largefiles.mdwn b/doc/tips/largefiles.mdwn
index b976615ba..7764325af 100644
--- a/doc/tips/largefiles.mdwn
+++ b/doc/tips/largefiles.mdwn
@@ -6,7 +6,7 @@ while `git add` adds files to git.
 Let's suppose you're developing a video game, written in C. You have
 source code, and some large game assets. You want to ensure the source
 code is stored in git -- that's what git's for! And you want to store
-the game assets in the git annex -- to avod bloating your git repos with
+the game assets in the git annex -- to avoid bloating your git repos with
 possibly enormous files, but still version control them.
 
 You could take care to use `git annex add` after changes to the assets,

initial report
diff --git a/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink.mdwn b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink.mdwn
new file mode 100644
index 000000000..619f316c5
--- /dev/null
+++ b/doc/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink.mdwn
@@ -0,0 +1,43 @@
+### Please describe the problem.
+
+Ref: Original issue and finding [report against datalad](https://github.com/datalad/datalad/issues/5015#issuecomment-709436065).
+
+
+### What version of git-annex are you using? On what operating system?
+
+Windows 8.20200815-g335aae266  (I see now that I might benefit from update, but I found no related issue so likely it is something new)
+
+### Please provide any additional information below.
+
+[[!format sh """
+
+{920}[Level 5] CommandError: '"git" "annex" "add" "-c" "annex.dotfiles=true" "-c" "annex.retry=3" "--json" "--json-error-messages" "--debug" "--" "file.dat"' failed with exitcode 1 under C:\Users\DataLad\AppData\Local\Temp\datalad_temp_tree_q4wxn05d [out: '{"command":"add","success":false,"error-messages":["  file.dat failed to link to annex"],"file":"file.dat"}'] [err: '[2020-10-15 11:57:07.4657537] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.retry=3","show-ref","git-annex"]
+| [2020-10-15 11:57:07.4813787] process done ExitSuccess
+| [2020-10-15 11:57:07.4813787] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.retry=3","show-ref","--hash","refs/heads/git-annex"]
+...
+| [2020-10-15 11:57:07.6376287] call: cp ["--reflink=auto","-a","file.dat",".git\\annex\\objects\\6f0\\fbd\\SHA256E-s7--ed7002b439e9ac845f22357d822bac1444730fbdb6016d3ec9432297b9ec9f73.dat\\SHA256E-s7--ed7002b439e9ac845f22357d822bac1444730fbdb6016d3ec9432297b9ec9f73.dat"]
+| "cp": unrecognized option `--reflink=auto'
+| Try `"cp" --help' for more information.
+| [2020-10-15 11:57:07.6532537] process done ExitFailure 1
+
+C:\tmp>cp --help | grep ref
+  -d                           same as --no-dereference --preserve=link
+  -L, --dereference            always follow symbolic links
+  -P, --no-dereference         never follow symbolic links
+
+C:\tmp>cp --version
+cp (GNU coreutils) 5.97
+Copyright (C) 2006 Free Software Foundation, Inc.
+This is free software.  You may redistribute copies of it under the terms of
+the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
+There is NO WARRANTY, to the extent permitted by law.
+
+Written by Torbjorn Granlund, David MacKenzie, and Jim Meyering.
+"""]]
+
+if I parsed NEWS.gz correctly, that option was added in `* Noteworthy changes in release 8.0 (2009-10-06) [beta]` . So I wonder if `git-annex` could sense for version of `cp` or presence of `--reflink` option and not use it if `cp` is too old?
+
+Meanwhile I will see if there is some sensible way to get more uptodate coreutils (or `cp`) at least within conda env
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

Added a comment
diff --git a/doc/tips/deleting_unwanted_files/comment_2_8471758f2e02ebcfd9c015691ba9f15f._comment b/doc/tips/deleting_unwanted_files/comment_2_8471758f2e02ebcfd9c015691ba9f15f._comment
new file mode 100644
index 000000000..9eb461de9
--- /dev/null
+++ b/doc/tips/deleting_unwanted_files/comment_2_8471758f2e02ebcfd9c015691ba9f15f._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Lukey"
+ avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
+ subject="comment 2"
+ date="2020-10-15T14:46:33Z"
+ content="""
+`git annex sync/copy/move/etc.` don't operate on old/unused files by default. You have to use the `--all` option to operate on every file (even old/unused ones) or the `--unused` option to operate only on files found by `git annex unused`.
+"""]]

Added a comment: dropunused and copies
diff --git a/doc/tips/deleting_unwanted_files/comment_1_2d3cf347c754047875a26d7f73c6fbb2._comment b/doc/tips/deleting_unwanted_files/comment_1_2d3cf347c754047875a26d7f73c6fbb2._comment
new file mode 100644
index 000000000..5d0e70921
--- /dev/null
+++ b/doc/tips/deleting_unwanted_files/comment_1_2d3cf347c754047875a26d7f73c6fbb2._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="fbaca1610@41d03d827c5fd52d594eef456b6196c057c0944f"
+ nickname="fbaca1610"
+ avatar="http://cdn.libravatar.org/avatar/43f0d5bb6f1b878853d43f69600310cd"
+ subject="dropunused and copies"
+ date="2020-10-15T10:19:16Z"
+ content="""
+Hey there, love git annex!
+
+I have a (silly?) question: `git annex dropunused` tells me that there are not enough copies. I agree it is a good thing or at least might be in some cases.
+
+The problem is this: I have synced and used `git annex copy --to` for all my repositories and still I have unused files that `dropunused` will not delete, since it finds no copies.
+
+Am I doing something wrong here? Why are these (now) unused files not copied and therefore okay to delete locally?
+"""]]

comment
diff --git a/doc/todo/memory_use_increase/comment_9_a04a9de3b767d644c0f359af39ee857b._comment b/doc/todo/memory_use_increase/comment_9_a04a9de3b767d644c0f359af39ee857b._comment
new file mode 100644
index 000000000..f3c6ff0dc
--- /dev/null
+++ b/doc/todo/memory_use_increase/comment_9_a04a9de3b767d644c0f359af39ee857b._comment
@@ -0,0 +1,43 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 9"""
+ date="2020-10-13T20:31:30Z"
+ content="""
+100k files: 122364k  
+40k files: 115476k  
+1k files: 97920k  
+0 files: 94808k  
+
+So, it's using a little bit more for more files, reasonable if something
+is scaling up under load.
+
+But where did that use of 90mb for 0 files come from?
+
+My earlier numbers for 8.20200617 were for the debian package of it. I
+built that version with a current toolchain:
+
+100k files: 127240k  (vs 37188k with old toolchain)  
+0 files: 105748k
+
+So, much of the increase is due to a change in ghc or the toolchain or
+libraries. Maybe its memory manager is preallocating more or a buffer has
+started to default to significantly larger?
+
+For that matter, the old build should not have needed 37mb and I'm pretty
+sure it used to be more like 2-5mb?
+
+A haskell hello world still only needs a few kb of memory.
+
+Made git-annex's `main = print "hello"` -- 61716k
+
+Made git-annex exit before it runs git ls-files -- 66892k
+
+Made it exit after starting ls-files and cat-file -- 67320k
+
+Made it exit just before running the first action on the first file, but
+after seekFilteredKeys did its processing -- 93300k
+
+So this is partly toolchain or compiler caused, but I don't know if the
+extra 30mb it grows by that point is because of the toolchain changing,
+or another memory leak.
+"""]]

devblog
diff --git a/doc/devblog/day_631-632__memory_leak.mdwn b/doc/devblog/day_631-632__memory_leak.mdwn
index 2d4fef4d8..dea5e71e4 100644
--- a/doc/devblog/day_631-632__memory_leak.mdwn
+++ b/doc/devblog/day_631-632__memory_leak.mdwn
@@ -1,14 +1,14 @@
 I've spent two days trying to track down a recently introduced memory leak,
 or leaks. This was unusually hard because all the profiler could tell
 me is the memory is "PINNED", but not what allocated it or anything else
-about it.
+about it. 
 
 I probably should have bisected it, rather than staring at the code and
 randomly reimplementing things I thought could be pinning memory. Oops.
 
-Anyway, I've solved one of them, and the other one, if it's a memory leak
-at all, is memory that the profiler does not even show is in use, but that
-does appear to be allocated, at least as far as mmap goes.
+And there is more memory that the profiler doesn't even show
+being allocated, which got much bigger with a new toolchain, and I have not
+gotten to the bottom of that yet.
 
 --