Recent changes to this wiki:

comment
diff --git a/doc/forum/rsync_test_remote_fail/comment_3_0b910e98485832dd7f305decdc39fa7f._comment b/doc/forum/rsync_test_remote_fail/comment_3_0b910e98485832dd7f305decdc39fa7f._comment
new file mode 100644
index 0000000000..b890f99d8a
--- /dev/null
+++ b/doc/forum/rsync_test_remote_fail/comment_3_0b910e98485832dd7f305decdc39fa7f._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2024-02-26T19:40:15Z"
+ content="""
+Thanks for following up that it's fixed in current version.
+
+Please use [[bugs]] in the future for bug reports.
+"""]]

Added a comment: ignore, upgrading to v10.20231227 solved this issue
diff --git a/doc/forum/rsync_test_remote_fail/comment_2_824e45450cc01b30144ac5f8265b930e._comment b/doc/forum/rsync_test_remote_fail/comment_2_824e45450cc01b30144ac5f8265b930e._comment
new file mode 100644
index 0000000000..4eec6a6026
--- /dev/null
+++ b/doc/forum/rsync_test_remote_fail/comment_2_824e45450cc01b30144ac5f8265b930e._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="tchen_annex"
+ avatar="http://cdn.libravatar.org/avatar/758a07ad226fbf59a287e6cbff1c11ed"
+ subject="ignore, upgrading to v10.20231227 solved this issue"
+ date="2024-02-25T17:34:31Z"
+ content="""
+Okay, I tried the latest version of git-annex via git-annex-standalone package in NeuroDebian deb-src. Problem solved with latest version. Please this error report.
+"""]]

diff --git a/doc/forum/Backup_of_whole_Linux_system.mdwn b/doc/forum/Backup_of_whole_Linux_system.mdwn
index afafc55a08..1408c9a52b 100644
--- a/doc/forum/Backup_of_whole_Linux_system.mdwn
+++ b/doc/forum/Backup_of_whole_Linux_system.mdwn
@@ -1,4 +1,4 @@
-If I would want to backup my whole Linux system, what's unclear or maybe missing from Git Annex:
+If I would want to back up my whole Linux system, what's unclear or maybe missing from Git Annex:
 
 I'm not exactly sure about the best way to import the files. Should I just copy over all the files (e.g. using `cp -ax /* .`, or maybe `rsync -a /* .` or so) to the repo, and then use `git annex add`? (Let's skip `/dev` and maybe other special files for now.)
 

diff --git a/doc/contribute.mdwn b/doc/contribute.mdwn
index ed6b362da9..a005e22e81 100644
--- a/doc/contribute.mdwn
+++ b/doc/contribute.mdwn
@@ -2,7 +2,7 @@ Help make git-annex better!
 
 ## user support
 
-Hang out in git-annex support areas and answer user's questions.
+Hang out in git-annex support areas and answer users' questions.
 Don't be afraid to get an answer not 100% right, but avoid wild guesses.
 If you understand the basics of how git-annex works and know how to use it,
 you are ahead of many users, and can help them out tremendously.

Added a comment: this also fails with encryption=none
diff --git a/doc/forum/rsync_test_remote_fail/comment_1_deffffd5112a60ad7837479da045d39f._comment b/doc/forum/rsync_test_remote_fail/comment_1_deffffd5112a60ad7837479da045d39f._comment
new file mode 100644
index 0000000000..84bf746520
--- /dev/null
+++ b/doc/forum/rsync_test_remote_fail/comment_1_deffffd5112a60ad7837479da045d39f._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="tchen_annex"
+ avatar="http://cdn.libravatar.org/avatar/758a07ad226fbf59a287e6cbff1c11ed"
+ subject="this also fails with encryption=none"
+ date="2024-02-25T01:08:43Z"
+ content="""
+One potentially useful info. A bit more experimentation shows that this fails with encryption=none as well. Seems to be purely a rsync issue.
+"""]]

diff --git a/doc/forum/rsync_test_remote_fail.mdwn b/doc/forum/rsync_test_remote_fail.mdwn
new file mode 100644
index 0000000000..0e30d449c5
--- /dev/null
+++ b/doc/forum/rsync_test_remote_fail.mdwn
@@ -0,0 +1,84 @@
+Hi,
+
+I am running into a bizarre testremote failure (with an rsync+encryption=shared special remote). 
+
+Here are the exact steps to replicate:
+
+    $ mkdir test
+    $ cd test/
+    $ git init
+    $ git annex init
+    $ git annex initremote newremote type=rsync rsyncurl=10.0.0.100:/backup/annex-enc/test encryption=shared
+    $ git annex testremote newremote
+
+    Here is what I will get
+
+    testremote newremote (generating test keys...) Remote Tests
+      unavailable remote
+        removeKey:                                       OK
+        storeKey:                                        OK
+        checkPresent:                                    OK
+        retrieveKeyFile:                                 OK
+        retrieveKeyFileCheap:                            OK
+      key size Just 1048576; remote chunksize=0 encryption=none
+        removeKey when not present:                      OK (1.51s)
+        present False:                                   OK (0.62s)
+        storeKey:                                        OK (1.10s)
+        present True:                                    FAIL (0.62s)
+          ./Command/TestRemote.hs:290:
+          failed
+        storeKey when already present:                   OK (0.37s)
+        present True:                                    FAIL (0.63s)
+          ./Command/TestRemote.hs:290:
+          failed
+        retrieveKeyFile:                                 rsync: change_dir "/backup/annex-enc/test/242/793/'SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key" failed: No such file or directory (2)
+    rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1865) [Receiver=3.2.7]
+    rsync: [Receiver] write error: Broken pipe (32)
+    rsync exited 23
+    rsync: change_dir "/backup/annex-enc/test/W4/4g/'SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key" failed: No such file or directory (2)
+    rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1865) [Receiver=3.2.7]
+    rsync: [Receiver] write error: Broken pipe (32)
+    rsync exited 23
+    FAIL (0.61s)
+          ./Command/TestRemote.hs:290:
+          failed
+        fsck downloaded object:                          OK
+        retrieveKeyFile resume from 33%:                 FAIL
+          Exception: .git/annex/objects/W4/4g/SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key/SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key: openBinaryFile: does not exist (No such file or directory)
+        fsck downloaded object:                          OK
+        retrieveKeyFile resume from 0:                   rsync: change_dir "/backup/annex-enc/test/242/793/'SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key" failed: No such file or directory (2)
+    rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1865) [Receiver=3.2.7]
+    rsync: [Receiver] write error: Broken pipe (32)
+    rsync exited 23
+    rsync: change_dir "/backup/annex-enc/test/W4/4g/'SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key" failed: No such file or directory (2)
+    rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1865) [Receiver=3.2.7]
+    rsync: [Receiver] write error: Broken pipe (32)
+    rsync exited 23
+    FAIL (0.61s)
+          ./Command/TestRemote.hs:290:
+          failed
+        fsck downloaded object:                          OK
+        retrieveKeyFile resume from end:                 cp: cannot stat '.git/annex/objects/W4/4g/SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key/SHA256E-s1048576--26a83e410604bf3f0f9288c14b39387ce244a832a02b192c81a97a36a0e42296.this-is-a-test-key': No such file or directory
+
+This will repeat again and again (with different remote chunksize?) until finally it hangs where I would have to Ctrl+C it to force it to abort.
+
+I am using Ubuntu 22.04.3 LTS (Jammy) with the following git annex version:
+
+    $ git annex version
+    git-annex version: 8.20210223
+    build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
+    dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
+    operating system: linux x86_64
+    supported repository versions: 8
+    upgrade supported from repository versions: 0 1 2 3 4 5 6 7
+    local repository version: 8
+
+I can't see why git-annex rsync would be failing. I can do normal rsync via command line w/o any issues.
+
+Any help would be appreciated.
+
+
+
+

comment
diff --git a/doc/submodules/comment_11_78fffae9facce1600915bc19429d81e2._comment b/doc/submodules/comment_11_78fffae9facce1600915bc19429d81e2._comment
new file mode 100644
index 0000000000..8e7289cc69
--- /dev/null
+++ b/doc/submodules/comment_11_78fffae9facce1600915bc19429d81e2._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""Re: comment 6"""
+ date="2024-02-22T16:50:34Z"
+ content="""
+@TTTTAAAx kindly posted a full example of their problem, 
+which I've moved to 
+[[todo/detect_and_handle_submodules_after_path_changed_by_mv]].
+
+I do think that using `git mv` to rename directories that contain
+submodules is the right way to avoid that kind of problem. 
+Note that renaming such a directory without using git followed by running
+`git add` on the new directory has the same behavior as running
+`git-annex assist` does. This is not a git-annex problem, but I think it
+could be considered a git problem; git could make `git add` of a moved
+submodule do the right thing.
+"""]]

close
diff --git a/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv.mdwn b/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv.mdwn
index d954204ad4..2e41a31131 100644
--- a/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv.mdwn
+++ b/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv.mdwn
@@ -70,4 +70,6 @@ git annex assist
 echo
 
 echo test done
-```
+``
+
+> [[wontfix|done]] as it is out of scope --[[Joey]]`

comment
diff --git a/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv/comment_1_9f9da126a031d2a4c98555b381dbcbdc._comment b/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv/comment_1_9f9da126a031d2a4c98555b381dbcbdc._comment
new file mode 100644
index 0000000000..1c315954de
--- /dev/null
+++ b/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv/comment_1_9f9da126a031d2a4c98555b381dbcbdc._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-22T16:43:01Z"
+ content="""
+AFAICS, the behavior is the same if you do not use git-annex at all and mv
+submodules around generally.
+
+This strongly suggests that dealing with this is out of scope for
+git-annex.
+"""]]

move misplaced bug or todo to a better place
diff --git a/doc/submodules/bug.mdwn b/doc/todo/detect_and_handle_submodules_after_path_changed_by_mv.mdwn
similarity index 100%
rename from doc/submodules/bug.mdwn
rename to doc/todo/detect_and_handle_submodules_after_path_changed_by_mv.mdwn

Added a comment: Datalad cannot detect submodule path changed on disk
diff --git a/doc/submodules/comment_10_936e145bbe3b6bd928571c17d3416937._comment b/doc/submodules/comment_10_936e145bbe3b6bd928571c17d3416937._comment
new file mode 100644
index 0000000000..eda12b3fb8
--- /dev/null
+++ b/doc/submodules/comment_10_936e145bbe3b6bd928571c17d3416937._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="TTTTAAAx"
+ avatar="http://cdn.libravatar.org/avatar/9edd4b69b9f9fc9b8c1cb8ecd03902d5"
+ subject="Datalad cannot detect submodule path changed on disk"
+ date="2024-02-22T08:34:47Z"
+ content="""
+> Sounds like you might want to use datalad, which is built around git annex and where submodules are a first-class citizen. 
+
+Datalad handles submodules as subdatasets and add python code layers on it to handle datasets(e.g. dedup submodules). But it doesn't detect the submodules path changed like git.
+
+So, it doesn't do my needs sadly.
+"""]]

http://git-annex.branchable.com/submodules/#comment-2f017a0f1a13329f4f738568b3eb82ae
diff --git a/doc/submodules/bug.mdwn b/doc/submodules/bug.mdwn
new file mode 100644
index 0000000000..d954204ad4
--- /dev/null
+++ b/doc/submodules/bug.mdwn
@@ -0,0 +1,73 @@
+It's an enhancement feature to handle submodules to manage data with associated its projects.
+
+I want `git-annex` could detect submodule paths changed on disks which was cause by `mv` or file explorer.
+If user uses `git-annex-assist daemon` or `git-annex-assist` command directly after `mv` command, The submodules would be totally broken.
+
+Currently, the workaround is just use `git-mv` on each submodules manually.
+
+I made a testing shell script for this. 
+
+
+```shell
+#!/bin/bash
+# This is test script for submodule path changing.
+# set -e
+USE_GIT_MV=false # USE_GIT_MV=true works correctly
+cd /tmp/
+mkdir -p test_sub/{archive/projects,projects/2023_01_personal_some_cool_project,resources}
+cd test_sub
+git init
+git annex init
+git annex version
+cd projects/2023_01_personal_some_cool_project
+
+echo NOTE: Add some data and sub-projects for testing
+touch README.md 01_dataset_lists.csv 09_reports.md
+git submodule add https://github.com/Lykos153/git-annex-remote-googledrive.git
+git submodule add https://github.com/alpernebbi/git-annex-adapter.git
+git submodule status # check it
+git annex assist
+echo
+
+echo NOTE: I think that the projects are need to be changed "01_Projects" for sorting order.
+cd /tmp/test_sub
+if $USE_GIT_MV; then
+    git mv projects 01_Projects
+else
+    # NOTE: Just rename file makes submodules broken. directory depth is same
+    mv projects 01_Projects
+    (
+        cd 01_Projects/2023_01_personal_some_cool_project/git-annex-adapter
+        git status # it shows 'No such file or directory'
+    )
+fi
+git submodule status # check it
+git annex assist
+echo
+
+echo NOTE: I want to change some submodule name is for referencing just for work.
+cd /tmp/test_sub/01_Projects/2023_01_personal_some_cool_project
+if $USE_GIT_MV; then
+    git mv git-annex-adapter ref_sample_adapter_code
+else
+    # NOTE: Just rename file makes submodules broken. directory depth is same
+    mv git-annex-adapter ref_sample_adapter_code
+fi
+git submodule status # check it
+git annex assist
+echo
+
+echo NOTE: Now, i want to archive my old projects.
+cd /tmp/test_sub
+if $USE_GIT_MV; then
+    git mv 01_Projects/2023_01_personal_some_cool_project archive/projects/2023_01_personal_some_cool_project
+else
+    # NOTE: Just rename file makes submodules broken. directory depth is changed
+    mv 01_Projects/2023_01_personal_some_cool_project archive/projects/2023_01_personal_some_cool_project
+fi
+git submodule status # check it
+git annex assist
+echo
+
+echo test done
+```

Added a comment
diff --git a/doc/forum/copying_annex_between_remotes_manually__63__/comment_2_1237297cfb596d59a48c8359d43a3928._comment b/doc/forum/copying_annex_between_remotes_manually__63__/comment_2_1237297cfb596d59a48c8359d43a3928._comment
new file mode 100644
index 0000000000..b1abc59cc4
--- /dev/null
+++ b/doc/forum/copying_annex_between_remotes_manually__63__/comment_2_1237297cfb596d59a48c8359d43a3928._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="lell"
+ avatar="http://cdn.libravatar.org/avatar/4c4138a71d069e290240a3a12367fabe"
+ subject="comment 2"
+ date="2024-02-22T07:49:58Z"
+ content="""
+In the example above, of course I forgot one line that creates the file \"file\"  in the repository gta, e.g. like this `echo test > file`
+"""]]

Added a comment: Using fsck is an option?
diff --git a/doc/forum/copying_annex_between_remotes_manually__63__/comment_1_c13a0cded3a5fee06106f101f122eb33._comment b/doc/forum/copying_annex_between_remotes_manually__63__/comment_1_c13a0cded3a5fee06106f101f122eb33._comment
new file mode 100644
index 0000000000..ca7ba701a9
--- /dev/null
+++ b/doc/forum/copying_annex_between_remotes_manually__63__/comment_1_c13a0cded3a5fee06106f101f122eb33._comment
@@ -0,0 +1,50 @@
+[[!comment format=mdwn
+ username="lell"
+ avatar="http://cdn.libravatar.org/avatar/4c4138a71d069e290240a3a12367fabe"
+ subject="Using fsck is an option?"
+ date="2024-02-22T07:47:36Z"
+ content="""
+Hi, this worked for me:
+
+```
+~$ mkdir ~/test/gta
+~$ cd ~/test/gta
+~/test/gta$ git init
+
+Initialized empty Git repository in /home/lell/test/gta/.git/
+
+~/test/gta$ git annex init
+~/test/gta$ git annex add file
+~/test/gta$ git commit -m test
+
+~/test/gta$ cd ..
+~/test$ git clone a b
+
+~/test$ cd gta/
+~/test/gta$ cd ..
+~/test$ git clone gta gta2
+~/test$ cd gta2
+
+~/test/gta2$ mkdir .git/annex/objects
+~/test/gta2$ cp -r ../gta/.git/annex/objects/* .git/annex/objects/
+~/test/gta2$ cp -r ../gta/.git/annex/objects .git/annex/objects
+~/test/gta2$ git annex list   # it does not yet know that the file is now also here
+here
+|origin
+||web
+|||bittorrent
+||||
+_X__ file
+~/test/gta2$ git annex fsck
+fsck file (fixing location log) (checksum...) ok
+(recording state in git...)
+$ git annex list   # now it knows
+here
+|origin
+||web
+|||bittorrent
+||||
+XX__ file
+```
+
+"""]]

Added a comment: Colocating git-annex and git-lfs
diff --git a/doc/tips/storing_data_in_git-lfs/comment_1_e4eb406a4bf146c1f442047a4c1b1e1a._comment b/doc/tips/storing_data_in_git-lfs/comment_1_e4eb406a4bf146c1f442047a4c1b1e1a._comment
new file mode 100644
index 0000000000..24908e6a6a
--- /dev/null
+++ b/doc/tips/storing_data_in_git-lfs/comment_1_e4eb406a4bf146c1f442047a4c1b1e1a._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="beryllium@5bc3c32eb8156390f96e363e4ba38976567425ec"
+ nickname="beryllium"
+ avatar="http://cdn.libravatar.org/avatar/62b67d68e918b381e7e9dd6a96c16137"
+ subject="Colocating git-annex and git-lfs"
+ date="2024-02-22T02:55:25Z"
+ content="""
+Is it possible to add git-lfs capabilities to a git-annex, without using a special remote?
+
+I guess what I want is, are there any reasonable instructions to graft the hooks so that this is possible:
+
+
+	$ git init
+	$ git-lfs install
+	$ git-annex init
+
+And you can alternate between something like below:
+
+	$ git-lfs track \"*.exif_thumbnail.*\"
+	$ git-annex add IMG_0001.jpg
+	$ git add IMG_0001.exif_thumbnail.jpg
+
+Obviously this betrays the scenario of extracting thumbnails from the EXIF header and storing them alongside, as another form of metadata. If there's a better workflow to this, that would be appreciated too.
+"""]]

Added a comment: Multi-line string in WHEREIS-SUCCESS?
diff --git a/doc/design/external_special_remote_protocol/comment_53_a2d1778e6d61f9cb606fb83f0a4a73f4._comment b/doc/design/external_special_remote_protocol/comment_53_a2d1778e6d61f9cb606fb83f0a4a73f4._comment
new file mode 100644
index 0000000000..5669cae726
--- /dev/null
+++ b/doc/design/external_special_remote_protocol/comment_53_a2d1778e6d61f9cb606fb83f0a4a73f4._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="matrss"
+ avatar="http://cdn.libravatar.org/avatar/59541f50d845e5f81aff06e88a38b9de"
+ subject="Multi-line string in WHEREIS-SUCCESS?"
+ date="2024-02-21T12:29:14Z"
+ content="""
+Is it possible to somehow make `git annex whereis` show the response of the special remote to `WHEREIS` over multiple lines? Just including newlines obviously results in an error, since that ends the WHEREIS-SUCCESS message.
+
+I am implementing a special remote for which the data is fully described by what is essentially a json-encoded request to a third-party API, and I would like to show this json string pretty-printed over multiple lines in the whereis output, instead of as a single line.
+"""]]

diff --git a/doc/forum/copying_annex_between_remotes_manually__63__.mdwn b/doc/forum/copying_annex_between_remotes_manually__63__.mdwn
new file mode 100644
index 0000000000..6e3fc6d7a6
--- /dev/null
+++ b/doc/forum/copying_annex_between_remotes_manually__63__.mdwn
@@ -0,0 +1,5 @@
+I'm currently trying to migrate a git-annex repository to a new machine, one of whose remotes is the one from [this issue](https://git-annex.branchable.com/bugs/apparent_hang_in_git-annex-smudge/). I determined that the root cause there seemed to be under git rather than git-annex; in particular, any whole-repository operation would take multiple days to execute, for unclear reasons. Pulling the commit data to a new repository seems to fix this.
+
+I'm now trying to move all the annexed data from the original, broken remote to the new one. The default option here would be `git annex move`. However, when I run this, it apparently does some git operation for every key moved, taking hours to days; there are tens of thousands of keys, so this is obviously unworkable.
+
+Is there a way to simply mass-move the annexed data into the new repo, via rsync or similar, and then update the new repo's metadata all at once? The state of the old repository does not matter, since I intend to discard it as soon as this migration is done.

Add logo version with '> git annex' text below
diff --git a/doc/logo-with-cli.png b/doc/logo-with-cli.png
new file mode 100644
index 0000000000..40e47fdb5d
Binary files /dev/null and b/doc/logo-with-cli.png differ
diff --git a/doc/logo-with-cli.svg b/doc/logo-with-cli.svg
new file mode 100644
index 0000000000..9d30581484
--- /dev/null
+++ b/doc/logo-with-cli.svg
@@ -0,0 +1,107 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   id="svg2"
+   sodipodi:version="0.32"
+   inkscape:version="1.3.2 (091e20ef0f, 2023-11-25, custom)"
+   width="337.62668"
+   height="275.14575"
+   version="1.0"
+   sodipodi:docname="logo-with-cli.svg"
+   xml:space="preserve"
+   inkscape:export-filename="logo-with-cli.png"
+   inkscape:export-xdpi="300"
+   inkscape:export-ydpi="300"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:dc="http://purl.org/dc/elements/1.1/"><metadata
+     id="metadata7"><rdf:RDF><cc:Work
+         rdf:about="Git Logo"><dc:format>image/svg+xml</dc:format><dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><defs
+     id="defs5"><inkscape:perspective
+       sodipodi:type="inkscape:persp3d"
+       inkscape:vp_x="0 : 101.13537 : 1"
+       inkscape:vp_y="0 : 1000 : 0"
+       inkscape:vp_z="97 : 101.13537 : 1"
+       inkscape:persp3d-origin="48.5 : 69.802067 : 1"
+       id="perspective11" /><inkscape:perspective
+       id="perspective3682"
+       inkscape:persp3d-origin="0.5 : 7.4687698 : 1"
+       inkscape:vp_z="1 : 7.6353698 : 1"
+       inkscape:vp_y="0 : 1000 : 0"
+       inkscape:vp_x="0 : 7.6353698 : 1"
+       sodipodi:type="inkscape:persp3d" /><inkscape:perspective
+       sodipodi:type="inkscape:persp3d"
+       inkscape:vp_x="0 : -140.76773 : 1"
+       inkscape:vp_y="0 : 1000 : 0"
+       inkscape:vp_z="97 : -140.76773 : 1"
+       inkscape:persp3d-origin="48.5 : -172.10103 : 1"
+       id="perspective11-7" /><inkscape:perspective
+       id="perspective3682-8"
+       inkscape:persp3d-origin="0.5 : -234.43433 : 1"
+       inkscape:vp_z="1 : -234.26773 : 1"
+       inkscape:vp_y="0 : 1000 : 0"
+       inkscape:vp_x="0 : -234.26773 : 1"
+       sodipodi:type="inkscape:persp3d" /></defs><sodipodi:namedview
+     inkscape:window-height="1517"
+     inkscape:window-width="2560"
+     inkscape:pageshadow="2"
+     inkscape:pageopacity="0.0"
+     guidetolerance="10.0"
+     gridtolerance="10.0"
+     objecttolerance="10.0"
+     borderopacity="1.0"
+     bordercolor="#666666"
+     pagecolor="#ffffff"
+     id="base"
+     inkscape:zoom="1.6910175"
+     inkscape:cx="20.401918"
+     inkscape:cy="180.36478"
+     inkscape:window-x="0"
+     inkscape:window-y="0"
+     inkscape:current-layer="svg2"
+     showguides="false"
+     showgrid="false"
+     inkscape:window-maximized="1"
+     inkscape:showpageshadow="2"
+     inkscape:pagecheckerboard="0"
+     inkscape:deskcolor="#d1d1d1"
+     showborder="true" /><text
+     xml:space="preserve"
+     style="font-size:48.1739px;line-height:1.25;font-family:Frutiger;-inkscape-font-specification:'Frutiger, Normal';font-variation-settings:normal;word-spacing:0px;vector-effect:none;fill:#666666;fill-opacity:1;stroke-width:0.735607;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;-inkscape-stroke:none;stop-color:#000000"
+     x="18.510771"
+     y="248.40559"
+     id="text1-7"><tspan
+       sodipodi:role="line"
+       id="tspan1-0"
+       x="18.510771"
+       y="248.40559"
+       style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:48.1739px;font-family:'JetBrains Mono';-inkscape-font-specification:'JetBrains Mono, Bold';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-variation-settings:normal;vector-effect:none;fill:#666666;fill-opacity:1;stroke-width:0.735607;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;-inkscape-stroke:none;stop-color:#000000;stop-opacity:1"
+       dx="0 0 0"><tspan
+   style="font-variation-settings:normal;letter-spacing:0px;vector-effect:none;fill:#666666;fill-opacity:1;stroke-width:0.735607;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;-inkscape-stroke:none;stop-color:#000000;stop-opacity:1"
+   id="tspan3-4"><tspan
+   style="font-variation-settings:normal;vector-effect:none;fill:#666666;fill-opacity:1;stroke-width:0.735607;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;-inkscape-stroke:none;stop-color:#000000;stop-opacity:1"
+   id="tspan4-0">&gt;</tspan> </tspan>git annex</tspan></text><path
+     sodipodi:nodetypes="ccccccccccccccccccc"
+     id="path1927-4-1"
+     d="m 160.79525,192.35577 c -11.60008,-1.49439 -23.53847,-12.46256 -25.20639,-25.33023 -1.59972,-10.75758 0.70817,-23.31443 9.6007,-30.41399 3.90042,-2.65848 9.20606,-5.63712 13.88482,-5.28265 0.14013,4.76254 -0.14013,10.55125 0,15.31379 -5.57282,0.62044 -10.96717,6.95467 -10.85492,12.75434 0.1366,7.54733 3.47685,14.48175 10.73818,16.7406 8.03172,2.70257 17.22586,1.93392 24.66094,-2.55921 6.14217,-4.7856 6.71028,-13.6958 3.49781,-20.4765 -1.44495,-3.73134 -4.56003,-6.47068 -8.78786,-6.45923 v 13.28344 H 166.40691 V 131.3289 h 36.10126 v 10.28372 l -9.88833,0.0915 c 9.22745,5.26008 10.1733,16.6128 9.70037,24.48894 0.3325,13.5299 -13.03437,24.56116 -26.292,26.03608 -4.79652,0.35961 -8.85473,0.37853 -15.23296,0.12667 z"
+     style="fill:#666666;fill-opacity:1;stroke-width:0.735607" /><path
+     sodipodi:nodetypes="ccccc"
+     id="path1925-1-7"
+     d="m 135.00354,125.64811 -0.0758,-15.00768 h 67.67588 l -0.008,15.07995 -67.59231,-0.0723 z"
+     style="fill:#d8382d;fill-opacity:1;stroke-width:0.735607" /><path
+     sodipodi:nodetypes="ccccccccccccc"
+     id="path1917-90-1"
+     d="M 160.79658,105.36859 V 89.307826 H 134.5599 v -13.97654 h 26.23668 v -18.02238 h 15.69293 v 18.02238 h 26.11406 v 13.97654 h -26.11406 l -0.25871,16.027244 -15.43422,0.0335 z"
+     style="fill:#40bf4c;fill-opacity:1;stroke-width:0.735607" /><path
+     style="fill:#40bf4c;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.735607"
+     d="M 91.825744,18.068854 76.171139,39.953171 h 8.68936 c 0.01763,2.418934 0.222525,4.801478 0.574654,7.126197 l 13.81566,-2.091884 c -0.247091,-1.644053 -0.396048,-3.325842 -0.413778,-5.034313 h 8.643395 z M 100.3312,49.906858 86.975373,53.92971 c 0.679483,2.253194 1.505492,4.445378 2.482673,6.551503 l 12.666194,-5.861871 c -0.70324,-1.517249 -1.30665,-3.088285 -1.79304,-4.712484 z m 4.18377,9.011189 -11.746687,7.562963 c 5.285482,8.195887 12.907037,14.747249 21.930287,18.689025 l 5.58598,-12.804167 c -6.49673,-2.835317 -11.97061,-7.5448 -15.76958,-13.447821 z m 20.41311,15.07995 -3.77,13.470807 c 2.13297,0.596378 4.31508,1.045614 6.55155,1.356282 L 129.6406,74.98647 c -1.60694,-0.22606 -3.17878,-0.559429 -4.71252,-0.988473 z"
+     id="path2836-9-1" /><path
+     style="fill:#40bf4c;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:0.735607"
+     d="m 245.29182,18.068854 -15.65468,21.884317 h 8.68936 c -0.0176,1.708471 -0.16669,3.39026 -0.41378,5.034313 l 13.81566,2.091884 c 0.35214,-2.324719 0.55679,-4.707263 0.57466,-7.126197 h 8.64339 z m -8.45948,31.838004 c -0.48638,1.624199 -1.08981,3.195235 -1.79305,4.712484 l 12.66621,5.861871 c 0.9771,-2.106125 1.80319,-4.298309 2.48267,-6.551503 z m -4.18377,9.011189 c -3.79897,5.903021 -9.27284,10.612504 -15.76958,13.447821 l 5.58598,12.804167 c 9.02325,-3.941776 16.64481,-10.493138 21.93029,-18.689025 z m -20.4131,15.07995 c -1.53374,0.429044 -3.10559,0.762413 -4.71252,0.988473 l 1.93097,13.838616 c 2.23647,-0.310668 4.41864,-0.759904 6.55153,-1.356282 z"
+     id="path3649-8-1" /></svg>
diff --git a/doc/logo.mdwn b/doc/logo.mdwn
index 9055917996..ae006ce3a0 100644
--- a/doc/logo.mdwn
+++ b/doc/logo.mdwn
@@ -4,6 +4,12 @@ Variants of the git-annex logo.
 
 [[logo.svg]]
 
+[[logo-with-cli.svg]]
+
+[[logo-with-cli.png]]
+
+(The font of the `> git annex` text is [JetBrains Mono](https://www.jetbrains.com/lp/mono/))
+
 [[logo-old.png]]
 
 [[logo-old_small.png]]
@@ -16,6 +22,7 @@ Variants of the git-annex logo.
 
 	Copyright: 2007 Henrik Nyh <http://henrik.nyh.se/>
 	           2010 Joey Hess <id@joeyh.name>
-		   2013 John Lawrence
+	           2013 John Lawrence
+	           2024 Yann Büchau <nobodyinperson at posteo de>
 	License: other
 	  Free to modify and redistribute with due credit, and obviously free to use.

expand on what i feel a tweak could help
diff --git a/doc/bugs/unlocked_files_feel_too_slow_.mdwn b/doc/bugs/unlocked_files_feel_too_slow_.mdwn
index 629c28949a..cf9ba5162f 100644
--- a/doc/bugs/unlocked_files_feel_too_slow_.mdwn
+++ b/doc/bugs/unlocked_files_feel_too_slow_.mdwn
@@ -26,7 +26,9 @@ It's not quite clear to me how to reproduce this from scratch. I guess convert a
 
 ### Please provide any additional information below.
 
-Not sure what could help here.
+Not sure what could help here. Maybe having more feedback on what's actually happening in those minutes-long stretches would help. Git, for example, has a percentage/object count thing that pops up when things like `git status` take too long. I would love to see something similar here.
+
+Also, I understand the [[todo/git_smudge_clean_interface_suboptiomal]] problem, and this might just be a specific manifestation of it. It still seems to me having some feedback, if even around the git filter boundaries, would alleviate some of those problems.
 
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 

diff --git a/doc/bugs/unlocked_files_feel_too_slow_.mdwn b/doc/bugs/unlocked_files_feel_too_slow_.mdwn
new file mode 100644
index 0000000000..629c28949a
--- /dev/null
+++ b/doc/bugs/unlocked_files_feel_too_slow_.mdwn
@@ -0,0 +1,33 @@
+### Please describe the problem.
+
+I recently switched a large (~6TiB) video repository from "normal" to "unlocked" mode, mainly because some external tool was getting confused by symlinks.
+
+As a result, all remotes related to that repositories have this painful, multi-hour process by which they convert their work tree to follow the new layout. That, in itself, is disturbingly slow, with little progress shown to the user. In fact, I often fear that git-annex is messing up and actually adding files as git blobs itself, which makes me interrupt it and start over again, which, obviously, makes things much worse.
+
+But there are other weird things. At the moment, I'm copying files from my laptop with:
+
+    git annex copy --from angela -J2
+
+Being worried about the lack of progress, I interrupted it and ran instead:
+
+    git annex copy --from angela -J2 --not --in here --in angela
+
+That command generated zero output for 12 minutes straight. Looking at strace, it seemed to have read a large number of files in `.git/annex/objects`, which is slow.
+
+Another example is, after the above command completed, i ran `git annex sync -g`, which i expected to take a handful of seconds, but the first line was a `commit` that took nearly a minute.
+
+### What steps will reproduce the problem?
+
+It's not quite clear to me how to reproduce this from scratch. I guess convert an existing large repo?
+
+### What version of git-annex are you using? On what operating system?
+
+10.20230802-1~bpo12+1 on debian bookworm.
+
+### Please provide any additional information below.
+
+Not sure what could help here.
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+I use git-annex daily, and it's a lifesaver. :)

Add annextimelog project
diff --git a/doc/projects/annextimelog.mdwn b/doc/projects/annextimelog.mdwn
new file mode 100644
index 0000000000..031032b84b
--- /dev/null
+++ b/doc/projects/annextimelog.mdwn
@@ -0,0 +1,10 @@
+[annextimelog](https://gitlab.com/nobodyinperson/annextimelog) (`atl`) is a cli time tracker and logbook made by [[users/nobodyinperson]]. It uses git-annex to store and sync data. `atl` has a loosely similar usage to [timewarrior](https://timewarrior.net/) but improves on many of its pain points:
+
+- tracking of simultaneous events
+- conflict-free syncing
+- more powerful tagging system
+- more powerful search
+- more output formats (JSON, hledger timeclock, ...)
+- a more sophisticated event specification language
+
+annextimelog is also an experiment how useable git-annex is as a backend for syncable application data storage. While first sounding silly, it works very well.

add openneuro project
diff --git a/doc/projects/openneuro.mdwn b/doc/projects/openneuro.mdwn
new file mode 100644
index 0000000000..e822f437c6
--- /dev/null
+++ b/doc/projects/openneuro.mdwn
@@ -0,0 +1,28 @@
+OpenNeuro ([openneuro.org](https://openneuro.org))
+==================================================
+
+[OpenNeuro](https://openneuro.org) is a free and open platform for sharing [BIDS](https://bids.neuroimaging.io)-compliant neuroimaging (MRI, PET, MEG, EEG, and iEEG) data. 
+Git-annex is the core tool for data logistics at OpenNeuro, e.g. for exporting the content to AWS S3 into open `openneuro.org` S3 bucket, and providing all datasets as DataLad datasets (git/git-annex repositories with a unique DataLad ID) from the [github.com/OpenNeuroDatasets](https://github.com/OpenNeuroDatasets) organization.
+
+## TODOs
+
+[[!inline pages="todo/* and !todo/done and !link(todo/done) and tagged(projects/openneuro)" sort=mtime feeds=no actions=yes archive=yes show=0 template=buglist]] 
+
+<details>
+<summary>Done</summary>
+
+[[!inline pages="todo/* and !todo/done and link(todo/done) and tagged(projects/openneuro)" sort=mtime feeds=no actions=yes archive=yes show=0 template=buglist]] 
+
+</details>
+
+## BUGs
+
+[[!inline pages="bugs/* and !bugs/done and !link(bugs/done) and tagged(projects/openneuro)" sort=mtime feeds=no actions=yes archive=yes show=0 template=buglist]] 
+
+<details>
+<summary>Done</summary>
+
+[[!inline pages="((bugs/* and !bugs/done and link(bugs/done)) or projects/openneuro/bugs-done/*) and tagged(projects/openneuro)" sort=mtime feeds=no actions=yes archive=yes show=0 template=buglist]] 
+
+</details>
+

comment
diff --git a/doc/forum/Git_Annex_shirts_at_hellotux.com/comment_1_fa72689070c39054d166d32fa144fabc._comment b/doc/forum/Git_Annex_shirts_at_hellotux.com/comment_1_fa72689070c39054d166d32fa144fabc._comment
new file mode 100644
index 0000000000..7be35c8765
--- /dev/null
+++ b/doc/forum/Git_Annex_shirts_at_hellotux.com/comment_1_fa72689070c39054d166d32fa144fabc._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-16T18:10:20Z"
+ content="""
+I did get back to that email today.
+
+For what it's worth, I'm generally ok with anyone making git-annex merch. 
+See the [[logo]] page for its license.
+"""]]

Added a comment
diff --git a/doc/bugs/identically_configured_remotes_behave_differently/comment_1_14fa5263ee9feedff8c36f86e4c33fd8._comment b/doc/bugs/identically_configured_remotes_behave_differently/comment_1_14fa5263ee9feedff8c36f86e4c33fd8._comment
new file mode 100644
index 0000000000..40bbc863db
--- /dev/null
+++ b/doc/bugs/identically_configured_remotes_behave_differently/comment_1_14fa5263ee9feedff8c36f86e4c33fd8._comment
@@ -0,0 +1,351 @@
+[[!comment format=mdwn
+ username="jstritch"
+ avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
+ subject="comment 1"
+ date="2024-02-16T17:48:55Z"
+ content="""
+I have attached a script and log file demonstrating the problem. The script:
+
+1) sets up an integration repository and two backup repositories each containing one file.
+
+2) adds another file to the integration repository, pushes it to both backups, and does a `git annex list` which shows correctly.
+
+3) does a `git annex sync music-backup-one` to introduce the problem.
+
+4) adds another file to the integration repository, pushes it to both backups, and does a `git annex list` which shows the file missing from `music-backup-one`.
+
+5) does `git annex mirror . --to=music-backup-one` then another `git annex list` which shows the files as expected.
+
+If the sync command is commented out, the last two lines are unnecessary. The sync command should not change the behavior of other commands.
+
+[[!format sh \"\"\"
+mkdir music-repo
+mkdir music-backup-one
+mkdir music-backup-two
+
+cd music-repo
+git init --initial-branch main
+git annex init music-repo
+git config annex.adjustedbranchrefresh true
+git config receive.denyCurrentBranch updateInstead
+git annex config --set annex.largefiles include=*.xtx
+git annex group here manual
+git annex wanted here standard
+echo \"test file one\" >> test-file-one.xtx
+git annex add .
+git commit -m \"add test file one\"
+git annex adjust --unlock
+
+cd ../music-backup-one
+git clone --origin music-repo ../music-repo .
+git annex init music-backup-one
+cd ../music-repo
+git remote add music-backup-one ../music-backup-one
+git annex push music-backup-one
+
+cd ../music-backup-two
+git clone --origin music-repo ../music-repo .
+git annex init music-backup-two
+cd ../music-repo
+git remote add music-backup-two ../music-backup-two
+git annex push music-backup-two
+
+cd ../music-backup-one
+git config annex.adjustedbranchrefresh true
+git config receive.denyCurrentBranch updateInstead
+git annex config --set annex.largefiles include=*.xtx
+git annex group here manual
+git annex wanted here standard
+git annex adjust --lock
+
+cd ../music-backup-two
+git config annex.adjustedbranchrefresh true
+git config receive.denyCurrentBranch updateInstead
+git annex config --set annex.largefiles include=*.xtx
+git annex group here manual
+git annex wanted here standard
+git annex adjust --lock
+
+cd ../music-repo
+echo \"test file two\" >> test-file-two.xtx
+git annex add .
+git commit -m \"add test file two\"
+git annex push music-backup-one
+git annex push music-backup-two
+git annex list
+
+# *** comment out line below to fix ***
+git annex sync music-backup-one
+
+echo \"test file three\" >> test-file-three.xtx
+git annex add .
+git commit -m \"add test file three\"
+git annex push music-backup-one
+git annex push music-backup-two
+git annex list
+
+# *** lines below only needed if sync is not commented out ***
+git annex mirror . --to=music-backup-one
+git annex list
+\"\"\"]]
+
+Here's the log file:
+
+[[!format sh \"\"\"
+Initialized empty Git repository in /home/juanito/Documents/music-repo/.git/
+init music-repo ok
+(recording state in git...)
+annex.largefiles include=*.xtx ok
+(recording state in git...)
+group here ok
+(recording state in git...)
+wanted here ok
+(recording state in git...)
+add test-file-one.xtx 
+ok                                
+(recording state in git...)
+[main (root-commit) 3358556] add test file one
+ 1 file changed, 1 insertion(+)
+ create mode 120000 test-file-one.xtx
+adjust  
+Switched to branch 'adjusted/main(unlocked)'
+ok
+Cloning into '.'...
+done.
+init music-backup-one (merging music-repo/git-annex into git-annex...)
+ok
+(recording state in git...)
+copy test-file-one.xtx (to music-backup-one...) 
+(recording state in git...)       
+ok
+(recording state in git...)
+push music-backup-one 
+Enumerating objects: 9, done.
+Counting objects: 100% (9/9), done.
+Delta compression using up to 12 threads
+Compressing objects: 100% (4/4), done.
+Writing objects: 100% (5/5), 422 bytes | 422.00 KiB/s, done.
+Total 5 (delta 2), reused 0 (delta 0), pack-reused 0
+To ../music-backup-one
+ * [new branch]      main -> synced/main
+ * [new branch]      git-annex -> synced/git-annex
+ok
+Cloning into '.'...
+done.
+init music-backup-two (merging music-repo/git-annex into git-annex...)
+(recording state in git...)
+ok
+(recording state in git...)
+copy test-file-one.xtx (to music-backup-two...) 
+(recording state in git...)       
+ok
+(recording state in git...)
+push music-backup-two 
+Enumerating objects: 9, done.
+Counting objects: 100% (9/9), done.
+Delta compression using up to 12 threads
+Compressing objects: 100% (4/4), done.
+Writing objects: 100% (5/5), 447 bytes | 447.00 KiB/s, done.
+Total 5 (delta 2), reused 0 (delta 0), pack-reused 0
+To ../music-backup-two
+ * [new branch]      main -> synced/main
+ * [new branch]      git-annex -> synced/git-annex
+ok
+annex.largefiles include=*.xtx (merging synced/git-annex into git-annex...)
+(recording state in git...)
+ok
+group here ok
+(recording state in git...)
+wanted here ok
+(recording state in git...)
+adjust  
+Switched to branch 'adjusted/main(locked)'
+ok
+annex.largefiles include=*.xtx (merging synced/git-annex into git-annex...)
+(recording state in git...)
+ok
+group here ok
+(recording state in git...)
+wanted here ok
+(recording state in git...)
+adjust  
+Switched to branch 'adjusted/main(locked)'
+ok
+add test-file-two.xtx 
+ok                                
+(recording state in git...)
+[adjusted/main(unlocked) e5a8b04] add test file two
+ 1 file changed, 1 insertion(+)
+ create mode 100644 test-file-two.xtx
+copy test-file-two.xtx (to music-backup-one...) 
+ok                                
+(recording state in git...)
+push music-backup-one 
+Enumerating objects: 23, done.
+Counting objects: 100% (23/23), done.
+Delta compression using up to 12 threads
+Compressing objects: 100% (15/15), done.
+Writing objects: 100% (18/18), 1.49 KiB | 1.49 MiB/s, done.
+Total 18 (delta 6), reused 0 (delta 0), pack-reused 0
+remote: merge synced/main (Merging into main...) 
+remote: Updating 3358556..e4f11f9
+remote: Fast-forward
+remote:  test-file-two.xtx | 1 +
+remote:  1 file changed, 1 insertion(+)

(Diff truncated)
removed
diff --git a/doc/bugs/identically_configured_remotes_behave_differently/comment_1_d69d6a014c7265fb901dc407516e232b._comment b/doc/bugs/identically_configured_remotes_behave_differently/comment_1_d69d6a014c7265fb901dc407516e232b._comment
deleted file mode 100644
index 2677b04dab..0000000000
--- a/doc/bugs/identically_configured_remotes_behave_differently/comment_1_d69d6a014c7265fb901dc407516e232b._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="jstritch"
- avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
- subject="comment 1"
- date="2024-02-06T19:54:40Z"
- content="""
-This is possibly another manifestation of `pull doesn't work on adjusted branches`. I'll await your reply to it.
-"""]]

Fix typo
diff --git a/doc/forum/git_annex_lock__47__unlock__47__fix_on_non-committed_files.mdwn b/doc/forum/git_annex_lock__47__unlock__47__fix_on_non-committed_files.mdwn
index ec8f1f08fe..3e7c562114 100644
--- a/doc/forum/git_annex_lock__47__unlock__47__fix_on_non-committed_files.mdwn
+++ b/doc/forum/git_annex_lock__47__unlock__47__fix_on_non-committed_files.mdwn
@@ -4,7 +4,7 @@ In that case, `git annex unlock` refuses to work:
 
 ```
 mv file file_version1
-git annex unlock file
+git annex unlock file_version1
 error: pathspec 'file' did not match any file(s) known to git
 Did you forget to 'git add'?
 unlock: 1 failed

diff --git a/doc/forum/git_annex_lock__47__unlock__47__fix_on_non-committed_files.mdwn b/doc/forum/git_annex_lock__47__unlock__47__fix_on_non-committed_files.mdwn
new file mode 100644
index 0000000000..ec8f1f08fe
--- /dev/null
+++ b/doc/forum/git_annex_lock__47__unlock__47__fix_on_non-committed_files.mdwn
@@ -0,0 +1,15 @@
+In an ideal world, one could just use `git annex diffdriver` to compare annex'ed files. However, sometimes one might need an approach that is constructed simpler. For example, I want to run a comparison utility that is in a container which translates my paths for better reproducibility of scientific results. Or I have a colleage who is not familiar with all the intricacies of git annex and knows git just enough `git checkout` version 1, move it, the `git checkout` the second version and compare these two. Then the situation can arise that one has a symlink to `.git/annex/objects/...` which is named differently than what is saved in the tree. 
+
+In that case, `git annex unlock` refuses to work:
+
+```
+mv file file_version1
+git annex unlock file
+error: pathspec 'file' did not match any file(s) known to git
+Did you forget to 'git add'?
+unlock: 1 failed
+```
+
+Similar story goes with `git lock` and `git fix`. I think it would be a nice behaviour if these commands would just do their work on the new file name. For example, git unlock would replace a symlink with its content even if the file is not committed with this name. 
+
+Is it possible to implement this in git annex?

Added a comment: a script to hide history
diff --git a/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_3_7e92db3d8a392e43b305fa8ecc0e39d3._comment b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_3_7e92db3d8a392e43b305fa8ecc0e39d3._comment
new file mode 100644
index 0000000000..404146779a
--- /dev/null
+++ b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_3_7e92db3d8a392e43b305fa8ecc0e39d3._comment
@@ -0,0 +1,39 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="a script to hide history"
+ date="2024-02-15T22:05:12Z"
+ content="""
+Here is a script I crafted to use to make it easy and reuse current tree object for new \"squashed history\" commit
+
+```bash
+#!/bin/bash
+#
+# A helper to establish an alternative history to hide commits which could have
+# leaked personal data etc.
+#
+# More information on motivation etc and another implementation could be
+# found at https://git-annex.branchable.com/tips/redacting_history_by_converting_git_files_to_annexed/
+#
+
+set -eu
+
+BRANCH=$(git rev-parse --abbrev-ref HEAD)
+: \"${SECRET_BRANCH:=unredacted-$BRANCH}\"
+SAFE_BASE=\"$1\"
+
+git branch \"${SECRET_BRANCH}\"
+
+rm -f .git/COMBINED_COMMIT_MESSAGE
+echo -e \"Combined commits to hide away sensitive data\n\" >> .git/COMBINED_COMMIT_MESSAGE
+git log --format=%B \"$SAFE_BASE..HEAD\" >> .git/COMBINED_COMMIT_MESSAGE
+
+# the tree we are on ATM
+TREE_HASH=$(git log -1 --format=%T HEAD)
+NEW_COMMIT=$(git commit-tree $TREE_HASH -p \"$SAFE_BASE\" -F .git/COMBINED_COMMIT_MESSAGE)
+rm -f .git/COMBINED_COMMIT_MESSAGE
+git reset --hard $NEW_COMMIT
+
+git replace \"$BRANCH\" \"$SECRET_BRANCH\"
+```
+"""]]

update thanks and separate out 2023 name list
diff --git a/doc/thanks.mdwn b/doc/thanks.mdwn
index 9240246c72..aa0d090b4c 100644
--- a/doc/thanks.mdwn
+++ b/doc/thanks.mdwn
@@ -31,6 +31,20 @@ tips, user support, etc. Two names that come to mind are Anarcat and
 CandyAngel. John Lawrence made the logo. And many others have
 contributed good bug reports and great ideas.
 
+## financial support, 2024
+
+git-annex development is supported in large part by:
+
+* [DANDI](https://www.dandiarchive.org/), funded by
+  [a NIH grant](https://projectreporter.nih.gov/project_info_description.cfm?aid=9981835)
+  awarded to MIT, Dartmouth College, and Kitware.
+* [ReproNim](https://repronim.org/), funded by
+  [a NIH grant](https://projectreporter.nih.gov/project_info_details.cfm?aid=8999833)
+  awarded to UMass Medical School Worcester, Dartmouth College, MIT, et al.
+
+Thanks also to these folks for their support: 
+[[!inline raw=yes pages="thanks/list"]] and anonymous supporters.
+
 ## financial support, 2019-2023
 
 <img alt="McGill logo" src="https://mcgill.ca/hbhl/sites/all/themes/moriarty/images/logo-red.svg" width=100>
@@ -56,7 +70,7 @@ git-annex development was supported in large part by:
 * Gioele Barabucci ENK
 
 Thanks also to these folks for their support: 
-[[!inline raw=yes pages="thanks/list"]] and anonymous supporters.
+[[!inline raw=yes pages="thanks/list.2023"]] and anonymous supporters.
 
 ## financial support, 2014-2018
 
diff --git a/doc/thanks/list b/doc/thanks/list
index a549f5fc64..f64604c81f 100644
--- a/doc/thanks/list
+++ b/doc/thanks/list
@@ -1,109 +1,77 @@
-Ethan Aubin, 
-Thomas May, 
-Jake Vosloo, 
-Trenton Cronholm, 
+Kanak Kshetri, 
+Andrew Gilbert, 
+Noam Kremen, 
+Mark Reidenbach, 
+Martin D, 
+James Greenaway, 
+Timothy Schumann, 
+Graham Spencer, 
+sirmacik, 
+Cesar, 
+Rian McGuire, 
+Jelmer Vernooij, 
+Svenne Krap, 
+Ryan Rix, 
+James Read, 
+Luke Shumaker, 
+EVAN HENSHAWPLATH, 
+Nikhil, 
+Josh Moller-Mara, 
+André Pereira, 
+Brennen Bearnes, 
+N. A., 
 Denis Dzyubenko, 
-Thomas Koch, 
-mo, 
-Jochen Bartl, 
-Brock Spratlen, 
-Jack Hill, 
+paul walmsley, 
+Pluralist Extremist, 
+Falk Hüffner, 
 Nick Piper, 
-Eric Drechsel, 
 Brett Eisenberg, 
+Eric Prestemon, 
 John Pellman, 
-Boyd Stephen Smith Jr., 
-Ilya Shlyakhter, 
 Markus Hauru, 
-Kuno Woudt, 
 Ewen McNeill, 
-FBC, 
-Amitai Schleier, 
-Michal Sojka, 
-Brennen Bearnes, 
-Eric Prestemon, 
-BonusWavePilot, 
-Baldur Kristinsson, 
-Matthias Urlichs, 
-Diederik de Haas, 
-Boris, 
-Jason Woofenden, 
-William Hay, 
-Nathan Howell, 
-Josh, 
+Nicolas Pouillard, 
+Johannes Grødem, 
+Gioele Barabucci, 
+TD, 
+Erik Bjäreholt, 
 Silvio Ankermann, 
-John Carr, 
-Henrik Riomar, 
-Eskild Hustvedt, 
-Nick Daly, 
-Daniel Takamori, 
-John Lee, 
+Michael Alan Dorman, 
 Lars Wallenborn, 
-Willard Korfhage, 
-Erik Bjäreholt, 
-Calvin Beck, 
-Johannes Grødem, 
+John Kozak, 
 Michael Tolly, 
-AlexS, 
-paul walmsley, 
-tdittr, 
-Peter, 
-Fernando Jimenez, 
-Sergey Karpukhin, 
-Jeff Moe, 
-Alexander Thompson, 
-Nicolas Pouillard, 
-Tyler Cipriani, 
-Josh Moller-Mara, 
-Lp and Nick Daly, 
-Walltime, 
-Caleb Allen, 
-TD, 
-Pedro Araújo, 
-Ryan Newton, 
-David W, 
-L N D, 
-EVAN HENSHAWPLATH, 
-James Read, 
-Luke Shumaker, 
-Marius Konitzer, 
-Ryan Rix, 
-Svenne Krap, 
-Jelmer Vernooij, 
-Rian McGuire, 
-Cesar, 
-LND, 
-sirmacik, 
-Abdó Roig-Maranges, 
+Daniel Takamori, 
+Jack Hill, 
 Alex Watson, 
-allanfranta, 
-André Pereira, 
+William Hay, 
+Michal Sojka, 
+Rob Hunter, 
+Boris, 
+Trenton Cronholm, 
 E. Dunham, 
-Falk Hüffner, 
+Jake Vosloo, 
+Brock Spratlen, 
 Filippo Giunchedi, 
-Gioele Barabucci, 
-John Kozak, 
+LND, 
 Karl Semich, 
-Madison McGaffin, 
-Marco Roeland, 
+Boyd Stephen Smith Jr., 
+Jochen Bartl, 
+Nathan Howell, 
+Kuno Woudt, 
+Baldur Kristinsson, 
+BonusWavePilot, 
 Mark Eichin, 
-Michael Alan Dorman, 
-N. A., 
-Nikhil, 
-Rob Hunter, 
+FBC, 
+Marco Roeland, 
+Josh, 
+visnescire, 
 Simon Michael, 
-Stephan Burkhardt, 
 Thomas, 
-visnescire, 
-Graham Spencer, 
-Timothy Schumann, 
-Mark Reidenbach, 
-Martin D, 
-James Greenaway, 
-Kanak Kshetri, 
-Andrew Gilbert, 
-Noam Kremen, 
-Pluralist Extremist, 
+Henrik Riomar, 
+allanfranta, 

(Diff truncated)
Suggest cooperating with hellotux.com for git-annex-branded shirts
diff --git a/doc/forum/Git_Annex_shirts_at_hellotux.com.mdwn b/doc/forum/Git_Annex_shirts_at_hellotux.com.mdwn
new file mode 100644
index 0000000000..d6c3357d3e
--- /dev/null
+++ b/doc/forum/Git_Annex_shirts_at_hellotux.com.mdwn
@@ -0,0 +1,3 @@
+The fine people over at https://hellotux.com would love to make git-annex branded shirts, hoodies and currently also backpacks with git-annex branding. We recently started a cooperation for DataLad shirts, which are now available [here](https://www.hellotux.com/datalad). I have one of those and they look and feel very good. I would love to have the same with git annex's logo embroidered on it. Especially as the distribits meeting is upcoming, now would be a good time to give Gabor from hellotux a quick go that he may start. It won't cost git-annex anything, for datalad the process was very simple: send Gabor the SVG, they make a test stitch and if that looks fine, he'll add it to the webpage.
+
+@joey if you're against this for some reason, you may just say so, then I'll stop bringing this up but I'd like to know what you think of it. 😃

pre-commit: Avoid committing the git-annex branch
Except when a commit is made in a view, which changes metadata.
Make the assistant commit the git-annex branch after git commit of working
tree changes.
This allows using the annex.commitmessage-command in the assistant to
generate a commit message for the git-annex branch that relies on state
gathered during the commit of the working tree. Eg, it might reuse the
commit message.
Note that, when not using the assistant, a git-annex add still commits
the git-annex branch, so such a annex.commitmessage-command set up would
not work then. But if someone is using the assistant and wants
programmatic control over commit messages, this is useful. Someone not
using the assistant can get the same result by using annex.alwayscommit=false
during the git-annex add, and git-annex merge after they git commit.
pre-commit was never really intended to commit the git-annex branch
(except after recording changed metadata), but the assistant did sort of
rely on it. It does later commit the git-annex branch before pushing to
remotes, but I didn't want to risk building up lots of uncommitted changes
to it if that didn't happen frequently.
Sponsored-by: the NIH-funded NICEMAN (ReproNim TR&D3) project
diff --git a/Assistant/Threads/Committer.hs b/Assistant/Threads/Committer.hs
index 79eed77e95..07013c0486 100644
--- a/Assistant/Threads/Committer.hs
+++ b/Assistant/Threads/Committer.hs
@@ -1,6 +1,6 @@
 {- git-annex assistant commit thread
  -
- - Copyright 2012-2023 Joey Hess <id@joeyh.name>
+ - Copyright 2012-2024 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -19,7 +19,6 @@ import Assistant.TransferQueue
 import Assistant.Drop
 import Types.Transfer
 import Logs.Location
-import qualified Annex.Queue
 import Utility.ThreadScheduler
 import qualified Utility.Lsof as Lsof
 import qualified Utility.DirWatcher as DirWatcher
@@ -35,6 +34,8 @@ import Annex.InodeSentinal
 import Annex.CurrentBranch
 import Annex.FileMatcher
 import qualified Annex
+import qualified Annex.Queue
+import qualified Annex.Branch
 import Utility.InodeCache
 import qualified Database.Keys
 import qualified Command.Sync
@@ -248,6 +249,11 @@ commitStaged msg = do
 				]
 			when ok $
 				Command.Sync.updateBranches =<< getCurrentBranch
+			{- Commit the git-annex branch. This comes after
+			 - the commit of the staged changes, so that
+			 - annex.commitmessage-command can examine that
+			 - commit. -}
+			Annex.Branch.commit =<< Annex.Branch.commitMessage
 			return ok
 
 {- If there are PendingAddChanges, or InProcessAddChanges, the files
diff --git a/CHANGELOG b/CHANGELOG
index e4d178c408..59efa6a74e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -12,6 +12,8 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
   * assistant, undo: When committing, let the usual git commit
     hooks run.
   * Added annex.commitmessage-command config.
+  * pre-commit: Avoid committing the git-annex branch
+    (except when a commit is made in a view, which changes metadata).
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Jan 2024 15:59:33 -0400
 
diff --git a/Command/PreCommit.hs b/Command/PreCommit.hs
index d8fdeea197..bc69e4a210 100644
--- a/Command/PreCommit.hs
+++ b/Command/PreCommit.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2010-2014 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2024 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -20,12 +20,14 @@ import Logs.View
 import Logs.MetaData
 import Types.View
 import Types.MetaData
+import qualified Annex
+import qualified Annex.Branch
 
 import qualified Data.Set as S
 import qualified Data.Text as T
 
 cmd :: Command
-cmd = command "pre-commit" SectionPlumbing
+cmd = noCommit $ command "pre-commit" SectionPlumbing
 	"run by git pre-commit hook"
 	paramPaths
 	(withParams seek)
@@ -47,9 +49,14 @@ seek ps = do
 	-- committing changes to a view updates metadata
 	currentView >>= \case
 		Nothing -> noop
-		Just (v, _madj) -> withViewChanges
-			(addViewMetaData v)
-			(removeViewMetaData v)
+		Just (v, _madj) -> do
+			withViewChanges
+				(addViewMetaData v)
+				(removeViewMetaData v)
+			-- Manually commit in this case, because
+			-- noCommit prevents automatic commit.
+			whenM (annexAlwaysCommit <$> Annex.getGitConfig) $
+				Annex.Branch.commit =<< Annex.Branch.commitMessage
 
 addViewMetaData :: View -> ViewedFile -> Key -> CommandStart
 addViewMetaData v f k = starting "metadata" ai si $
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg.mdwn b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg.mdwn
index 37af943de4..44b9e7c603 100644
--- a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg.mdwn
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg.mdwn
@@ -134,3 +134,5 @@ Date:   Fri Feb 2 12:06:41 2024 -0500
 
 [[!meta author=yoh]]
 [[!tag projects/repronim]]
+
+> [[done]] --[[Joey]]
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_6_b8bb9e5c7156e4defd7a2e8b7777e1ba._comment b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_6_b8bb9e5c7156e4defd7a2e8b7777e1ba._comment
index 4aa985a935..257e7afd4f 100644
--- a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_6_b8bb9e5c7156e4defd7a2e8b7777e1ba._comment
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_6_b8bb9e5c7156e4defd7a2e8b7777e1ba._comment
@@ -3,16 +3,34 @@
  subject="""comment 6"""
  date="2024-02-12T17:36:32Z"
  content="""
-Turns out that the assistant doesn't commit to the git-annex branch itself,
-instead the pre-commit hook runs `git-annex pre-commit`, and 
-the git-annex branch commit on process shutdown is where the commit
-happens.
+I've implemented annex.commitmessage-command and made sure that the
+assistant calls it after committing the working tree changes.
 
-A bit surprising! If the pre-commit hook didn't run git-annex, 
-the assistant would later explicitly commit the branch before
-pushing to remotes.
+Example of using the prepare-commit-msg hook to generate a commit message
+and then reusing that message for the git-annex branch commit:
 
-Anyway, this does mean you can rely on the git-annex branch commit
-happening after the working tree commit. At least, when there are
-no other git-annex processes running.
+	joey@darkstar:~/tmp/a>cat .git/hooks/prepare-commit-msg
+	#!/bin/sh
+	msg="files: $(git diff --name-only --cached)"
+	echo $msg > .git/last-commit-msg
+	echo $msg > $1
+	
+	joey@darkstar:~/tmp/a>git config annex.commitmessage-command
+	cat .git/last-commit-msg
+	
+	joey@darkstar:~/tmp/a>git-annex assistant
+	joey@darkstar:~/tmp/a>date > bar
+
+	joey@darkstar:~/tmp/a>git log -n1 master
+	commit 676f87d02f031192c7eb0b7d29a6ce2429b8c727 (HEAD -> master, synced/master)
+	Author: Joey Hess <joeyh@joeyh.name>
+	Date:   Mon Feb 12 14:29:12 2024 -0400
+	
+	    files: bar
+	joey@darkstar:~/tmp/a>git log -n1 git-annex
+	commit 676588dd6d921115782e61be85f75e395cc480b6 (git-annex)
+	Author: Joey Hess <joeyh@joeyh.name>
+	Date:   Mon Feb 12 14:29:12 2024 -0400
+	
+	    files: bar
 """]]

added annex.commitmessage-command config
Sponsored-by: the NIH-funded NICEMAN (ReproNim TR&D3) project
diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index 8f927a78b4..42942d886f 100644
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -500,11 +500,19 @@ append jl f appendable toappend = do
 {- Commit message used when making a commit of whatever data has changed
  - to the git-annex branch. -}
 commitMessage :: Annex String
-commitMessage = fromMaybe "update" . annexCommitMessage <$> Annex.getGitConfig
+commitMessage = fromMaybe "update" <$> getCommitMessage
 
 {- Commit message used when creating the branch. -}
 createMessage :: Annex String
-createMessage = fromMaybe "branch created" . annexCommitMessage <$> Annex.getGitConfig
+createMessage = fromMaybe "branch created" <$> getCommitMessage
+
+getCommitMessage :: Annex (Maybe String)
+getCommitMessage = do
+	config <- Annex.getGitConfig
+	case annexCommitMessageCommand config of
+		Nothing -> return (annexCommitMessage config)
+		Just cmd -> catchDefaultIO (annexCommitMessage config) $
+			Just <$> liftIO (readProcess "sh" ["-c", cmd])
 
 {- Stages the journal, and commits staged changes to the branch. -}
 commit :: String -> Annex ()
diff --git a/CHANGELOG b/CHANGELOG
index 8057fb3c81..e4d178c408 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -11,6 +11,7 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
   * stack.yaml: Update to lts-22.9 and use crypton.
   * assistant, undo: When committing, let the usual git commit
     hooks run.
+  * Added annex.commitmessage-command config.
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Jan 2024 15:59:33 -0400
 
diff --git a/Types/GitConfig.hs b/Types/GitConfig.hs
index f9e0149ddb..f97ee52192 100644
--- a/Types/GitConfig.hs
+++ b/Types/GitConfig.hs
@@ -88,6 +88,7 @@ data GitConfig = GitConfig
 	, annexAlwaysCommit :: Bool
 	, annexAlwaysCompact :: Bool
 	, annexCommitMessage :: Maybe String
+	, annexCommitMessageCommand :: Maybe String
 	, annexMergeAnnexBranches :: Bool
 	, annexDelayAdd :: Maybe Int
 	, annexHttpHeaders :: [String]
@@ -176,6 +177,7 @@ extractGitConfig configsource r = GitConfig
 	, annexAlwaysCommit = getbool (annexConfig "alwayscommit") True
 	, annexAlwaysCompact = getbool (annexConfig "alwayscompact") True
 	, annexCommitMessage = getmaybe (annexConfig "commitmessage")
+	, annexCommitMessageCommand = getmaybe (annexConfig "commitmessage-command")
 	, annexMergeAnnexBranches = getbool (annexConfig "merge-annex-branches") True
 	, annexDelayAdd = getmayberead (annexConfig "delayadd")
 	, annexHttpHeaders = getlist (annexConfig "http-headers")
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index c4bd2b8df3..9d61e675a1 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -1088,6 +1088,11 @@ repository, using [[git-annex-config]]. See its man page for a list.)
   This works well in combination with annex.alwayscommit=false,
   to gather up a set of changes and commit them with a message you specify.
 
+* `annex.commitmessage-command`
+
+  This command is run and its output is used as the commit message to the
+  git-annex branch.
+
 * `annex.alwayscompact`
 
   By default, git-annex compacts data it records in the git-annex branch.
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_6_b8bb9e5c7156e4defd7a2e8b7777e1ba._comment b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_6_b8bb9e5c7156e4defd7a2e8b7777e1ba._comment
new file mode 100644
index 0000000000..4aa985a935
--- /dev/null
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_6_b8bb9e5c7156e4defd7a2e8b7777e1ba._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 6"""
+ date="2024-02-12T17:36:32Z"
+ content="""
+Turns out that the assistant doesn't commit to the git-annex branch itself,
+instead the pre-commit hook runs `git-annex pre-commit`, and 
+the git-annex branch commit on process shutdown is where the commit
+happens.
+
+A bit surprising! If the pre-commit hook didn't run git-annex, 
+the assistant would later explicitly commit the branch before
+pushing to remotes.
+
+Anyway, this does mean you can rely on the git-annex branch commit
+happening after the working tree commit. At least, when there are
+no other git-annex processes running.
+"""]]

comment
diff --git a/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_2_6b682415956756b969767f7de267f1f0._comment b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_2_6b682415956756b969767f7de267f1f0._comment
new file mode 100644
index 0000000000..081ca287dc
--- /dev/null
+++ b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_2_6b682415956756b969767f7de267f1f0._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2024-02-12T17:01:13Z"
+ content="""
+> Another note worthwhile making IMHO is that AFAIK those `git replace` markers
+> are local only, and whoever has unredacted-master later on might need to set
+> them up as well for their local clones to make such a \"collapse\" of histories
+
+Right, any repository you fetch unredacted-master into, you will also want to 
+fetch refs/redacted/ to as well, and run git replace there, as shown in the last
+code block of the tip above.
+"""]]

Added a comment
diff --git a/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_aece062800c6ed78988161d40e36bba4._comment b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_aece062800c6ed78988161d40e36bba4._comment
new file mode 100644
index 0000000000..e8af23f4c9
--- /dev/null
+++ b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_aece062800c6ed78988161d40e36bba4._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 1"
+ date="2024-02-11T18:54:45Z"
+ content="""
+Thank you!  In my case, since safe commit is too far in the past, what I had in mind is a little different: I wanted to have a completely disconnected history with a commit which had `$privatefile`s moved to annex, but I think the approach is \"the same\" in effect.
+The only thing I would do differently is to first convert files to git-annex in `master` (to become `unredacted-master`), so I end up with the same tree in `unredacted-master` and `master` happen that later I would need to cherry-pick some changes accidentally committed (e.g. by collaborators) on top of `unredacted-master`.
+
+Another note worthwhile making IMHO is that AFAIK those `git replace` markers are local only, and whoever has unredacted-master later on might need to set them up as well for their local clones to make such a \"collapse\" of histories.
+"""]]

removed
diff --git a/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_68e96510108ca47c80c8a9ac671c5878._comment b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_68e96510108ca47c80c8a9ac671c5878._comment
deleted file mode 100644
index bf3a6270ca..0000000000
--- a/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_68e96510108ca47c80c8a9ac671c5878._comment
+++ /dev/null
@@ -1,11 +0,0 @@
-[[!comment format=mdwn
- username="yarikoptic"
- avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
- subject="comment 1"
- date="2024-02-11T18:50:18Z"
- content="""
-Thank you!  In my case though it is pretty much the entire history of the What I had in mind is a little different and probably not really possible: I wanted to have a completely disconnected history with a commit which had `$privatefile`s but in effect you are doing the same so I think the approach is \"the same\" in effect.
-The only thing I would do differently is to first convert files to git-annex in `master` (to become `unredacted-master`), so I end up with the same tree in `unredacted-master` and `master` happen that later I would need to cherry-pick some changes accidentally committed (e.g. by collaborators) on top of `unredacted-master`.
-
-Another note worthwhile making IMHO is that AFAIK those `git replace` markers are local only, and whoever has unredacted-master later on might need to set them up as well for their local clones to make such a \"collapse\" of histories.
-"""]]

Added a comment
diff --git a/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_68e96510108ca47c80c8a9ac671c5878._comment b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_68e96510108ca47c80c8a9ac671c5878._comment
new file mode 100644
index 0000000000..bf3a6270ca
--- /dev/null
+++ b/doc/tips/redacting_history_by_converting_git_files_to_annexed/comment_1_68e96510108ca47c80c8a9ac671c5878._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 1"
+ date="2024-02-11T18:50:18Z"
+ content="""
+Thank you!  In my case though it is pretty much the entire history of the What I had in mind is a little different and probably not really possible: I wanted to have a completely disconnected history with a commit which had `$privatefile`s but in effect you are doing the same so I think the approach is \"the same\" in effect.
+The only thing I would do differently is to first convert files to git-annex in `master` (to become `unredacted-master`), so I end up with the same tree in `unredacted-master` and `master` happen that later I would need to cherry-pick some changes accidentally committed (e.g. by collaborators) on top of `unredacted-master`.
+
+Another note worthwhile making IMHO is that AFAIK those `git replace` markers are local only, and whoever has unredacted-master later on might need to set them up as well for their local clones to make such a \"collapse\" of histories.
+"""]]

top
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_5_d536cb5c2622d18802e6a32b03102f14._comment b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_5_d536cb5c2622d18802e6a32b03102f14._comment
new file mode 100644
index 0000000000..f992f50f2b
--- /dev/null
+++ b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_5_d536cb5c2622d18802e6a32b03102f14._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2024-02-10T16:49:33Z"
+ content="""
+Wrote up a tip with an even more polished version of that,
+[[tips/redacting_history_by_converting_git_files_to_annexed]]
+"""]]
diff --git a/doc/tips/redacting_history_by_converting_git_files_to_annexed.mdwn b/doc/tips/redacting_history_by_converting_git_files_to_annexed.mdwn
new file mode 100644
index 0000000000..5eca035138
--- /dev/null
+++ b/doc/tips/redacting_history_by_converting_git_files_to_annexed.mdwn
@@ -0,0 +1,53 @@
+git-annex can be used to let people clone a repository, without being
+able to access the content of annexed files whose content you want to keep
+private.
+
+But what to do if you're using a repository like that, and accidentially
+add a file to git that you intended to keep private? And you didn't notice
+for a while and made other changes.
+
+Here's a way to recover from that mistake without throwing away the commits
+you made. It creates a separate, redacted history where the private
+file (`$privatefile`) is an annexed file. And uses `git replace` to let you
+locally keep using the unredacted history.
+
+Start by identifiying the parent commit of the commit that added the
+private file to git (`$lastsafecommit`).
+
+Reset back to `$lastsafecommit` and squash in all changes made since then:
+
+    git branch unredacted-master master
+    git reset --hard $lastsafecommit
+    git merge --squash unredacted-master
+
+Then convert `$privatefile` to an annexed file:
+
+    git rm --cached $privatefile
+    git annex add --force-large $privatefile
+
+Commit the redacted version of master, and locally replace it with your
+original unredacted history.
+
+    git commit
+    git replace master unredacted-master
+
+Now you can push master to other remotes, and it will push the redacted
+form of master:
+
+    git push --force origin master
+
+(Note that, if you already pushed the unredacted commits to origin, this
+push will not necessarily completely delete the private content from it.
+Making a new origin repo and pushing to that is an easy way to be sure.)
+
+If you do want to share the unredacted history with any other repositories,
+you can, by fetching the replacement refs into them:
+
+    git fetch myhost:myrepo 'refs/replace/*'
+    git fetch myhost:myrepo unredacted-master
+    git replace master unredacted-master
+
+Note that the above example makes the redacted history contain a single
+squashed commit, but this technique is not limited to that. You can make
+redacted versions of individual commits too, and build up whatever form of
+redacted history you want to publish.

improved approach
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_73e37a27ae3ff6673f34d8b1a6bed4a0._comment b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_73e37a27ae3ff6673f34d8b1a6bed4a0._comment
new file mode 100644
index 0000000000..d4fb071421
--- /dev/null
+++ b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_73e37a27ae3ff6673f34d8b1a6bed4a0._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2024-02-10T16:06:26Z"
+ content="""
+A simpler approach is to make a redacted history, publish that, and locally
+replace the redacted history with the unredacted history.
+
+1. make a new empty bare repo (eg on github)
+2. git branch unredacted-master master
+3. git branch -D master
+4. start a new master branch at the commit before the problem file
+   was added, that contains one or more rewritten commits, where the problem
+   file is added to the annex (eg, use `git merge --squash
+   unredacted-master` and the convert the problem file to annexed before
+   committing)
+4. push the (new) master branch to the new bare repo
+5. `git replace master unredacted-master`
+
+That last step lets you locally access all your unredacted history locally,
+but pushes of further changes to master will still push the redacted history.
+
+You can do the same git replace in each existing clone of the repository
+and keep on referring to the unredacted history in those, while publishing
+the redacted history.
+"""]]

remove comment that doesn't actually work
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_f07514643cbc841f79135ed5db9fe022._comment b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_f07514643cbc841f79135ed5db9fe022._comment
deleted file mode 100644
index c32e36ca27..0000000000
--- a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_f07514643cbc841f79135ed5db9fe022._comment
+++ /dev/null
@@ -1,48 +0,0 @@
-[[!comment format=mdwn
- username="joey"
- subject="""comment 4"""
- date="2024-02-10T14:50:03Z"
- content="""
-Another way to use git replace, that is simpler, and requires less
-repository hacking:
-
-Checkout the commit just before that problem file was added:
-
-	git checkout 32358
-
-Squash merge the content of master into the working tree:
-
-	git merge --squash master
-
-Convert the problem file to an annexed file:
-
-	git rm --cached $file
-	git annex add --force-large $file
-
-Commit the result and replace the master branch with the new commit:
-
-	git commit -m 'rewritten history with $file converted to annexed'
-	git replace master HEAD
-
-Now you can checkout the master branch, and see the file is annexed there
-now:
-
-	git checkout master
-	git-annex find $file
-
-And finally, make one more commit, which can be empty, to have something
-new to force push to origin:
-
-	git commit --allow-empty -m 'empty commit after rewrite of history'
-	git push --force origin master
-
-Now a new clone from origin will get only the rewritten history, and there
-is no need to mess with replace refs, and no errors. You will still have
-access to the original history in your repo, and can interoperate with
-those clones. And origin will still contain the content of the problem file
-until you go in and delete it.
-
-Of course, it's also possible, rather than squashing the whole problematic
-history into one commit, to regenerate specific commits to use the annexed
-file.
-"""]]

update
diff --git a/doc/todo/verified_relaxed_urls.mdwn b/doc/todo/verified_relaxed_urls.mdwn
index a5aa18cce8..90e76cd09c 100644
--- a/doc/todo/verified_relaxed_urls.mdwn
+++ b/doc/todo/verified_relaxed_urls.mdwn
@@ -28,7 +28,8 @@ download from the web. (Call this a "dynamic" url key.)
 And handle all existing relaxed url keys as before.
 
 That would leave it up to the user to migrate their relaxed url keys to
-dynamic urls keys, if desired.
+dynamic urls keys, if desired. Now that distributed migration is
+implemented, that seems sufficiently easy.
 
 ## other special remotes
 
@@ -62,3 +63,5 @@ compared with migrating the key to the desired hash backend.
 
 Does seem to be some chance this could help implementing 
 [[wishlist_degraded_files]].
+
+[[!tag projects/datalad/potential]]

comment
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_f07514643cbc841f79135ed5db9fe022._comment b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_f07514643cbc841f79135ed5db9fe022._comment
new file mode 100644
index 0000000000..c32e36ca27
--- /dev/null
+++ b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_4_f07514643cbc841f79135ed5db9fe022._comment
@@ -0,0 +1,48 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2024-02-10T14:50:03Z"
+ content="""
+Another way to use git replace, that is simpler, and requires less
+repository hacking:
+
+Checkout the commit just before that problem file was added:
+
+	git checkout 32358
+
+Squash merge the content of master into the working tree:
+
+	git merge --squash master
+
+Convert the problem file to an annexed file:
+
+	git rm --cached $file
+	git annex add --force-large $file
+
+Commit the result and replace the master branch with the new commit:
+
+	git commit -m 'rewritten history with $file converted to annexed'
+	git replace master HEAD
+
+Now you can checkout the master branch, and see the file is annexed there
+now:
+
+	git checkout master
+	git-annex find $file
+
+And finally, make one more commit, which can be empty, to have something
+new to force push to origin:
+
+	git commit --allow-empty -m 'empty commit after rewrite of history'
+	git push --force origin master
+
+Now a new clone from origin will get only the rewritten history, and there
+is no need to mess with replace refs, and no errors. You will still have
+access to the original history in your repo, and can interoperate with
+those clones. And origin will still contain the content of the problem file
+until you go in and delete it.
+
+Of course, it's also possible, rather than squashing the whole problematic
+history into one commit, to regenerate specific commits to use the annexed
+file.
+"""]]

design document for verified relaxed urls
Sponsored-by: Graham Spencer on Patreon
diff --git a/doc/todo/verified_relaxed_urls.mdwn b/doc/todo/verified_relaxed_urls.mdwn
new file mode 100644
index 0000000000..a5aa18cce8
--- /dev/null
+++ b/doc/todo/verified_relaxed_urls.mdwn
@@ -0,0 +1,64 @@
+Adding a relaxed url currently prevents verifying the content of the object
+when transferring it between repositories. This risks data corruption going
+unnoticed. Could the hash be recorded when a relaxed url is downloaded from
+the web, and then used for verification of other transfers?
+
+This would need a new per-key file in the git-annex branch, that can list
+one or more hash-based keys. Then implement verification for url keys, that
+checks if hash-based keys are recorded in the file, and if so uses them to
+verify the content.
+
+The web special remote can hash the content as it's downloading it from the
+web, and record the resulting hash-based key.
+
+## handling upgrades
+
+A repository that currently contains the content of a relaxed url needs to
+keep working. So, the object stored in the repository has to be treated as
+valid, even though it does not correspond to any hash-based keys listed for
+the url key. 
+
+This presents a problem; there is no way to tell the difference between
+a valid object in such a repository and an object that was downloaded from
+the web with its hash recorded, but has since gotten corrupted.
+
+Seems that the only possible way to resolve this problem is to change to a
+new type of url key, that is known to always have its hash recorded on
+download from the web. (Call this a "dynamic" url key.)
+And handle all existing relaxed url keys as before.
+
+That would leave it up to the user to migrate their relaxed url keys to
+dynamic urls keys, if desired.
+
+## other special remotes
+
+If the web special remote is what takes care of hashing the content on
+download and recording the hash-based key, what about other special remotes
+that claim an url?
+
+This could also be implemented in the bittorrent special remote, but what
+about external special remotes?
+
+An alternative would be to add a downloadDynamicUrl that is called instead
+of retrieveKeyFile and returns a hash-based key (allowing hashing the
+download on the fly). Then git-annex would take
+care of recording the hash-based key. The external special remote interface
+could be extended to include that.
+
+## hash-based key choice
+
+Should annex.backend gitconfig be used to pick which hash-based key to use?
+The risk is that config changes and several different hash-based keys get
+recorded for a dynamic url. Not really a problem, but would increase the
+size of the git-annex branch unncessarily, and require extra work when
+verifying the key.)
+
+## use for other types of keys
+
+It would also be possible to use these new git-annex branch log files
+for other types of keys, like WORM, or perhaps SHA1. But, the same upgrade
+problem would apply. So, there's problably no benefit in doing that,
+compared with migrating the key to the desired hash backend.
+
+Does seem to be some chance this could help implementing 
+[[wishlist_degraded_files]].

close as distributed migration meets this use case
diff --git a/doc/todo/git-annex-migrate_using_git-replace.mdwn b/doc/todo/git-annex-migrate_using_git-replace.mdwn
index 8d3a857ac3..783b294541 100644
--- a/doc/todo/git-annex-migrate_using_git-replace.mdwn
+++ b/doc/todo/git-annex-migrate_using_git-replace.mdwn
@@ -3,3 +3,5 @@ Currently, git-annex-migrate leads to content (and metadata) being stored under
 More generally, git-annex-replace could be implemented this way, doing what git-replace does, but for git-annex keys rather than git hashes.  [[git-annex-pre-commit]] might need to be changed to implement replacement of keys added later.
 
 [[!tag needsthought]]
+
+> [[wontfix|done]], use distributed migration instead --[[Joey]]
diff --git a/doc/todo/git-annex-migrate_using_git-replace/comment_2_e760213fc51f6a2cfc68dec711622f25._comment b/doc/todo/git-annex-migrate_using_git-replace/comment_2_e760213fc51f6a2cfc68dec711622f25._comment
new file mode 100644
index 0000000000..9e4b0be36c
--- /dev/null
+++ b/doc/todo/git-annex-migrate_using_git-replace/comment_2_e760213fc51f6a2cfc68dec711622f25._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2024-02-10T14:22:31Z"
+ content="""
+In the meantime, distributed migration has been implemented, allowing
+to `git-annex migrate --update` to catch up with all migrations that were
+started elsewhere.
+
+This is essentially the same goal, and it is much more efficient in terms
+of storage than git-replace of a large number of refs.
+
+So, I'm going to close this.
+"""]]

Added a comment: Update: workaround with branches
diff --git a/doc/forum/Preserving_Directories_in_Metadata_Views/comment_2_0decf76e18bb9fbc9b55e94f647a466d._comment b/doc/forum/Preserving_Directories_in_Metadata_Views/comment_2_0decf76e18bb9fbc9b55e94f647a466d._comment
new file mode 100644
index 0000000000..7289595c72
--- /dev/null
+++ b/doc/forum/Preserving_Directories_in_Metadata_Views/comment_2_0decf76e18bb9fbc9b55e94f647a466d._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="psxvoid"
+ avatar="http://cdn.libravatar.org/avatar/fde068fbdeabeea31e3be7aa9c55d84b"
+ subject="Update: workaround with branches"
+ date="2024-02-10T10:53:56Z"
+ content="""
+Seems like there is a way to use git annex as I would like, though it may not be obvious.
+
+In order to organize files and then burn them on to BDXL, instead of using tags, I've just switched to a new branch.
+e.g.:
+
+```sh
+git checkout -b view/bdxl
+```
+
+and then moved files and folders to a new folder which I intend to burn later. I've also assigned `git annex tag bdxl-disk-1 . --force` directly in this folder, so later I can see which files are burned even on the main branch (because tags are tied to git annex content and shared across branches).
+
+P.S.: In any case, preserving the folder structure on the first `git annex view tag=*` may be much more powerful and intuitive.
+"""]]

Added a comment: This behavior should be the default one
diff --git a/doc/forum/Preserving_Directories_in_Metadata_Views/comment_1_30332d3828324455600b0b452e3b6c5a._comment b/doc/forum/Preserving_Directories_in_Metadata_Views/comment_1_30332d3828324455600b0b452e3b6c5a._comment
new file mode 100644
index 0000000000..f07a32715b
--- /dev/null
+++ b/doc/forum/Preserving_Directories_in_Metadata_Views/comment_1_30332d3828324455600b0b452e3b6c5a._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="psxvoid"
+ avatar="http://cdn.libravatar.org/avatar/fde068fbdeabeea31e3be7aa9c55d84b"
+ subject="This behavior should be the default one"
+ date="2024-02-10T08:11:30Z"
+ content="""
+This behavior needs to be the default. Though, I understand that everyone have different needs.
+
+At least it may be better to configure the behavior via a command argument, e.g. by default `git annex view` preserves the directory structure.
+
+And when `--plain` argument is provided, then it would behave as it currently does - the directory structure is not preserved.
+
+Personally, I do not see any use cases for the active behavior because usually I set tags to many files (thousands), and it's not practical for me to see thousand files in a single directory (view tag). As a more real-world example - I've tried to prepare files to be burned into BDXL disk, and for that purpose I tagged the files I want to burn as `bdxl-disk-1` tag. I then used `ncdu` to analyze the space taken by that tag. But then when I started \"fine tuning\" what should and should not go to that disk, I found it too difficult because `git annex view` places all tagged files into a single directory.
+
+Are there any workarounds? I'm using git annex version `10.20231228`.
+"""]]

Added a comment
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_3_4cff971451dd7038ffe8f7f2fa3b28c9._comment b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_3_4cff971451dd7038ffe8f7f2fa3b28c9._comment
new file mode 100644
index 0000000000..0ab0b04460
--- /dev/null
+++ b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_3_4cff971451dd7038ffe8f7f2fa3b28c9._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 3"
+ date="2024-02-09T21:58:56Z"
+ content="""
+Thank you!  Yet to try/follow the steps. FWIW -- we have not pushed anything to github yet, so we have everything only locally and thus should potentially be able to not reveal older objects.
+"""]]

comment
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_2_aab850db06cbd3c33067cff567d2699e._comment b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_2_aab850db06cbd3c33067cff567d2699e._comment
new file mode 100644
index 0000000000..9aa0709a5a
--- /dev/null
+++ b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_2_aab850db06cbd3c33067cff567d2699e._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2024-02-09T21:02:09Z"
+ content="""
+Oh and I doubt github lets you delete .git/object files from a repo hosted
+there, so you might need to delete the repo and re-create it.
+
+Given how github shares git object stores amoung forks, even that might not
+be enough to eliminate the objects from it.
+"""]]

comment
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_1_345f9a32f5fc1ea21af9fe18f902d1b0._comment b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_1_345f9a32f5fc1ea21af9fe18f902d1b0._comment
new file mode 100644
index 0000000000..296d534232
--- /dev/null
+++ b/doc/forum/move_files_under_git-annex_and_graft_history__63__/comment_1_345f9a32f5fc1ea21af9fe18f902d1b0._comment
@@ -0,0 +1,78 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-09T20:01:01Z"
+ content="""
+Well `git replace` does turn out to be able to do this. With some caveats.
+
+I started with this history:
+
+	6df78e79d688de8392d952dc8705d094384ed2c0 add file (accidentially to git)
+	3235872149f1767efd282635ca12ec2f9056f9cc initial commit
+
+Then amended 6df78e, following the usual
+process to convert git to annexed, as outlined in [[tips/largefiles]],
+producing commit f36b42e.
+
+Then `git replace 6df78e f36b42e` and the log changed to:
+
+	6df78e79d688de8392d952dc8705d094384ed2c0 (replaced) add file (to annex)
+	3235872149f1767efd282635ca12ec2f9056f9cc initial commit
+
+Note that, when there are further commits made on top of
+the bad commit, they all would need to be replaced with amended commits
+as well.
+
+I'd already pushed to origin before this. And origin still showed the bad
+commit in its log. But pushing the replace refs fixed that:
+
+	git push origin 'refs/replace/*'
+
+Now looking at the origin repo showed the same amended history.
+
+In another clone, the old history is still visible, but that can be fixed, by running:
+
+	git fetch origin 'refs/replace/*:refs/replace/*'
+
+Then I deleted the git object for 6df78e from the origin repo, along with the
+blob object that contained the content of the file accidentially added to git.
+
+This is where it fell down, because cloning from that origin repo then
+fails:
+
+	joey@darkstar:~/tmp/test>git clone demo.git/ democlone
+	Cloning into 'democlone'...
+	done.
+	fatal: update_ref failed for ref 'HEAD': cannot update ref 'refs/heads/master': trying to write ref 'refs/heads/master' with nonexistent object 6df78e79d688de8392d952dc8705d094384ed2c0
+
+But, that can be worked around. Just need to make an additional commit on top of the
+replaced commit, so git clone will see a commit that has not been deleted. I did that,
+and:
+
+	joey@darkstar:~/tmp/test>git clone demo.git/ democlone
+	Cloning into 'democlone'...
+	done.
+	joey@darkstar:~/tmp/test>cd democlone
+	joey@darkstar:~/tmp/test/democlone>git log
+	error: Could not read 6df78e79d688de8392d952dc8705d094384ed2c0
+	fatal: Failed to traverse parents of commit 167b6e6842a669bc90925dd4b6df0c1a6ec4c141
+
+	joey@darkstar:~/tmp/test/democlone>git fetch origin 'refs/replace/*:refs/replace/*'
+	From /home/joey/tmp/test/demo
+	 * [new ref]         refs/replace/6df78e79d688de8392d952dc8705d094384ed2c0 -> refs/replace/6df78e79d688de8392d952dc8705d094384ed2c0
+	joey@darkstar:~/tmp/test/democlone>git log --pretty=oneline
+	167b6e6842a669bc90925dd4b6df0c1a6ec4c141 (HEAD -> master, origin/master, origin/HEAD) empty
+	6df78e79d688de8392d952dc8705d094384ed2c0 (replaced) add file (to annex)
+	3235872149f1767efd282635ca12ec2f9056f9cc initial commit
+
+So, you can do this if you're ok with clones needing to manually fetch the replace refs
+in order to access the replaced history.
+
+And of course, existing clones need to be manually updated to fetch the replace refs.
+And probably ought to have the bad objects deleted out of their .git/objects/
+to avoid accidental data leakage.
+
+I'd also caution that, if the history you rewrite with `git replace` contains a lot
+of commits, the number of refs in refs/replace/* could get large, and a large number
+of git refs can be innefficient in various ways.
+"""]]

comment
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_5_c7a888605dd063fe7df7c569f1193f8c._comment b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_5_c7a888605dd063fe7df7c569f1193f8c._comment
new file mode 100644
index 0000000000..6b250668f9
--- /dev/null
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_5_c7a888605dd063fe7df7c569f1193f8c._comment
@@ -0,0 +1,36 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2024-02-09T17:33:10Z"
+ content="""
+I think this needs two things:
+
+1. A hook to get the commit message when committing to
+   the git-annex branch. Apparently none of the git hooks
+   are well suited to be reused for that.
+2. Ensure that the assistant always waits to commit the git-annex branch
+   until after it's committed changes to the working tree.
+
+Then you could do a variety of things...
+
+You could make the git-annex branch commit hook look for the last
+change committed to the master branch, and reuse its commit message.
+(If it's sufficiently close to the present that it's probably for
+the same change.)
+
+You could use the git commit hook to generate a git commit message but also
+store the message or a list of files somewhere for later use in the
+git-annex branch commit hook.
+
+You could write a description of your current task to a file somewhere,
+and have both hooks use that as their commit message. (Note that this does
+not need #2 above actually, only needs the hook.)
+
+---
+
+Does the assistant always wait to commit to the git-annex branch until
+after committing changes to the working tree? I'm unsure. A quick test
+indiciated it usually does, but I'm not sure why or if that's always the
+case. It would take some analysis and making this behavior more explicit to
+be able to rely on it.
+"""]]

initial question on grafting the history when we have sensitive data
diff --git a/doc/forum/move_files_under_git-annex_and_graft_history__63__.mdwn b/doc/forum/move_files_under_git-annex_and_graft_history__63__.mdwn
new file mode 100644
index 0000000000..d2520e1429
--- /dev/null
+++ b/doc/forum/move_files_under_git-annex_and_graft_history__63__.mdwn
@@ -0,0 +1,15 @@
+I do not know yet if that it possible since I have only cursory knowledge of former grafts in git and what new came to replace them.
+
+The use case:  we have a git-annex repo (heudiconv converted MRI data) where some files with sensitive information were added directly to git not git-annex.  We do not want to rewrite the entire history since that repository already saw a good number of clones and forks.  I wondered if we could
+
+1. move those files under git-annex
+2. establish a new history from that new tree object
+3. "graft" new history commit into the old "sensitive" one as to establish correspondence between the two
+   - I would expect then something like "git pull --ff-only" to work for people seamlessly jumping between those points
+4. git push only new history to github thus not revealing old  history with sensitive data under git
+
+
+Joey, WDYT? any words of wisdom?
+
+[[!meta author=yoh]]
+[[!tag projects/repronim?]]

Added a comment
diff --git a/doc/todo/addurl_--raw-except_REMOTEs__63__/comment_2_2cd05802f25c9868b1b975dd359b66b4._comment b/doc/todo/addurl_--raw-except_REMOTEs__63__/comment_2_2cd05802f25c9868b1b975dd359b66b4._comment
new file mode 100644
index 0000000000..0886468b92
--- /dev/null
+++ b/doc/todo/addurl_--raw-except_REMOTEs__63__/comment_2_2cd05802f25c9868b1b975dd359b66b4._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2024-02-09T13:50:21Z"
+ content="""
+fixed in [10.20240129-42-g6b38d0c427](https://git.kitenet.net/index.cgi/git-annex.git/commit/?id=6b38d0c427a645d421ebf2d21111c0ca224dfef3)
+"""]]

Added a comment
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_4_203a356c1edfaf8c0c9cf01bedc2ec5d._comment b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_4_203a356c1edfaf8c0c9cf01bedc2ec5d._comment
new file mode 100644
index 0000000000..085c25fdfd
--- /dev/null
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_4_203a356c1edfaf8c0c9cf01bedc2ec5d._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 4"
+ date="2024-02-08T22:08:41Z"
+ content="""
+Thank you Joey!!!  Re
+
+> so you might commit with \"added 57 photos\" to the master branch, and then use the same message for the git-annex branch commit. annex.commitmessage seems to already cover that case though?
+
+how to implement it using that config setting for `git annex assistant` mode of operation when we do not have some kind of a wrapper helper and only hooks?
+
+"""]]

update
diff --git a/doc/thanks.mdwn b/doc/thanks.mdwn
index 6ab5cf9fff..9240246c72 100644
--- a/doc/thanks.mdwn
+++ b/doc/thanks.mdwn
@@ -243,7 +243,7 @@ Sherif Abouseda, Ben Strawbridge, chee rabbits, Pedro Côrte-Real
 And special thanks to Kevin McKenzie, who also gave me a login to a Mac OSX
 machine, which has proven invaluable, Jimmy Tang who has helped
 with Mac OSX autobuilding and packaging, and Yury V. Zaytsev who
-provides the Windows autobuilder.
+provided the Windows autobuilder.
 
 ### Other Backers
 

close
diff --git a/doc/todo/documentation__58___improve_on_special_remotes.mdwn b/doc/todo/documentation__58___improve_on_special_remotes.mdwn
index ed44caec40..703689702f 100644
--- a/doc/todo/documentation__58___improve_on_special_remotes.mdwn
+++ b/doc/todo/documentation__58___improve_on_special_remotes.mdwn
@@ -7,3 +7,6 @@ Without such a documentation it is hard to "on board" new git-annex users and de
 
 [[!meta author=yoh]]
 [[!tag projects/dandi]]
+
+> Hard to say that documentation is ever done, but I've made some
+> improvements and I guess am going to call this [[done]] --[[Joey]]

assistant, undo: When committing, let the usual git commit hooks run
Was doing a Git.Branch.commit for historical reasons to do with direct
mode, which no longer apply.
Note that the preCommitAnnexHook is no longer called in commitStaged
because git-annex installs a pre-commit hook that runs the pre-commit-annex
hook. And git commit will run the pre-commit hook.
Sponsored-by: the NIH-funded NICEMAN (ReproNim TR&D3) project
diff --git a/CHANGELOG b/CHANGELOG
index 77f57c0c75..8057fb3c81 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -9,6 +9,8 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
     already downloaded files when yt-dlp or a special remote was used.
   * addurl, importfeed: Added --raw-except option.
   * stack.yaml: Update to lts-22.9 and use crypton.
+  * assistant, undo: When committing, let the usual git commit
+    hooks run.
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Jan 2024 15:59:33 -0400
 
diff --git a/Command/Sync.hs b/Command/Sync.hs
index c4cbb0ccd1..52fe4c4aa0 100644
--- a/Command/Sync.hs
+++ b/Command/Sync.hs
@@ -36,7 +36,6 @@ import qualified Annex
 import qualified Annex.Branch
 import qualified Remote
 import qualified Types.Remote as Remote
-import Annex.Hook
 import qualified Git.Command
 import qualified Git.LsFiles as LsFiles
 import qualified Git.Branch
@@ -431,15 +430,12 @@ commitMsg = do
 	return $ "git-annex in " ++ maybe "unknown" fromUUIDDesc (M.lookup u m)
 
 commitStaged :: Git.Branch.CommitMode -> String -> Annex Bool
-commitStaged commitmode commitmessage = do
-	runAnnexHook preCommitAnnexHook
-	mb <- inRepo Git.Branch.currentUnsafe
-	let (getparent, branch) = case mb of
-		Just b -> (Git.Ref.sha b, b)
-		Nothing -> (Git.Ref.headSha, Git.Ref.headRef)
-	parents <- maybeToList <$> inRepo getparent
-	void $ inRepo $ Git.Branch.commit commitmode False commitmessage branch parents
-	return True
+commitStaged commitmode commitmessage =
+	inRepo $ Git.Branch.commitCommand commitmode
+		(Git.Branch.CommitQuiet True)
+		[ Param "-m"
+		, Param commitmessage
+		]
 
 mergeLocal :: [Git.Merge.MergeConfig] -> SyncOptions -> CurrBranch -> CommandStart
 mergeLocal mergeconfig o currbranch = stopUnless (notOnlyAnnex o) $
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_1_e7f759dccbfa134a772f15bc5f9c4b29._comment b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_1_e7f759dccbfa134a772f15bc5f9c4b29._comment
new file mode 100644
index 0000000000..0a5ff2e239
--- /dev/null
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_1_e7f759dccbfa134a772f15bc5f9c4b29._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-07T18:59:34Z"
+ content="""
+Re the default messages, let's just say that any choice git-annex makes
+about a default commit message will not be to some people's liking.
+
+Re the hook not being run, this is due to it using `git commit-tree`.
+As well as being used for commits to the git-annex branch, it's also
+used in a few other places, including `git-annex import`, and
+generating adjusted branches and view branches, and the assistant
+and `git-annex undo`.
+"""]]
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_2_acd656bf30f7e0b5def7ad7a564cf914._comment b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_2_acd656bf30f7e0b5def7ad7a564cf914._comment
new file mode 100644
index 0000000000..bf511cc113
--- /dev/null
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_2_acd656bf30f7e0b5def7ad7a564cf914._comment
@@ -0,0 +1,39 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2024-02-07T19:09:44Z"
+ content="""
+Git has two hooks, prepare-commit-msg and commit-msg. prepare-commit-msg
+is used prior to interactive editing of the commit message. Since these
+commits do not involve interactive editing, I don't think it
+necesarily makes sense for git-annex to use that hook.
+
+The commit-msg hook is simpler, and it would certianly be possible for
+git-annex to call it everywhere it does a `git commit-tree`.
+
+But if git-annex called either of these hooks, how is the hook supposed to
+know what is being committed? In the usual case, the hook 
+is called with `GIT_INDEX_FILE` pointed at an index file that 
+has been modified to include whatever is going to be in the commit
+(which may be different than what is staged in the index), and it knows
+that the working directory is being committed. So it can
+do things like `git diff --name-stage --cached` to get the files that
+are being committed, for example.
+
+In the git-annex case, `GIT_INDEX_FILE` is pointed at .git/annex/index
+when committing the git-annex branch. If a commit-msg hook uses
+`git diff --name-stage --cached`, it will diff from the working directory
+to new git-annex branch contents, which will not be a useful set of file changes
+to summarize in a commit message!
+
+Even if the hook somehow knows that the git-annex branch is being
+committed, the best it could do is diff from git-annex branch to
+`GIT_INDEX_FILE` and then it has a bunch of git-annex branch filenames
+that have changed, and is supposed to somehow generate a useful commit
+message from that. How?
+
+It seems more likely to me that you know that git-annex branch changes are
+associated with an operation, so you might commit with "added 57 photos"
+to the master branch, and then use the same message for the git-annex
+branch commit. annex.commitmessage seems to already cover that case though?
+"""]]
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_3_0f49414f9b4a68a16a8d47a336242f13._comment b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_3_0f49414f9b4a68a16a8d47a336242f13._comment
new file mode 100644
index 0000000000..1fe2228d48
--- /dev/null
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg/comment_3_0f49414f9b4a68a16a8d47a336242f13._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2024-02-07T19:58:11Z"
+ content="""
+Irrespective of comment #2, I do think that it makes sense for the
+assistant and `git-annex undo`, when committing to the HEAD branch,
+to run all the usual commit hooks.
+
+Actually, I don't see any reason for those to not run `git commit`.
+It seems that the use of `git commit-tree` in those dates back to 
+[[!commit 03932212ecb8940e0fe2dd09c04951990bc29422]] which involved
+direct mode.
+
+Fixed those to run commits in the usual way (though noninteractive).
+"""]]

Added a comment
diff --git a/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_4_46ad0779ff71cfbadd663fe413d32ffb._comment b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_4_46ad0779ff71cfbadd663fe413d32ffb._comment
new file mode 100644
index 0000000000..82faf99196
--- /dev/null
+++ b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_4_46ad0779ff71cfbadd663fe413d32ffb._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 4"
+ date="2024-02-07T18:47:19Z"
+ content="""
+> `git-annex registerurl --move-from=foo $key`
+
+looks good
+
+> Since registering an url also marks it present, all you would need after that is to mark it as not present any longer on your special remote.
+
+it is `--move-from`, not `--copy-from`, so wouldn't it mark it not present any longer on `foo`?
+"""]]

Added a comment
diff --git a/doc/todo/add_--json-progress_support_in_push_and_pull/comment_3_a11ee71f4ee907b011aebcfe270b7ee3._comment b/doc/todo/add_--json-progress_support_in_push_and_pull/comment_3_a11ee71f4ee907b011aebcfe270b7ee3._comment
new file mode 100644
index 0000000000..aec53b9cf7
--- /dev/null
+++ b/doc/todo/add_--json-progress_support_in_push_and_pull/comment_3_a11ee71f4ee907b011aebcfe270b7ee3._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="jstritch"
+ avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
+ subject="comment 3"
+ date="2024-02-07T15:52:18Z"
+ content="""
+Additional clarification of my posts above:
+
+1) JSON allows optional fields. That is, fields which are only present when needed. I am specifically saying not to include all fields every time with `null` values as needed.
+
+2) Occasionally tacking the remote name onto the end of a string named `command` is not discoverable. The `command` key would have to be renamed to something like `commandAndSometimesRemote` to be discoverable. jstritch's naming rule #5: If a name needs a conjunction to accurately describe it, the design can be improved.
+
+3) The optional field name is more easily observed than the optional string content.
+
+4) Consider the application code consuming the JSON. A test for the presence of the remote name is required either way. Do you want those applications writing the test \"if the command value contains one space\" or \"if the remote key is present\"? Code is written once and read hundreds of times. The second test conveys the intent, reducing maintenance cost.
+
+5) The application code to deal with splitting the string and handling each part becomes unnecessary with the optional field.
+
+6) The documentation of the JSON could include a matrix showing the key name and its data type versus the commands, similar to a feature comparison table.
+
+7) Documenting the JSON does not make it less discoverable.
+
+I hope you find this information helpful to improve the end-user experience. Let me know if you have any questions.
+"""]]

Added a comment
diff --git a/doc/bugs/identically_configured_remotes_behave_differently/comment_1_d69d6a014c7265fb901dc407516e232b._comment b/doc/bugs/identically_configured_remotes_behave_differently/comment_1_d69d6a014c7265fb901dc407516e232b._comment
new file mode 100644
index 0000000000..2677b04dab
--- /dev/null
+++ b/doc/bugs/identically_configured_remotes_behave_differently/comment_1_d69d6a014c7265fb901dc407516e232b._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="jstritch"
+ avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
+ subject="comment 1"
+ date="2024-02-06T19:54:40Z"
+ content="""
+This is possibly another manifestation of `pull doesn't work on adjusted branches`. I'll await your reply to it.
+"""]]

Added a comment
diff --git a/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches/comment_2_632a3616516605de477930ae09d4ca60._comment b/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches/comment_2_632a3616516605de477930ae09d4ca60._comment
new file mode 100644
index 0000000000..a2c32b66c2
--- /dev/null
+++ b/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches/comment_2_632a3616516605de477930ae09d4ca60._comment
@@ -0,0 +1,131 @@
+[[!comment format=mdwn
+ username="jstritch"
+ avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
+ subject="comment 2"
+ date="2024-02-06T18:27:33Z"
+ content="""
+If I commit a change, setting annex.adjustedbranchrefresh to true is not working as I *expected*. I want to pull to one specified remote only. I do not want to sync with any other repositories or special remotes that may happen to be available at the moment. This is possibly another manifestation of the issue I submitted about export not copying.
+
+The `git annex sync` command is not happy with `here` as a remote name.
+
+I see the same result if I change both `git annex adjust` from `--unlock` to `--lock`. If both adjust statements are commented out, pull-repo1 ends up with a broken link to the second file. The workflow should be the same for all cases.
+
+Here's a script reproducing what I have:
+[[!format sh \"\"\"
+mkdir pull-repo1
+mkdir pull-repo2
+
+cd pull-repo1
+git init --initial-branch main
+git annex init pull-repo1
+git config annex.adjustedbranchrefresh true
+git config receive.denyCurrentBranch updateInstead
+git annex config --set annex.largefiles include=*.xtx
+git annex group here manual
+git annex wanted here standard
+echo \"test file one\" >> test-file-one.xtx
+git annex add .
+git commit -m \"add test file one\"
+git annex adjust --unlock
+
+cd ../pull-repo2
+git clone --origin pull-repo1 ../pull-repo1 .
+git annex init pull-repo2
+cd ../pull-repo1
+git remote add pull-repo2 ../pull-repo2
+git annex push pull-repo2
+
+cd ../pull-repo2
+git config annex.adjustedbranchrefresh true
+git config receive.denyCurrentBranch updateInstead
+git annex config --set annex.largefiles include=*.xtx
+git annex group here manual
+git annex wanted here standard
+git annex adjust --unlock
+echo \"test file two\" >> test-file-two.xtx
+git annex add .
+git commit -m \"add test file two\"
+
+cd ../pull-repo1
+git annex pull --allow-unrelated-histories --content pull-repo2
+git annex list
+\"\"\"]]
+
+And here's its output:
+[[!format sh \"\"\"
+Initialized empty Git repository in /home/juanito/Documents/pull-repo1/.git/
+init pull-repo1 ok
+(recording state in git...)
+annex.largefiles include=*.xtx ok
+(recording state in git...)
+group here ok
+(recording state in git...)
+wanted here ok
+(recording state in git...)
+add test-file-one.xtx 
+ok                                
+(recording state in git...)
+[main (root-commit) 6eff4e8] add test file one
+ 1 file changed, 1 insertion(+)
+ create mode 120000 test-file-one.xtx
+adjust  
+Switched to branch 'adjusted/main(unlocked)'
+ok
+Cloning into '.'...
+done.
+init pull-repo2 (merging pull-repo1/git-annex into git-annex...)
+(recording state in git...)
+ok
+(recording state in git...)
+copy test-file-one.xtx (to pull-repo2...) 
+(recording state in git...)       
+ok
+(recording state in git...)
+push pull-repo2 
+Enumerating objects: 9, done.
+Counting objects: 100% (9/9), done.
+Delta compression using up to 12 threads
+Compressing objects: 100% (4/4), done.
+Writing objects: 100% (5/5), 426 bytes | 426.00 KiB/s, done.
+Total 5 (delta 2), reused 0 (delta 0), pack-reused 0
+To ../pull-repo2
+ * [new branch]      main -> synced/main
+ * [new branch]      git-annex -> synced/git-annex
+ok
+annex.largefiles include=*.xtx (merging synced/git-annex into git-annex...)
+(recording state in git...)
+ok
+group here ok
+(recording state in git...)
+wanted here ok
+(recording state in git...)
+adjust  ok
+add test-file-two.xtx 
+ok                                
+(recording state in git...)
+[adjusted/main(unlocked) 57a9dab] add test file two
+ 1 file changed, 1 insertion(+)
+ create mode 100644 test-file-two.xtx
+pull pull-repo2 
+remote: Enumerating objects: 32, done.
+remote: Counting objects: 100% (29/29), done.
+remote: Compressing objects: 100% (21/21), done.
+remote: Total 23 (delta 8), reused 0 (delta 0), pack-reused 0
+Unpacking objects: 100% (23/23), 2.24 KiB | 1.12 MiB/s, done.
+From ../pull-repo2
+ * [new branch]      adjusted/main(unlocked) -> pull-repo2/adjusted/main(unlocked)
+ * [new branch]      git-annex  -> pull-repo2/git-annex
+ok
+(merging pull-repo2/git-annex into git-annex...)
+here
+|pull-repo2
+||web
+|||bittorrent
+||||
+XX__ test-file-one.xtx
+\"\"\"]]
+
+Please observe test-file-two.xtx is not listed and also not in the first repository.
+
+How should I proceed?
+"""]]

Added a comment
diff --git a/doc/todo/add_--json-progress_support_in_push_and_pull/comment_2_ff0b1ca16b03d2e6c8331645adad84a6._comment b/doc/todo/add_--json-progress_support_in_push_and_pull/comment_2_ff0b1ca16b03d2e6c8331645adad84a6._comment
new file mode 100644
index 0000000000..73609e31df
--- /dev/null
+++ b/doc/todo/add_--json-progress_support_in_push_and_pull/comment_2_ff0b1ca16b03d2e6c8331645adad84a6._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="jstritch"
+ avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
+ subject="comment 2"
+ date="2024-02-06T16:10:56Z"
+ content="""
+To me, the purpose of any progress message is twofold: 1) let the user know the program is not stuck and 2) provide information about what was happening if it does get stuck. Push and pull do this now without JSON.
+
+I agree the same JSON progress object format should be used for all commands. Currently the action progress information includes: command, file, input, byte-progress, total-size, and percent-progress.
+
+The calling application is able to \"know\" which command it issued to cause the progress reports.
+
+A separate, optional field could be added for the particular remote involved. For example, messages like \"Copy file A to remote-one X% complete\" and \"Export file B to remote-two X% complete\" should be possible.
+
+I'm in favor of documenting data structures. Forcing developers to reverse-engineer the structure is inefficient at best. While the principle that all git-annex json output should be discoverable by simply running the command sounds good, I find it leaves much to be desired. Just to find out what's available, someone must setup and run the command. I suggest updating the requirement to \"All git-annex JSON output objects are documented.\"
+
+Users will get impatient and grow frustrated without some type of indication work is in progress.
+
+I'm OK with leaving Git operations silent for now.
+
+Please improve the end-user experience by devising a way to inform the user git-annex is making progress with JSON.
+"""]]

Added a comment: centralized repository workflow
diff --git a/doc/tips/centralized_git_repository_tutorial/on_your_own_server/comment_9_915965b46b03186eec466c51131c734c._comment b/doc/tips/centralized_git_repository_tutorial/on_your_own_server/comment_9_915965b46b03186eec466c51131c734c._comment
new file mode 100644
index 0000000000..6dfad4987e
--- /dev/null
+++ b/doc/tips/centralized_git_repository_tutorial/on_your_own_server/comment_9_915965b46b03186eec466c51131c734c._comment
@@ -0,0 +1,50 @@
+[[!comment format=mdwn
+ username="zoran.bosnjak@683708e9d46ca9d9fa8957bd513e6648cbcbf421"
+ nickname="zoran.bosnjak"
+ avatar="http://cdn.libravatar.org/avatar/83221ce6eaa0b62d27a6507c96a61fae"
+ subject="centralized repository workflow"
+ date="2024-02-06T12:33:20Z"
+ content="""
+Hi, what would be a recommended setup and the working procedures for the following scenario:
+
+- using git-annex version: 8.20210223, which is the one in ubuntu-22.04 (can't upgrade easily)
+- a central server as a mutable central archive, many users (over ssh)
+- users are all trusted
+- the server shall keep all annexed files, but only the HEAD version is relevant, that is:
+  if the file is removed by the user, it shall eventually be permanently removed from the central server too,
+  to save space.
+- users would tipically not need all the files, but only some, so `git annex get files...` would do
+- users would also add or remove annexed files (and push them to the central repository)
+- a user might remove his/her local repository at any time, so the central server shall not keep track about clones
+  or at least shall not care if any or all clones get removed
+
+I have created central repository like this (please correct me):
+
+    git init test --bare                                                                                       
+    cd test                                                                                                    
+    git annex init                                                                                             
+    git annex required . \"include=*\" 
+
+On the user site
+
+    git clone ssh://some.server/repo/test test
+    cd test
+    dd if=/dev/random of=./bigfile bs=1M count=10
+    git annex add bigfile
+    # how to sync (push only)?
+    # how to permanently remove big file?
+    cd ..
+    # done with the task
+    chmod -R 777 test
+    rm -rf test
+
+What I am looking for is the sequence of commands for the users, to:
+
+- sync to the latest state (without fetching the content)
+- add new annexed file to the repository and push it
+- permanently delete annexed file
+
+There are several issues I am facing at the moment.
+I was expecting to push the new file with `git annex sync --content --no-pull`, but this command also pulls the contents of all annexd files, which I don't want. The server does not want to remove the old content. It looks like I am doing something wrong. Appreciate your suggestions about this scenario.
+
+"""]]

diff --git a/doc/forum/agent_keep_deleting_local_file.mdwn b/doc/forum/agent_keep_deleting_local_file.mdwn
new file mode 100644
index 0000000000..01fc9f365f
--- /dev/null
+++ b/doc/forum/agent_keep_deleting_local_file.mdwn
@@ -0,0 +1,24 @@
+I realize this is git 101, and I apologize in advance, but I honestly have no idea how to debug this.
+
+I have two Macs, with git annex assistant running headless on both. I renamed a file on one system somewhere along the line, and now whenever the assistant syncs changes from system B to system A, I always find myself in a situation like this:
+
+```
+On branch master
+Your branch is behind 'mbp/master' by 1 commit, and can be fast-forwarded.
+  (use "git pull" to update your local branch)
+
+Changes not staged for commit:
+  (use "git add/rm <file>..." to update what will be committed)
+  (use "git restore <file>..." to discard changes in working directory)
+	deleted:    myfile.md
+```
+
+This file was originally myfile-1234.md or whatever, but I renamed it on system A, and those changes were synced. Today, on system B, that file exists as myfile.md as I'd expect. On system A, however, it always gets deleted whenever I make changes made to any file on system B that the assistant then syncs over.
+
+I've tried re-adding the file and committing, but it continues to be deleted every time system B syncs changes to system A.
+
+I don't know what to do except abandon this file path completely and just use a different filename that git doesn't think is deleted on one of the systems; the correct solution to this eludes me.
+
+I appreciate any suggestions that you might have.
+
+Thanks.

wording
diff --git a/doc/special_remotes.mdwn b/doc/special_remotes.mdwn
index 74f473045e..7c227ee07e 100644
--- a/doc/special_remotes.mdwn
+++ b/doc/special_remotes.mdwn
@@ -88,12 +88,12 @@ uses.
 
 ## Setting up a special remote
 
-To create a new special remote, use 
+To initialize a new special remote, use 
 [[git-annex initremote|git-annex-initremote]]. See the documentation for
 the special remote you want to use for details about configuration and
 examples of how to initremote it.
 
-Once a special remote has been created, other clones of the repository can
+Once a special remote has been initialize, other clones of the repository can
 also enable it, by using [[git-annex enableremote|git-annex-enableremote]]
 with the same name that was used to initialize it. (Run the command without
 any name to get a list of available special remotes.)

forgot to add this comment earlier
diff --git a/doc/todo/addurl_--raw-except_REMOTEs__63__/comment_1_d9b88a0f157ce8143d52a35256beaaab._comment b/doc/todo/addurl_--raw-except_REMOTEs__63__/comment_1_d9b88a0f157ce8143d52a35256beaaab._comment
new file mode 100644
index 0000000000..f922e0d30e
--- /dev/null
+++ b/doc/todo/addurl_--raw-except_REMOTEs__63__/comment_1_d9b88a0f157ce8143d52a35256beaaab._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-05T18:12:31Z"
+ content="""
+Seems pretty reasonable.
+
+--raw-except=bittorrent would handle bittorrent, no special case needed for
+that.
+
+Since yt-dlp is effectively part of the web special remote (despite being
+implemented as a special case in addurl), seems reasonable to make
+--raw-except=web disable other special remotes while still letting yt-dlp
+be used for appropriate urls.
+
+Implemented.
+"""]]

improve special remote docs
Sponsored-by: Dartmouth College's DANDI project
diff --git a/doc/git-annex-testremote.mdwn b/doc/git-annex-testremote.mdwn
index cc03b769db..77f7aebaed 100644
--- a/doc/git-annex-testremote.mdwn
+++ b/doc/git-annex-testremote.mdwn
@@ -16,7 +16,9 @@ not altered), although it may perform expensive data transfers.
 
 It's best to make a new remote for testing purposes. While the test
 tries to clean up after itself, if the remote being tested had a bug,
-the cleanup might fail, leaving test data in the remote.
+the cleanup might fail, leaving test data in the remote. Also,
+some special remotes don't support removal of data that has been stored 
+in them, so test data won't be able to be cleaned up when testing those.
 
 Testing will use the remote's configuration, automatically varying
 the chunk sizes, and with simple shared encryption disabled and enabled,
diff --git a/doc/special_remotes.mdwn b/doc/special_remotes.mdwn
index f776605423..74f473045e 100644
--- a/doc/special_remotes.mdwn
+++ b/doc/special_remotes.mdwn
@@ -79,18 +79,30 @@ git annex move --to specialremote large files when your local drive is
 getting full, and then git annex move the files back when free space is
 again available. You could have one repository copy files to a special
 remote, and then git annex get them on another repository, to transfer the
-files between computers that do not communicate directly. 
-
-The git-annex assistant makes it easy to set up rsync remotes using this
-last scenario, which is referred to as a transfer repository, and arranges
-to drop files from the transfer repository once they have been transferred
-to all known clients.
+files between computers that do not communicate directly.
 
 None of these use cases are particular to particular special remote types.
 Most special remotes can all be used in these and other ways. It largely
 doesn't matter for your use what underlying transport the special remote
 uses.
 
+## Setting up a special remote
+
+To create a new special remote, use 
+[[git-annex initremote|git-annex-initremote]]. See the documentation for
+the special remote you want to use for details about configuration and
+examples of how to initremote it.
+
+Once a special remote has been created, other clones of the repository can
+also enable it, by using [[git-annex enableremote|git-annex-enableremote]]
+with the same name that was used to initialize it. (Run the command without
+any name to get a list of available special remotes.)
+
+Initializing or enabling a special remote adds it as a remote of your git
+repository. You can't use git commands like `git pull` with the remote
+(usually, there are exceptions like [[git-lfs]]), but you can use git-annex
+commands.
+
 ## Unused content on special remotes
 
 Over time, special remotes can accumulate file content that is no longer
@@ -108,20 +120,16 @@ on special remotes, instead use `git annex unused --from`. Example:
 	$ git annex dropunused --from mys3 1
 	dropunused 12948 (from mys3...) ok
 
-## Testing special remotes
-
-To make sure that a special remote is working correctly, you can use the
-`git annex testremote` command. This expects you to have set up the remote
-as usual, and it then runs a lot of tests, using random data. It's
-particularly useful to test new implementations of special remotes.
+## Removing special remotes
 
-By default it will upload and download files of around 1MiB to the remote
-it tests; the `--size` parameter can adjust it to test using smaller files.
+Like git remotes, a special remote can be removed from your repository
+by using `git remote remove`. Note that does not delete the special remote,
+or prevent other repositories from enabling or using it.
 
-It's safe to use this command even when you're already storing data in a
-remote; it won't touch your existing files stored on the remote.
+## Testing special remotes
 
-For most remotes, it also won't bloat the remote with any data, since
-it cleans up the stuff it uploads. However, the bup, ddar, and tahoe
-special remotes don't support removal of uploaded files, so be careful
-with those.
+To make sure that a special remote is working correctly, you can use the
+[[git annex testremote|git-annex-testremote]] command. This expects you to
+have set up the remote as usual, and it then runs a lot of tests, using
+random data. It's particularly useful to test new implementations of
+special remotes.
diff --git a/doc/todo/documentation__58___improve_on_special_remotes/comment_1_edd0a4e9894d07e7bae9fef1fe4acf6d._comment b/doc/todo/documentation__58___improve_on_special_remotes/comment_1_edd0a4e9894d07e7bae9fef1fe4acf6d._comment
new file mode 100644
index 0000000000..634aaa1970
--- /dev/null
+++ b/doc/todo/documentation__58___improve_on_special_remotes/comment_1_edd0a4e9894d07e7bae9fef1fe4acf6d._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-05T19:44:16Z"
+ content="""
+The walkthrough has enough to get a user started in the right direction
+in <https://git-annex.branchable.com/walkthrough/#index12h2>.
+
+I don't know if <https://git-annex.branchable.com/special_remotes> is a
+good entry point for a user to learn the basics of what they are. It's
+mostly a list, and I doubt most readers make it past the list.
+
+I have expanded it to mention the main commands.
+"""]]

really close
diff --git a/doc/todo/yt-dlp__58___parse__47__handle___40__error__41_____34__Video_unavailable__34__.mdwn b/doc/todo/yt-dlp__58___parse__47__handle___40__error__41_____34__Video_unavailable__34__.mdwn
index 3953257269..9b2ac720d8 100644
--- a/doc/todo/yt-dlp__58___parse__47__handle___40__error__41_____34__Video_unavailable__34__.mdwn
+++ b/doc/todo/yt-dlp__58___parse__47__handle___40__error__41_____34__Video_unavailable__34__.mdwn
@@ -91,4 +91,4 @@ ERROR: [youtube] jy01CnsQ9ed: Video unavailable
 ```
 
 > I don't consider this a bug because with --no-raw it will do what you
-> want. --[[Joey]]
+> want. [[done]] --[[Joey]]

addurl, importfeed: Added --raw-except option
--raw-except=web allows using yt-dlp but not any other special remotes.
Currently this option can only be used once, trying to use it repeatedly
will make option parsing fail. Perhaps it ought to support being used more
than once, but it seemed like an unlikely use case to need that.
Note that getParsed is called repeatedly when the option is used with
several urls. While implementing DeferredParseClass would avoid that
innefficiency, it didn't seem worth the added boilerplate since
getParsed only calls byNameWithUUID which does minimal work.
Sponsored-by: Dartmouth College's DANDI project
diff --git a/CHANGELOG b/CHANGELOG
index 2a751cfd7a..6dd0e36837 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,7 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
     10.20230626.)
   * importfeed --force: Avoid creating duplicates of existing
     already downloaded files when yt-dlp or a special remote was used.
+  * addurl, importfeed: Added --raw-except option.
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Jan 2024 15:59:33 -0400
 
diff --git a/Command/AddUrl.hs b/Command/AddUrl.hs
index d98c1f95c0..d67456a05d 100644
--- a/Command/AddUrl.hs
+++ b/Command/AddUrl.hs
@@ -64,6 +64,7 @@ data DownloadOptions = DownloadOptions
 	{ relaxedOption :: Bool
 	, rawOption :: Bool
 	, noRawOption :: Bool
+	, rawExceptOption :: Maybe (DeferredParse Remote)
 	, fileOption :: Maybe FilePath
 	, preserveFilenameOption :: Bool
 	, checkGitIgnoreOption :: CheckGitIgnore
@@ -105,6 +106,13 @@ parseDownloadOptions withfileoptions = DownloadOptions
 		( long "no-raw"
 		<> help "prevent downloading raw url content, must use special handling"
 		)
+	<*> optional
+		(mkParseRemoteOption <$> strOption
+		        ( long "raw-except" <> metavar paramRemote
+			<> help "disable special handling except by this remote"
+			<> completeRemotes
+			)
+		)
 	<*> (if withfileoptions
 		then optional (strOption
 			( long "file" <> metavar paramFile
@@ -123,7 +131,7 @@ seek :: AddUrlOptions -> CommandSeek
 seek o = startConcurrency commandStages $ do
 	addunlockedmatcher <- addUnlockedMatcher
 	let go (si, (o', u)) = do
-		r <- Remote.claimingUrl u
+		r <- checkClaimingUrl (downloadOptions o) u
 		if Remote.uuid r == webUUID || rawOption (downloadOptions o')
 			then void $ commandAction $
 				startWeb addunlockedmatcher o' si u
@@ -133,6 +141,13 @@ seek o = startConcurrency commandStages $ do
 			batchInput fmt (pure . parseBatchInput o) go
 		NoBatch -> forM_ (addUrls o) (\u -> go (SeekInput [u], (o, u)))
 
+checkClaimingUrl :: DownloadOptions -> URLString -> Annex Remote
+checkClaimingUrl o u = do
+	allowedremote <- case rawExceptOption o of
+                Nothing -> pure (const True)
+                Just f -> (==) <$> getParsed f
+	Remote.claimingUrl' allowedremote u
+
 parseBatchInput :: AddUrlOptions -> String -> Either String (AddUrlOptions, URLString)
 parseBatchInput o s
 	| batchFilesOption o =
@@ -284,7 +299,7 @@ performWeb addunlockedmatcher o url file urlinfo = lookupKey file >>= \case
   where
 	geturl = next $ isJust <$> addUrlFile addunlockedmatcher (downloadOptions o) url urlinfo file
 	addurl = addUrlChecked o url file webUUID $ \k ->
-		ifM (pure (not (rawOption (downloadOptions o))) <&&> youtubeDlSupported url)
+		ifM (useYoutubeDl (downloadOptions o) <&&> youtubeDlSupported url)
 			( return (Just (True, True, setDownloader url YoutubeDownloader))
 			, checkRaw Nothing (downloadOptions o) (pure Nothing) $
 				return (Just (Url.urlExists urlinfo, Url.urlSize urlinfo == fromKey keySize k, url))
@@ -332,7 +347,7 @@ downloadWeb addunlockedmatcher o url urlinfo file =
 	urlkey = addSizeUrlKey urlinfo $ Backend.URL.fromUrl url Nothing
 	downloader f p = Url.withUrlOptions $ downloadUrl False urlkey p Nothing [url] f
 	go Nothing = return Nothing
-	go (Just (tmp, backend)) = ifM (pure (not (rawOption o)) <&&> liftIO (isHtmlFile (fromRawFilePath tmp)))
+	go (Just (tmp, backend)) = ifM (useYoutubeDl o <&&> liftIO (isHtmlFile (fromRawFilePath tmp)))
 		( tryyoutubedl tmp backend
 		, normalfinish tmp backend
 		)
@@ -394,6 +409,15 @@ checkRaw failreason o f a
 		f
 	| otherwise = a
 
+useYoutubeDl :: DownloadOptions -> Annex Bool
+useYoutubeDl o
+	| rawOption o = pure False
+	| otherwise = case rawExceptOption o of
+                Nothing -> pure True
+                Just f -> do
+			remote <- getParsed f
+			pure (Remote.uuid remote == webUUID)
+
 {- The destination file is not known at start time unless the user provided
  - a filename. It's not displayed then for output consistency, 
  - but is added to the json when available. -}
diff --git a/Command/ImportFeed.hs b/Command/ImportFeed.hs
index a8d04ef2da..f96bcd869c 100644
--- a/Command/ImportFeed.hs
+++ b/Command/ImportFeed.hs
@@ -38,7 +38,7 @@ import Logs.File
 import qualified Utility.Format
 import Utility.Tmp
 import Utility.Metered
-import Command.AddUrl (addUrlFile, downloadRemoteFile, parseDownloadOptions, DownloadOptions(..), checkCanAdd, addWorkTree, checkRaw)
+import Command.AddUrl (addUrlFile, downloadRemoteFile, parseDownloadOptions, DownloadOptions(..), checkClaimingUrl, checkCanAdd, addWorkTree, checkRaw, useYoutubeDl)
 import Annex.UUID
 import Backend.URL (fromUrl)
 import Annex.Content
@@ -317,9 +317,8 @@ startDownload addunlockedmatcher opts cache cv todownload = case location todown
 	startdownloadenclosure url = checkknown url $ startUrlDownload cv todownload url $
 		downloadEnclosure addunlockedmatcher opts cache cv todownload url 
 
-	downloadmedia linkurl mediaurl mediakey
-		| rawOption (downloadOptions opts) = startdownloadlink
-		| otherwise = ifM (youtubeDlSupported linkurl)
+	downloadmedia linkurl mediaurl mediakey =
+		ifM (useYoutubeDl (downloadOptions opts) <&&> youtubeDlSupported linkurl)
 			( startUrlDownload cv todownload linkurl $
 				withTmpWorkDir mediakey $ \workdir -> do
 					dl <- youtubeDl linkurl (fromRawFilePath workdir) nullMeterUpdate
@@ -348,7 +347,7 @@ startDownload addunlockedmatcher opts cache cv todownload = case location todown
 		contdownloadlink = downloadEnclosure addunlockedmatcher opts cache cv todownload linkurl
 
 	addmediafast linkurl mediaurl mediakey =
-		ifM (pure (not (rawOption (downloadOptions opts)))
+		ifM (useYoutubeDl (downloadOptions opts)
 		     <&&> (pure (youtubedlscraped todownload) <||> youtubeDlSupported linkurl))
 			( startUrlDownload cv todownload linkurl $ do
 				runDownload todownload linkurl ".m" cache cv $ \f ->
@@ -362,7 +361,7 @@ downloadEnclosure :: AddUnlockedMatcher -> ImportFeedOptions -> Cache -> TMVar B
 downloadEnclosure addunlockedmatcher opts cache cv todownload url = 
 	runDownload todownload url (takeWhile (/= '?') $ takeExtension url) cache cv $ \f -> do
 		let f' = fromRawFilePath f
-		r <- Remote.claimingUrl url
+		r <- checkClaimingUrl (downloadOptions opts) url
 		if Remote.uuid r == webUUID || rawOption (downloadOptions opts)
 			then checkRaw (Just url) (downloadOptions opts) (pure Nothing) $ do
 				let dlopts = (downloadOptions opts)
diff --git a/Remote.hs b/Remote.hs
index 1638777277..dbbca32a5e 100644
--- a/Remote.hs
+++ b/Remote.hs
@@ -1,6 +1,6 @@
 {- git-annex remotes
  -
- - Copyright 2011-2020 Joey Hess <id@joeyh.name>
+ - Copyright 2011-2024 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -59,6 +59,7 @@ module Remote (
 	logStatus,
 	checkAvailable,
 	claimingUrl,
+	claimingUrl',
 	isExportSupported,
 ) where
 
@@ -432,9 +433,15 @@ hasKeyCheap = checkPresentCheap
 
 {- The web special remote claims urls by default. -}
 claimingUrl :: URLString -> Annex Remote
-claimingUrl url = do
+claimingUrl = claimingUrl' (const True)
+
+{- The web special remote still claims urls if there is no
+ - other remote that does, even when the remotefilter does
+ - not include it. -}
+claimingUrl' :: (Remote -> Bool) -> URLString -> Annex Remote
+claimingUrl' remotefilter url = do
 	rs <- remoteList
 	let web = Prelude.head $ filter (\r -> uuid r == webUUID) rs
-	fromMaybe web <$> firstM checkclaim rs
+	fromMaybe web <$> firstM checkclaim (filter remotefilter rs)
   where
 	checkclaim = maybe (pure False) (`id` url) . claimUrl
diff --git a/doc/git-annex-addurl.mdwn b/doc/git-annex-addurl.mdwn
index dd2bfee1a9..370e5be986 100644
--- a/doc/git-annex-addurl.mdwn
+++ b/doc/git-annex-addurl.mdwn
@@ -45,8 +45,8 @@ be used to get better filenames.
   
 * `--raw`
 
-  Prevent special handling of urls by yt-dlp, bittorrent, and other
-  special remotes. This will for example, make addurl
+  Prevent special handling of urls by yt-dlp, and by bittorrent
+  and other special remotes. This will for example, make addurl
   download the .torrent file and not the contents it points to.
 
 * `--no-raw`
@@ -55,6 +55,12 @@ be used to get better filenames.
   or a special remote, rather than the raw content of the url. if that
   cannot be done, the add will fail.
 
+* `--raw-except=remote`
+
+  Prevent special handling of urls by all special remotes except
+  for the specified one. To allow special handling only
+  by yt-dlp, use `--raw-except=web`.

(Diff truncated)
comment and close
diff --git a/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch.mdwn b/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch.mdwn
index 91b45aef8f..7cf57892e8 100644
--- a/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch.mdwn
+++ b/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch.mdwn
@@ -30,3 +30,6 @@ on Linux
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 Yes, I am developing the easy-git-annex interface.
+
+> Considering this not a bug as explained in my comment. [[done]]
+> --[[Joey]]
diff --git a/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch/comment_1_5ff140be453458e84bd6072594e70d8a._comment b/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch/comment_1_5ff140be453458e84bd6072594e70d8a._comment
new file mode 100644
index 0000000000..cee7fdf6d3
--- /dev/null
+++ b/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch/comment_1_5ff140be453458e84bd6072594e70d8a._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-05T18:06:59Z"
+ content="""
+You have to `git-annex sync` (or `push`) first to get your changes
+propagated from the adjusted branch to the main branch that you are
+exporting.
+
+(You could alternatively export the adjusted branch itself.)
+
+This is the same as if you tried `git push main origin` when on an adjusted
+branch. It would surely not be a bug that doesn't push changes you have
+only committed to the adjusted branch. So I think it's reasonable for the
+export workflow to need the same step that the git push workflow needs.
+"""]]

comment
diff --git a/doc/todo/add_--json-progress_support_in_push_and_pull/comment_1_403acf073a2a8d53eaa880cc2564347d._comment b/doc/todo/add_--json-progress_support_in_push_and_pull/comment_1_403acf073a2a8d53eaa880cc2564347d._comment
new file mode 100644
index 0000000000..70de5b1d77
--- /dev/null
+++ b/doc/todo/add_--json-progress_support_in_push_and_pull/comment_1_403acf073a2a8d53eaa880cc2564347d._comment
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-05T17:57:49Z"
+ content="""
+They don't have --json, which would be a necessary first step.
+
+This was considered in [[this_todo|todo/--json_for_unannex__and_ideally_any_other_command_]]
+when adding --json to many commands, and the thinking for not adding it
+was:
+
+> git-annex-sync (while it would be pretty easy to support, it outputs
+> different types of messages depending on what remotes it syncs with and
+> what needs to be done. Eg, copy to remote, or export to remote, or import
+> from remote. Each would be a different format of json message, which
+> violates the principle that all git-annex json output should be
+> discoverable by simply running the command. And of course, everything it
+> does can be done by other commands, which can support json without having
+> that problem.)
+
+sync had not been split into pull and push at that point. Being split does
+reduce the space of different things, but it's still multiple things, so
+still a problem for json output discoverability.
+
+(Also push and pull can drop from a remote or locally, and of course there are
+the git operations they do, which would probably have to become silent in
+json mode.)
+"""]]

comment
diff --git a/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_3_adebe3ba65ba9fc0a4a7f4b664afc069._comment b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_3_adebe3ba65ba9fc0a4a7f4b664afc069._comment
new file mode 100644
index 0000000000..b6772525ad
--- /dev/null
+++ b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_3_adebe3ba65ba9fc0a4a7f4b664afc069._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2024-02-05T17:33:32Z"
+ content="""
+The commands you would need do all support --batch, so it seems very
+scriptable.
+
+I'm struggling a bit with the idea of a dedicated git-annex command for
+this, because it seems a fairly unusual situation.
+
+Maybe `git-annex registerurl could have a switch that says to move urls
+from remote foo to the web special remote? That would avoid needing to
+query for the urls.
+
+	git-annex registerurl --move-from=foo $key
+
+Since registering an url also marks it present, all you would need after
+that is to mark it as not present any longer on your special remote.
+"""]]

comment
diff --git a/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches/comment_1_2c5f51483b2f2521d98f943d8ded3acd._comment b/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches/comment_1_2c5f51483b2f2521d98f943d8ded3acd._comment
new file mode 100644
index 0000000000..7c90cfe3c9
--- /dev/null
+++ b/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches/comment_1_2c5f51483b2f2521d98f943d8ded3acd._comment
@@ -0,0 +1,47 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-05T17:26:48Z"
+ content="""
+Works for me. Also if pull didn't work, sync wouldn't work and surely
+someone would have reported such a problem?
+
+But, you left out information including what kind of adjusted branch.
+Please provide the details needed to reproduce your problem.
+
+Here it is working:
+
+	joey@darkstar:~/tmp/bench/y#master(unlocked)>git-annex pull --allow-unrelated-histories origin
+	pull origin
+	remote: Enumerating objects: 10, done.
+	remote: Counting objects: 100% (10/10), done.
+	remote: Compressing objects: 100% (7/7), done.
+	remote: Total 8 (delta 0), reused 0 (delta 0), pack-reused 0
+	Unpacking objects: 100% (8/8), 794 bytes | 397.00 KiB/s, done.
+	From /home/joey/tmp/bench/x
+	   2722cc7..f380042  git-annex  -> origin/git-annex
+	   3960dff..ae25b27  master     -> origin/master
+	(Merging into master...)
+	Updating 3960dff..ae25b27
+	Fast-forward
+	 x | 1 +
+	 1 file changed, 1 insertion(+)
+	 create mode 120000 x
+	(Merging into adjusted branch...)
+	Updating 6c80746..7a752e3
+	Fast-forward
+	 x | 1 +
+	 1 file changed, 1 insertion(+)
+	 create mode 100644 x
+	ok
+	(merging origin/git-annex into git-annex...)
+	(recording state in git...)
+	get x (from origin...)
+	ok
+	pull origin
+	ok
+	(recording state in git...)
+	(recording state in git...)
+	joey@darkstar:~/tmp/bench/y#master(unlocked)>cat x
+	hello, world
+"""]]

Added a comment: Same issue after kernel oops
diff --git a/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_17_7fcd45354f94756c34b2f3f912972c19._comment b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_17_7fcd45354f94756c34b2f3f912972c19._comment
new file mode 100644
index 0000000000..097b85d926
--- /dev/null
+++ b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_17_7fcd45354f94756c34b2f3f912972c19._comment
@@ -0,0 +1,32 @@
+[[!comment format=mdwn
+ username="felix@996ab1030f55d480e424a17116e03a48bd984549"
+ nickname="felix"
+ avatar="http://cdn.libravatar.org/avatar/d6198e60afe87cca852190852feeafa8"
+ subject="Same issue after kernel oops"
+ date="2024-02-05T14:58:33Z"
+ content="""
+Just to add one more data point – I ran into the situation after a kernel oops had caused my rootfs to become read-only. So maybe related to the \"out of inodes\" issue reported above.
+
+~~~
+$ git push
+gcrypt: Decrypting manifest
+gpg: error getting version from 'scdaemon': No SmartCard daemon
+gpg: Signature made Mo 05 Feb 2024 15:22:01 CET
+gpg:                using RSA key XXX
+gpg: Good signature from \"XXX\" [ultimate]
+Primary key fingerprint: XXX
+gcrypt: Due to a longstanding bug, this push implicitly has --force.
+gcrypt: Consider explicitly passing --force, and setting
+gcrypt: gcrypt's require-explicit-force-push git config key.
+gcrypt: Repacking remote origin, ...
+gcrypt: Packfile XXX does not match digest!
+fatal: early EOF
+error: failed to push some refs to 'XXX'
+~~~
+
+XXX is obviously my redactions. I was not running the assistant, just plain git-annex.
+
+There was no plaintext on the remote and the only way for me to recover was to nuke the remote repo.
+
+I kept copies of both repos, though I unfortunately cannot share the unencrypted one.
+"""]]

diff --git a/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn b/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
index c145b969a0..df28fd21e2 100644
--- a/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
+++ b/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
@@ -3,7 +3,7 @@
 
 > By default, all files added are added to the annex, the same as when you run git annex add. If you configure annex.largefiles, files that it does not match will instead be added with git add.
 
-## Actual behaiors
+## Actual behaviors
 Git annex replaces all old files (file is already stored by git before the `git annex` command) and newly added files with pointer files after using the `git annex assistant` command. Even I config `annex.largefiles` to include all files.
 
 

diff --git a/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn b/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
index e5c54fdf0b..c145b969a0 100644
--- a/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
+++ b/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
@@ -1,5 +1,6 @@
 ## Expect behaviors
-`git annex assistant` won't annex the old files and won't annex "non-large files".
+`git annex assistant` doesn't annex the old files and annex "non-large files". According to the below quote from the [`git-annex assistant`](https://git-annex.branchable.com/git-annex-assistant/) document:
+
 > By default, all files added are added to the annex, the same as when you run git annex add. If you configure annex.largefiles, files that it does not match will instead be added with git add.
 
 ## Actual behaiors

diff --git a/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn b/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
index b056931343..e5c54fdf0b 100644
--- a/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
+++ b/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
@@ -13,6 +13,7 @@ Git annex replaces all old files (file is already stored by git before the `git
 ## Context
 
 Output of `git annex version`:
+
 ```
 git-annex version: 10.20240129
 build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV

update for rename of forum/assistant.mdwn to forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
diff --git a/doc/forum/Can__39__t_init_git_annex/comment_5_53c33484bded57abc60f0449331c7b05._comment b/doc/forum/Can__39__t_init_git_annex/comment_5_53c33484bded57abc60f0449331c7b05._comment
index 5bbbb73608..996a7a5fcf 100644
--- a/doc/forum/Can__39__t_init_git_annex/comment_5_53c33484bded57abc60f0449331c7b05._comment
+++ b/doc/forum/Can__39__t_init_git_annex/comment_5_53c33484bded57abc60f0449331c7b05._comment
@@ -6,6 +6,6 @@
  content="""
 `git annex sync` only syncs the git repository, not file contents, and the special remote has no git repo, only file contents. `git annex copy` will avoid transferring things that are already there, so it's the way to go.
 
-If you want automatic syncing of file contents and lots of other magic including automatic commit of new files, you could try the [[assistant]].
+If you want automatic syncing of file contents and lots of other magic including automatic commit of new files, you could try the [[forum/`git_annex_assistant`_command_replace_all_files_with_pointer_file.]].
 
 """]]

update for rename of forum/assistant.mdwn to forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
diff --git a/doc/forum/Using_for_Music_repo/comment_1_3488ed85ad98f14cb17f229225ece26e._comment b/doc/forum/Using_for_Music_repo/comment_1_3488ed85ad98f14cb17f229225ece26e._comment
index 6176f755dd..d414ecb806 100644
--- a/doc/forum/Using_for_Music_repo/comment_1_3488ed85ad98f14cb17f229225ece26e._comment
+++ b/doc/forum/Using_for_Music_repo/comment_1_3488ed85ad98f14cb17f229225ece26e._comment
@@ -6,5 +6,5 @@
  content="""
 You can use [[direct_mode]] to keep all files unlocked all the time.
 
-You can use the [[assistant]] to automatically commit whatever changes are made to the directory, as well as automatically sync them to your other computers.
+You can use the [[forum/`git_annex_assistant`_command_replace_all_files_with_pointer_file.]] to automatically commit whatever changes are made to the directory, as well as automatically sync them to your other computers.
 """]]

rename forum/assistant.mdwn to forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
diff --git a/doc/forum/assistant.mdwn b/doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn
similarity index 100%
rename from doc/forum/assistant.mdwn
rename to doc/forum/__96__git_annex_assistant__96___command_replace_all_files_with_pointer_file..mdwn

diff --git a/doc/forum/assistant.mdwn b/doc/forum/assistant.mdwn
new file mode 100644
index 0000000000..b056931343
--- /dev/null
+++ b/doc/forum/assistant.mdwn
@@ -0,0 +1,29 @@
+## Expect behaviors
+`git annex assistant` won't annex the old files and won't annex "non-large files".
+> By default, all files added are added to the annex, the same as when you run git annex add. If you configure annex.largefiles, files that it does not match will instead be added with git add.
+
+## Actual behaiors
+Git annex replaces all old files (file is already stored by git before the `git annex` command) and newly added files with pointer files after using the `git annex assistant` command. Even I config `annex.largefiles` to include all files.
+
+
+
+
+[Example repo](https://github.com/BilderLoong/test)
+
+## Context
+
+Output of `git annex version`:
+```
+git-annex version: 10.20240129
+build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
+dependency versions: aws-0.24.1 bloomfilter-2.0.1.2 crypton-0.34 DAV-1.3.4 feed-1.3.2.1 ghc-9.6.3 http-client-0.7.16 persistent-sqlite-2.13.3.0 torrent-10000.1.3 uuid-1.3.15 yesod-1.6.2.1
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
+operating system: darwin x86_64
+supported repository versions: 8 9 10
+upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
+local repository version: 10
+```
+
+System: macOS Monterey Version 12.6 (21G115)
+

diff --git a/doc/todo/add_--json-progress_support_in_push_and_pull.mdwn b/doc/todo/add_--json-progress_support_in_push_and_pull.mdwn
new file mode 100644
index 0000000000..36c8337293
--- /dev/null
+++ b/doc/todo/add_--json-progress_support_in_push_and_pull.mdwn
@@ -0,0 +1 @@
+The pull and push commands do not have --json-progress support.  Please add.

diff --git a/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches.mdwn b/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches.mdwn
new file mode 100644
index 0000000000..0198cae250
--- /dev/null
+++ b/doc/bugs/pull_doesn__39__t_work_on_adjusted_branches.mdwn
@@ -0,0 +1,29 @@
+### Please describe the problem.
+
+Pull works fine if both the source and destination repositories are on unadjusted branches.
+
+If either of the repositories is on an adjusted branch, the pull operation does not copy anything.
+
+### What steps will reproduce the problem?
+
+git annex pull --allow-unrelated-histories pull-remote
+
+### What version of git-annex are you using? On what operating system?
+
+10.20231228-g5540f42e21afce6947d5410c5f18b178de6c336a
+
+on Linux
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes, I am developing the easy-git-annex interface.

Added a comment
diff --git a/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_2_c122906fd03bd723aa027fe32fd4fab6._comment b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_2_c122906fd03bd723aa027fe32fd4fab6._comment
new file mode 100644
index 0000000000..d2a8434623
--- /dev/null
+++ b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_2_c122906fd03bd723aa027fe32fd4fab6._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2024-02-03T01:17:39Z"
+ content="""
+in theory - yes. In practice, ATM I would probably just `git annex whereis | awk -e \"/^ *$remote:/{print $2;}\"` or alike in bash, and in Python it would be quite trivial to filter for a specific remote. 
+The main \"hurdle\" here would be the need to do that dance with registerurl/unregisterurl and given that we might need to do that for many thousands keys (imaging going through such repos as [dandizarrs](https://github.com/dandizarrs)), I felt like some internal dedicated function would be worthwhile.
+"""]]

compare urls irrespective of downloader
importfeed --force: Avoid creating duplicates of existing already
downloaded files when yt-dlp or a special remote was used.
diff --git a/CHANGELOG b/CHANGELOG
index 6eca06eba1..d0f759cc6a 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -2,6 +2,8 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
 
   * importfeed: Added --scrape option, which uses yt-dlp to screen scrape
     the equivilant of an RSS feed.
+  * importfeed --force: Avoid creating duplicates of existing
+    already downloaded files when yt-dlp or a special remote was used.
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Jan 2024 15:59:33 -0400
 
diff --git a/Command/ImportFeed.hs b/Command/ImportFeed.hs
index e4e78818ce..dbc8cb8bf0 100644
--- a/Command/ImportFeed.hs
+++ b/Command/ImportFeed.hs
@@ -462,7 +462,7 @@ runDownload todownload url extension cache cv getter = do
 				in d </> show n ++ "_" ++ base
 		tryanother = makeunique (n + 1) file
 		alreadyexists = liftIO $ isJust <$> catchMaybeIO (R.getSymbolicLinkStatus (toRawFilePath f))
-		checksameurl k = ifM (elem url <$> getUrls k)
+		checksameurl k = ifM (elem url . map fst . map getDownloader <$> getUrls k)
 			( return Nothing
 			, tryanother
 			)
diff --git a/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_4_0bf6d5b03032c0b9fd35c8410aea832c._comment b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_4_0bf6d5b03032c0b9fd35c8410aea832c._comment
new file mode 100644
index 0000000000..2b79698131
--- /dev/null
+++ b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_4_0bf6d5b03032c0b9fd35c8410aea832c._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2024-02-02T19:27:41Z"
+ content="""
+Oh, but it seems that `importfeed --force` already has code to handle this.
+It checks if the file already exists and has the same url recorded as the url
+being imported. For some reason in that case, it says it failed to import
+the url, which seems a bit odd behavior to me, but in any case it doesn't
+add a "2_" file.
+
+That works for regular rss feeds, but does not work for yt-dlp urls.
+
+The reason is the url mangling done for yt-dlp urls (and other special
+remote urls). Fixed this bug.
+"""]]

comment
diff --git a/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_1_d4fb3eff1b61c60224fec112df9f1136._comment b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_1_d4fb3eff1b61c60224fec112df9f1136._comment
new file mode 100644
index 0000000000..74b172240c
--- /dev/null
+++ b/doc/forum/how_to___34__move__34___URL_between_remotes__63__/comment_1_d4fb3eff1b61c60224fec112df9f1136._comment
@@ -0,0 +1,37 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-02T18:48:59Z"
+ content="""
+I see you're using CLAIMURL. What `git-annex addurl` does when a special
+remote claims an url is it records the url for the key in the git-annex
+branch, but mangled to indicate that it is not an url used by the web
+special remote.
+
+The mangling is just to prefix the url with ":".
+
+If you `git-annex registerurl --remote=web` the same url again
+but without that prefix, and  
+`git-annex setpresentkey $key 00000000-0000-0000-0000-000000000001 1`,
+the url will will able to be downloaded by the web special remote. 
+
+Then you can unregisterurl the (unmangled) url from your remote that you no
+longer want to use, and use setpresentkey with your remote's uuid and "0" to
+remove the mangled url.
+
+I don't think there's any plumbing currently that makes it easy to access
+and demangle the urls. `git-annex whereis --json` will list the demangled
+url in the "urls" field, but in amoung any other urls that other special
+remotes might have for the same content. Without --json there is a nice
+display that shows the remote that claims the url:
+
+	joey@darkstar:~/tmp/bench/rx>git-annex whereis
+	whereis foo (1 copy)
+	  	177d51e9-a115-443e-8100-49588f8a6ca2 -- foo [test]
+	
+	  test: http://example.com/foo.test
+
+Improving the whereis --json or adding some other machine-readable 
+way to list urls claimed by a remote seems like maybe worth doing?
+Let me know.
+"""]]

Added a comment
diff --git a/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_3_aa1077117a83ecffccb8d2095818fa02._comment b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_3_aa1077117a83ecffccb8d2095818fa02._comment
new file mode 100644
index 0000000000..2d3cccb343
--- /dev/null
+++ b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_3_aa1077117a83ecffccb8d2095818fa02._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 3"
+ date="2024-02-02T19:08:38Z"
+ content="""
+I don't think overwrite here should have done anything -- should have been the same key and filename, so should be the same as skipping I think.
+
+Overall indeed might be just nice to have `--existing noclob|overwrite|skip` so it either adds index to declob, or skips or overwrites?  I would have probably used either skip or overwrite ;)
+"""]]

oddly groundhog's day thematic response
diff --git a/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_2_b9885808ebe447c26283f910f4fc37c3._comment b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_2_b9885808ebe447c26283f910f4fc37c3._comment
new file mode 100644
index 0000000000..79b1eb5a62
--- /dev/null
+++ b/doc/forum/importfeed_on_multiple___34__overlapping__34___playlists/comment_2_b9885808ebe447c26283f910f4fc37c3._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2024-02-02T18:34:23Z"
+ content="""
+The reason it doesn't overwrite is that it's not impossible for a feed
+to have two posts with the same title (eg "happy groundhogs day") or
+whatever else is used to construct the filename (unless you use `${itemid}`
+in your --template).
+
+Maybe overwriting would have been the right choice in your situation.. do
+you think an option that overwrites would be useful?
+
+Or maybe better, an option that skips downloading when a file with the
+generated name already exists? If one groundhog's day greeting
+is enough for you..
+"""]]

followup
diff --git a/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_1_c5eec865ac5493f8bedf4fcba6759d6e._comment b/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_1_c5eec865ac5493f8bedf4fcba6759d6e._comment
new file mode 100644
index 0000000000..8256d1e1f2
--- /dev/null
+++ b/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_1_c5eec865ac5493f8bedf4fcba6759d6e._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2024-02-02T17:23:13Z"
+ content="""
+This would not have prevented `copy --auto` from trying to copy the files
+and failing the same way as `copy` without that option. So I think there
+must be something in your preferred content that made it skip trying to
+copy those files.
+
+	include=.datalad/* and (not metadata=distribution-restrictions=*)
+
+Maybe you meant to have an "or" there? With the and it only wants files
+that are in .datalad/ as well as not having the metadata set.
+"""]]
diff --git a/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_2_b6d694d34e6d5a684249b11f53a8532f._comment b/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_2_b6d694d34e6d5a684249b11f53a8532f._comment
new file mode 100644
index 0000000000..093a4b358e
--- /dev/null
+++ b/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_2_b6d694d34e6d5a684249b11f53a8532f._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2024-02-02T17:25:54Z"
+ content="""
+As for `pushInsteadOf`, in 2011 this was considered
+in <https://bugs.debian.org/644278>. And the result was that git-annex
+honors `insteadOf` but not `pushInsteadOf` or `pushurl`. With the (weak?)
+rationalle that what git-annex does is neither pushing or pulling really.
+
+So it seems to me better for you to use `insteadOf`. Unless there's some
+reason why you need git to pull from the http url rather than from the ssh
+url?
+
+Perhaps you're setting this up for many users,
+some of whom are limited to read-only access.
+Pulling from http would work for those users. And git-annex get
+from http also works read-only the way your repository is set up.
+
+If that is the reason you want to use `pushInsteadOf` rather than
+`insteadOf`, it would follow that you would want git-annex to use
+the pull url for getting files, but use the push url for putting/dropping
+files.
+"""]]
diff --git a/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_3_e53e69ae3adc42bb2bf36af1f07a4c83._comment b/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_3_e53e69ae3adc42bb2bf36af1f07a4c83._comment
new file mode 100644
index 0000000000..e34fd8eba8
--- /dev/null
+++ b/doc/todo/copy__47__move_support_for_pushinsteadOf_/comment_3_e53e69ae3adc42bb2bf36af1f07a4c83._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2024-02-02T17:57:51Z"
+ content="""
+But: If this change were made, it would risk breaking existing
+working setups, that happen to have a push url that points to a different
+repository. When git-annex was upgraded to use the push url, it would start
+noticing that the repository behind the url has a different uuid than the
+remote does.
+
+For a ssh repository, that would prevent it from using the repository until
+the user did something to fix the configuration. 
+
+For a local repository, git-annex currently automatically updates the
+cached repository uuid. It's not clear to me how that would work if there were
+two urls pointing to two different repositories. Does seem like this would
+prevent eg, getting files from the remote that it was able to get before.
+
+I don't know how common such a setup with a push url pointing to a 
+different repository might be. I think it is much more
+likely that `remote.foo.pushurl` be pointed to some other url that is not
+on the same server. pushInsteadOf is really intended to configure a
+different access method for the same server as the repository url.
+"""]]

the plea to respect prepare-commit-msg hook
diff --git a/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg.mdwn b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg.mdwn
new file mode 100644
index 0000000000..37af943de4
--- /dev/null
+++ b/doc/todo/make_annex___34__respect__34___.git__47__hooks__47__prepare-commit-msg.mdwn
@@ -0,0 +1,136 @@
+I am trying to work out a helper for creating more meaningful automated commit messages for reprostim where we collect videos and annex assistant moves them to the server.
+
+<details>
+<summary>I was about to file this TODO report describing more detail before I realized that in principle git has a hook I could use</summary> 
+
+ATM (git annex 10.20231227-1~ndall+1) git annex assistant just commits with a generic non-descript (besides location) commit message e.g.
+
+```
+git-annex in reprostim@reproiner:/data/reprostim
+```
+
+and to figure out what actually git-annex has done takes at least rerunning `git log --stat` but in git gui of some kind takes looking at each and every commit to identify the one of interest (which e.g. modified `config.yaml`) or specific filtering etc.
+
+I would have appreciated if commits became more descriptive. Since git-annex assistant is quite reactive and (at least in our case) mostly commits one file change at a time I would have appreciated commit messages like
+
+```
+Edited config.yaml
+```
+
+or `Addded` or `Removed`.  If file is under a folder and length of the path is over some limit, would have abbreviated path to smth like ```.../config.yaml```.
+
+In case of multiple files per "action", summarized in numbers and possibly again with a hint on location
+
+```
+Edited config.yaml, added 10 files under Videos/
+```
+
+Moreover, I would have still appreciated that information annex reports about location now, but I would have made it into the extended part of the commit message after a new line, and added a version info.  So altogether could have looked like
+
+```
+Edited config.yaml, added 10 files under Videos/
+
+---
+# git annex metadata
+repository: reprostim@reproiner:/data/reprostim
+version: 10.20231227-1~ndall+1
+```
+
+And even more "ideally", if it could pick up (if exists) an optional file (e.g. `.git/ANNEX_COMMIT_META.yaml`) or some `git config` field to add to the commit, I could have then added information about the system/software I care about, e.g.
+
+```
+reprostim-version: 0.20240202.0
+```
+
+and thus have commit carrying metadata about the version which produces those video files.  ATM the video capture is completely ignorant of git-annex so I cannot go and metadata annotate those files... but that would be a different issue ... ;)
+
+As some kind of testament to possibility and usability of such commits, could navigate through commits of our automated datalad dandisets e.g. [000108](https://github.com/dandisets/000108/commits/draft/) where now without any extra tools etc I can tell where we added or removed files or just modified some metadata. It is indeed custom and more specific to our use case, but I think aforementioned would already be better.
+
+</details>
+
+<details>
+<summary>But when I created this experimental version of the script to see what info I have available</summary> 
+
+```shell
+#!/bin/sh
+#
+# A hook to provide custom commit messages for
+# changes in the repo which would be better than default ones git-annex provides.
+
+COMMIT_MSG_FILE=$1
+COMMIT_SOURCE=$2
+SHA1=$3
+
+if [ -n "$COMMIT_MSG_FILE" ]; then
+	orig_msg=$(cat "$COMMIT_MSG_FILE")
+else
+	orig_msg=NONE
+fi
+
+cat <<EOF >| "$COMMIT_MSG_FILE"
+Custom commit msg for source=$COMMIT_SOURCE sha1=$SHA1
+
+orig_msg: $orig_msg
+git-annex version: `git annex version --raw`
+environment:
+`export`
+
+EOF
+
+```
+</details>
+
+I saw that `git-annex` somehow commits while avoiding this hook entirely!
+
+```
+❯ rm random && dd if=/dev/random of=random count=1 && git annex add random && git commit -m "Added random to annex without custom commit msg"; git show git-annex | head -n 5
+1+0 records in
+1+0 records out
+512 bytes copied, 0.000295152 s, 1.7 MB/s
+add random 
+ok                                
+(recording state in git...)
+[master 470d6ce] Custom commit msg for source=message sha1=
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+commit 97423356f56f1f16b6a9646614af1d3d4d3d8717
+Author: Yaroslav Halchenko <debian@onerussian.com>
+Date:   Fri Feb 2 12:03:18 2024 -0500
+
+    update
+
+```
+
+so we got hook working for `master` and commit to `git-annex` branch went without it.  FWIW if I use annex.commitmessage -- that one works but it is inflexible
+
+```shell
+❯ rm random && dd if=/dev/random of=random count=1 && git -c annex.commitmessage="custom one" annex add random && git commit -m "Added random to annex WITH custom commit msg"; git show git-annex | head -n 5
+1+0 records in
+1+0 records out
+512 bytes copied, 8.0667e-05 s, 6.3 MB/s
+add random 
+ok                                
+(recording state in git...)
+[master 65d4400] Custom commit msg for source=message sha1=
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+commit dff6b4833f4d0e2193d43576c834212f84e54f49
+Author: Yaroslav Halchenko <debian@onerussian.com>
+Date:   Fri Feb 2 12:05:08 2024 -0500
+
+    custom one
+```
+
+I think it would be great if I could just create a "generic" prepare-commit-msg hook I could use for any branch, git-annex included.
+
+The same applies to changes git-annex assistant commits to master -- apparently it seems to avoid that hook as well since even with the hook present I got 
+
+```shell
+commit 78c5dcb6bf91b056cba7dc4ee93dd5b31f15f297 (HEAD -> master, synced/master)
+Author: Yaroslav Halchenko <debian@onerussian.com>
+Date:   Fri Feb 2 12:06:41 2024 -0500
+
+    git-annex in yoh@bilena:~/proj/datalad/trash/try-commit-message
+```
+
+
+[[!meta author=yoh]]
+[[!tag projects/repronim]]

todo for support of pushinsteadof
diff --git a/doc/todo/copy__47__move_support_for_pushinsteadOf_.mdwn b/doc/todo/copy__47__move_support_for_pushinsteadOf_.mdwn
new file mode 100644
index 0000000000..48c440f1eb
--- /dev/null
+++ b/doc/todo/copy__47__move_support_for_pushinsteadOf_.mdwn
@@ -0,0 +1,58 @@
+ATM
+
+files didn't `datalad push` as they should have due to existing settings of `wanted`:
+
+```
+❯ git annex wanted datasets.datalad.org
+include=.datalad/* and (not metadata=distribution-restrictions=*)
+❯ git annex find --not --in datasets.datalad.org .
+crcns-2022-dataland.pdf
+crcns-2022-dataland.png
+crcns-2022-dataland.svg
+❯ git annex metadata *
+metadata crcns-2022-dataland.pdf 
+
+ok
+metadata crcns-2022-dataland.png 
+
+ok
+metadata crcns-2022-dataland.svg 
+
+ok
+❯ git annex copy --auto --to datasets.datalad.org *
+❯ git annex version
+git-annex version: 10.20231227-1~ndall+1
+
+```
+
+so I was confused... the reason was
+
+```
+❯ git annex copy  --to datasets.datalad.org *
+copy crcns-2022-dataland.pdf (to datasets.datalad.org...) 
+  copying to non-ssh repo not supported
+failed
+copy crcns-2022-dataland.png (to datasets.datalad.org...) 
+  copying to non-ssh repo not supported
+failed
+copy crcns-2022-dataland.svg (to datasets.datalad.org...) 
+  copying to non-ssh repo not supported
+failed
+copy: 3 failed
+```
+
+wherever I have 
+
+```shell
+❯ git remote show -n datasets.datalad.org
+* remote datasets.datalad.org
+  Fetch URL: https://datasets.datalad.org/datalad/artwork/.git
+  Push  URL: falkor.dartmouth.edu:/srv/datasets.datalad.org/www/datalad/artwork/.git
+...
+```
+
+and the use case is quite common for me and in particular for ReproNim/containers which is shared/adjusted in similar ways
+
+[[!meta author=yoh]]
+[[!tag projects/repronim]]
+

Added a comment
diff --git a/doc/bugs/file_field_omitted_from_import_--json-progress/comment_1_bea5a95e6fe120e1ee8c87ce6c55f161._comment b/doc/bugs/file_field_omitted_from_import_--json-progress/comment_1_bea5a95e6fe120e1ee8c87ce6c55f161._comment
new file mode 100644
index 0000000000..369a46d51f
--- /dev/null
+++ b/doc/bugs/file_field_omitted_from_import_--json-progress/comment_1_bea5a95e6fe120e1ee8c87ce6c55f161._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="jstritch"
+ avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
+ subject="comment 1"
+ date="2024-02-01T20:49:12Z"
+ content="""
+I'm not convinced the remote name should appear in the command field of the nested action. The whole point of using JSON is to avoid parsing strings.
+
+Adding a separate field with a string array of the command arguments would be ok, but I'm not requesting it now.
+"""]]

diff --git a/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch.mdwn b/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch.mdwn
new file mode 100644
index 0000000000..91b45aef8f
--- /dev/null
+++ b/doc/bugs/export_doesn__39__t_copy_from_adjusted_branch.mdwn
@@ -0,0 +1,32 @@
+### Please describe the problem.
+
+Exporting to a directory remote with exporttree=yes works if the source repository is not on an adjusted branch.
+
+
+If, before the export, git annex adjust is invoked with --unlock or --lock (the only two I tested) the new files are not copied.
+
+FWIW, annex.adjustedbranchrefresh is set true.
+
+### What steps will reproduce the problem?
+
+git annex export --json-progress --to export-special-remote main:Music
+
+### What version of git-annex are you using? On what operating system?
+
+10.20231228-g5540f42e21afce6947d5410c5f18b178de6c336a
+
+on Linux
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes, I am developing the easy-git-annex interface.

Added a comment: combine worm with partial hash
diff --git a/doc/backends/comment_35_8ac9d37cc2a7b0b361cf686fe27192dd._comment b/doc/backends/comment_35_8ac9d37cc2a7b0b361cf686fe27192dd._comment
new file mode 100644
index 0000000000..7c403b88af
--- /dev/null
+++ b/doc/backends/comment_35_8ac9d37cc2a7b0b361cf686fe27192dd._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="windfish@91c7bac51a452080189a071094cb6f914fe3435b"
+ nickname="windfish"
+ avatar="http://cdn.libravatar.org/avatar/648c1b6a023d94a30193b1b428c8d373"
+ subject="combine worm with partial hash"
+ date="2024-02-01T11:53:54Z"
+ content="""
+would it be possible to provide a combined backend of worm + partial hash? i'd imagine that this would make the backend faster than merely hashes while also lower the probability of erroneously identifying two different, but worm-equivalent files.
+"""]]

Added a comment
diff --git a/doc/bugs/Push_to_special_remote_includes_unwanted_files/comment_2_44143f10ac80a979513477fbac2637f3._comment b/doc/bugs/Push_to_special_remote_includes_unwanted_files/comment_2_44143f10ac80a979513477fbac2637f3._comment
new file mode 100644
index 0000000000..bbfc19da02
--- /dev/null
+++ b/doc/bugs/Push_to_special_remote_includes_unwanted_files/comment_2_44143f10ac80a979513477fbac2637f3._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="jstritch"
+ avatar="http://cdn.libravatar.org/avatar/56756b34ff409071074762951d270002"
+ subject="comment 2"
+ date="2024-01-31T22:12:47Z"
+ content="""
+Yes, copying .h but not .c would be an example. It makes a mess of the special remote's directory structure.
+
+Export has the same problem, so there is no way to copy files to the special remote without making a mess.
+
+Import does it correctly going the other way by *not* grabbing files *wanted* by the repository but *not wanted* the special remote. I believe symmetry is desirable in this case.
+
+Please provide a way to copy to a special remote with import-like filtering.
+
+"""]]

question about migrating URLs between remotes
diff --git a/doc/forum/how_to___34__move__34___URL_between_remotes__63__.mdwn b/doc/forum/how_to___34__move__34___URL_between_remotes__63__.mdwn
new file mode 100644
index 0000000000..bdf73143da
--- /dev/null
+++ b/doc/forum/how_to___34__move__34___URL_between_remotes__63__.mdwn
@@ -0,0 +1,6 @@
+In [backups2datalad](https://github.com/dandi/backups2datalad/pull/21) we are implementing support for "embargoed" data -- files which would require authorization initially. To provide authentication support we enable/configure `git-annex-remote-datalad` to handle them.  But in the future, when data gets unembargoed we would need to remove/disable `datalad` special remote and migrate (the same) URLs to no longer be associated with it and rather be handled by `web` remote.  Generally the use case might desire moving URLs between remotes (e.g. from `datalad` to `datalad-next`), or migrating from `web` into an external remote.
+
+Is it already possible reasonably well, or would require messing with `.web` files in the git-annex branch?
+
+[[!meta author=yoh]]
+[[!tag projects/dandi]]

Initial desire for `addurl --raw-except`
diff --git a/doc/todo/addurl_--raw-except_REMOTEs__63__.mdwn b/doc/todo/addurl_--raw-except_REMOTEs__63__.mdwn
new file mode 100644
index 0000000000..757beb5f92
--- /dev/null
+++ b/doc/todo/addurl_--raw-except_REMOTEs__63__.mdwn
@@ -0,0 +1,5 @@
+In [backups2datalad](https://github.com/dandi/backups2datalad/pull/21#issuecomment-1917911205) we ran into a behavior that `addurl --raw` does not only not considers all the fancy handling for youtube and .torrents, but also disregards our (`git-annex-remote-datalad`) external special remote (although still CLAIMURLs it first).
+I think generally we would still prefer to use `--raw` as to avoid possible side-effects when someone manages to add some `.torrent` file which we want to add as a file, not to download it. But we would like to explicitly allow interactions with our special remote. That is why I think the most viable solution would be to provide `--raw-except` which would be like `--raw` but allowing explicitly listed special remotes (or hardcoded keywords like `:torrents:`, `:youtube:`) to be handled if encountered.
+
+[[!meta author=yoh]]
+[[!tag projects/dandi]]