age v1.0.0 has been released: https://github.com/FiloSottile/age/releases/tag/v1.0.0

I have not been able to reproduce this, it works for me.

joey@darkstar:~/tmp>git init a1
Initialized empty Git repository in /home/joey/tmp/a1/.git/
joey@darkstar:~/tmp>cd a1
joey@darkstar:~/tmp/a1>git annex init
init  ok
(recording state in git...)
joey@darkstar:~/tmp/a1>date > foo
joey@darkstar:~/tmp/a1>date > bar
joey@darkstar:~/tmp/a1>git annex add
add bar
ok
add foo
ok
(recording state in git...)
joey@darkstar:~/tmp/a1>git commit -m add
[master (root-commit) fdd215f] add
 2 files changed, 2 insertions(+)
 create mode 120000 bar
 create mode 120000 foo
joey@darkstar:~/tmp/a1>rm foo
joey@darkstar:~/tmp/a1>git commit -a -m rm
[master 1f84de1] rm
 1 file changed, 1 deletion(-)
 delete mode 120000 foo
joey@darkstar:~/tmp/a1>git annex unused
unused . (checking for unused data...) (checking master...) 
  Some annexed data is no longer used by any files:
    NUMBER  KEY
    1       SHA256E-s30--11de88d41da5962620f4a0590a577710e067d72e0f24220c4a4d2e81d594388a
  (To see where this data was previously used, run: git annex whereused --historical --unused

  To remove unwanted data: git-annex dropunused NUMBER

ok
joey@darkstar:~/tmp/a1>cd ..
joey@darkstar:~/tmp>git clone a1 a2
Cloning into 'a2'...
done.
joey@darkstar:~/tmp>cd a1
joey@darkstar:~/tmp/a1>git remote add a2 ../a2
joey@darkstar:~/tmp/a1>git annex move --unused --to a2
move SHA256E-s30--11de88d41da5962620f4a0590a577710e067d72e0f24220c4a4d2e81d594388a (to a2...) 
ok

You did say it's not moving "some files" which suggests maybe it is moving other ones? Do you see it move any unused files at all?

Do other commands that use the same --unused option work? Eg, does git annex whereis --unused list them?

The only way I can reproduce this behavior is if the remote has the same uuid as the current repository. Then any move is a no-op and it avoids operating on the files at all, the same as your output shows. So it seems possible that could be your real problem.

Since that old version of git-annex, it has changed to using a different protocol than rsync for transfers over ssh. So the rsync options no longer apply to that. They are still used when git-annex does use rsync, either a rsync special remote or a server with too old a version of git-annex to use the new protocol.

I think the main thing lost by this is bandwidth throttling. There is an open todo at bwlimit to implement that in a way that will work more broadly than rsync's --bwlimit.

Maybe also --ipv4/--ipv6, but ssh configs can probably be used to accomplish the same thing as that.

It is perfectly safe. It can take some additional configuration to get the clients sending the objects that the other clients want to the server.

Interesting, I had not known that ASIC design would involve the kind of large files git-annex would be useful for.

I think you may want to use git annex unused --used-refspec='+refs/heads/*:+HEAD:reflog' That adds all versions that are in the reflog. Then you can can configure git to control how much reflog to keep around. (Seegit-gcman page)

The other possibility is a new git-annex feature, git-annex whereused --unused --historical
After you run git annex unused, you can run that to display each unused key, along with the git rev where that key was found to be used.

The git rev looks like eg "master~4:filename" or "HEAD@{4}:filename". It will usually be the most recent use, although it prefers older uses that made it into a branch over any revs from the reflog. So you can filter for keys with numbers > 8 or whatever, and get only the older versions of files. Then pipe the keys into git-annex dropkey --batch.

Improving git annex unused to be able to do this kind of filtering itself is also a possibility. (See also ?strong>move unused files older than x which was asking for a similar kind of thing with a similar response).

Each git add has to run a new git annex smudge process. git commit will often run it as well. This is discussed in detail in git smudge clean interface suboptiomal.

According to https://github.com/datalad/datalad/issues/5994#issuecomment-923744895 git-annex 8.20201127 had the same performance that you are seeing with the current version. So it cannot be any changes in the past year that impacted git-annex performance.

But https://github.com/datalad/datalad/issues/5317 seems like it would have been using around the same version of git-annex, and the timing there was much faster. Unfortunatly the issue does not say what version of git-annex was used, but it seems likely it would have been around 8.20201127 since that was released 2 months prior.

So are the timings in those issues comparable, or is it an apples/oranges comparison with different hardware or OS version?

And if the timings between those issues are not comparable, what exactly is this new issue comparing with when it says the peformance has worsened?

I've been using git-annex with very similar setups for many years and never had any problems. I can't say anything about bare repos being more or less risky than non-bare ones, I've been using both on my server. Repos that can't see each other is a very common setup.

Confirming that assistant also adds files unlocked for me and I haven't found a way to change this (this is one of the main reasons I've moved away from using assistant to manual sync). If there is a way to change this behavior, I'd be glad to know about it!

The metadata solution worked well, except that my cloud remote is unable to run the assistant to pull files from the desktop, and the assistant in the desktop was not pushing. That is fixed by running git annex sync --content cloud every few minutes in the desktop.

That's also not it:

### git branch
  git-annex
* master
  synced/master