Recent comments posted to this site:
Turns out Remote.Directory did not need to add the size at all. I've fixed it.
Heyo,
Is it possible to assign multi-file torrent links to files, without needing to download the entire torrent again using Annex?
Consider the following scenario:
A multi-file torrent has been downloaded in the past, say, using qBittorrent...
... I then add the files to Git Annex, and would like to efficiently register torrent/magnet URLs for them...
Annex maps files to a multi-file torrent using #<n> suffixes, so I've tried the following:
aria2c --show-files <torrent file>-- to get the numbers for each filegit annex addurl --file "file.mp4" "magnet:...#<N>"-- to map the file to its original torrent/magnet
The following error occurs:
(downloading torrent file...) git-annex: That url contains multiple files according to the bittorrent remote; cannot add it to a single file.
If there's any information or clarification needed, please don't hesitate to let me know 
$ git annex migrate --remove-size
migrate data.bin
git-annex: failed creating link from old to new key
That happens when the file is not present in the local repository. If you get the file first it will work.
Migrate needs the content present so it can populate the new key. Otherwise, there can be situation where the new key never ends up being populated with the content.
This bug only happens with the directory special remote and not others.
The problem is that the directory special remote implements importKey,
and explicitly adds the size back!
15000dee07a06d04285351616915794bd6ec7f14 explains why
it needs to do that.
Unfortunately it seems like I can't even remove the size after the import:
$ git switch --detach importdir/import
HEAD is now at 6d3aa43 import from importdir
$ git annex migrate --remove-size
migrate data.bin
git-annex: failed creating link from old to new key
failed
migrate: 1 failed
[ble: exit 1]
in fears against modification of files in git-annex branch directly, here is the commands to 'check'
$> f=Чат_рулетка/2026-03-05-_армянин_за_путина._Армянин_из_россии_Воевал_против_Украины.mkv
$> key=$(readlink "$f" | xargs basename); alog=$(git ls-tree -r git-annex | grep "$key" | awk '/.web$/{print $4;}'); git show "git-annex:$alog"
1772708470s 1 https://www.youtube.com/watch?v=0fcKYGsBZxU
First I tried to fix via re-addurl, and we do get some difference:
$> git rm "$f"; git annex addurl --no-raw --file "$f" "$url" rm 'Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv' addurl https://www.youtube.com/watch?v=0fcKYGsBZxU (using yt-dlp) (to Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv) ok (recording state in git...) $> git status On branch master Your branch is up to date with 'origin/master'.
Changes to be committed: (use "git restore --staged ..." to unstage) modified: Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv
$> git diff --cached diff --git a/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv b/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv index e59a58c35..e12bb1280 120000 --- a/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv +++ b/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv @@ -1 +1 @@ -../.git/annex/objects/KQ/x1/URL-s0--https&c%%www.youtube.com%watch,63v,610fcKYGsBZxU/URL-s0--https&c%%www.youtube.com%watch,63v,610fcKYGsBZxU \ No newline at end of file +../.git/annex/objects/wq/jM/URL--yt&chttps&c%%www.youtube.com%watch,63v,610fcKYGsBZxU/URL--yt&chttps&c%%www.youtube.com%watch,63v,610fcKYGsBZxU \ No newline at end of file
for which I did not really care as long as I got that file if metadata transferred, but it didn't:
$> git commit -m 'redownloaded "unlucky" video for which no yt: was added' $f [master 379d379ea] redownloaded "unlucky" video for which no yt: was added 1 file changed, 1 insertion(+), 1 deletion(-) $> git annex metadata "$f" metadata Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv
ok ```
Also when I used recent after 202601 version which would auto-upgrade to use VURL key difference was to switch from URL to VURL. Could you please point me on where to read up on VURLs and their benefit for relaxed URLs?
then I tried to do the dance with unregisterurl, rmurl, addurl, which ended up having
$> key=$(readlink "$f" | xargs basename); alog=$(git ls-tree -r git-annex | grep "$key" | awk '/.web$/{print $4;}'); git show "git-annex:$alog"
1772750261s 0 https://www.youtube.com/watch?v=0fcKYGsBZxU
1772750309s 1 yt:https://www.youtube.com/watch?v=0fcKYGsBZxU
and for which I still was not able to get it:
$> git annex get "$f"
get Чат_рулетка/2026-03-05-_армянин_за_путина._Армянин_из_россии_Воевал_против_Украины.mkv (from web...)
Verification of content failed
Unable to access these remotes: web
No other repository is known to contain the file.
(Note that these git remotes have annex-ignore set: origin)
failed
get: 1 failed
git annex get "$f" 8.22s user 3.63s system 112% cpu 10.505 total
although I think it did fetch it. But i guess it is because of the -s0 in the original key! So original way with git rm + addurl was kinda legit as it also fixed up the URL BUT it lost the metadata for the key.
Is there a quick way to copy metadata from another key? (like internally it does for the same path?)
Or is there a better way to 'fix up URL/key' which would you recommend Joey so I could retain metadata?
in fears against modification of files in git-annex branch directly, here is the commands to 'check'
$> f=Чат_рулетка/2026-03-05-_армянин_за_путина._Армянин_из_россии_Воевал_против_Украины.mkv
$> key=$(readlink "$f" | xargs basename); alog=$(git ls-tree -r git-annex | grep "$key" | awk '/.web$/{print $4;}'); git show "git-annex:$alog"
1772708470s 1 https://www.youtube.com/watch?v=0fcKYGsBZxU
First I tried to fix via re-addurl, and we do get some difference:
$> git rm "$f"; git annex addurl --no-raw --file "$f" "$url" rm 'Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv' addurl https://www.youtube.com/watch?v=0fcKYGsBZxU (using yt-dlp) (to Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv) ok (recording state in git...) $> git status On branch master Your branch is up to date with 'origin/master'.
Changes to be committed: (use "git restore --staged ..." to unstage) modified: Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv
$> git diff --cached diff --git a/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv b/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv index e59a58c35..e12bb1280 120000 --- a/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv +++ b/Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv @@ -1 +1 @@ -../.git/annex/objects/KQ/x1/URL-s0--https&c%%www.youtube.com%watch,63v,610fcKYGsBZxU/URL-s0--https&c%%www.youtube.com%watch,63v,610fcKYGsBZxU \ No newline at end of file +../.git/annex/objects/wq/jM/URL--yt&chttps&c%%www.youtube.com%watch,63v,610fcKYGsBZxU/URL--yt&chttps&c%%www.youtube.com%watch,63v,610fcKYGsBZxU \ No newline at end of file
for which I did not really care as long as I got that file if metadata transferred, but it didn't:
$> git commit -m 'redownloaded "unlucky" video for which no yt: was added' $f [master 379d379ea] redownloaded "unlucky" video for which no yt: was added 1 file changed, 1 insertion(+), 1 deletion(-) $> git annex metadata "$f" metadata Чатрулетка/2026-03-05-армянинзапутина.АрмянинизроссииВоевалпротивУкраины.mkv
ok ```
Also when I used recent after 202601 version which would auto-upgrade to use VURL key difference was to switch from URL to VURL. Could you please point me on where to read up on VURLs and their benefit for relaxed URLs?
then I tried to do the dance with unregisterurl, rmurl, addurl, which ended up having
$> key=$(readlink "$f" | xargs basename); alog=$(git ls-tree -r git-annex | grep "$key" | awk '/.web$/{print $4;}'); git show "git-annex:$alog"
1772750261s 0 https://www.youtube.com/watch?v=0fcKYGsBZxU
1772750309s 1 yt:https://www.youtube.com/watch?v=0fcKYGsBZxU
and for which I still was not able to get it:
$> git annex get "$f"
get Чат_рулетка/2026-03-05-_армянин_за_путина._Армянин_из_россии_Воевал_против_Украины.mkv (from web...)
Verification of content failed
Unable to access these remotes: web
No other repository is known to contain the file.
(Note that these git remotes have annex-ignore set: origin)
failed
get: 1 failed
git annex get "$f" 8.22s user 3.63s system 112% cpu 10.505 total
although I think it did fetch it. But i guess it is because of the -s0 in the original key! So original way with git rm + addurl was kinda legit as it also fixed up the URL BUT it lost the metadata for the key.
Is there a quick way to copy metadata from another key? (like internally it does for the same path?)
Or is there a better way to 'fix up URL/key' which would you recommend Joey so I could retain metadata?
Started developing this in the ephemeral branch.
It seems to also make sense to allow DELEGATE as a response to WHEREIS.
I'm on the fence about delegating GETORDERED. Probably most remotes won't bother to respond to GETORDERED at all, and the only time it makes sense to delegate it is when always delegating to the same type of special remote. If delegating to different special remotes at different times, it doesn't make sense to delegate it to a single on of them.
Similarly I don't think it makes sense to delegate GETINFO unless only delegating to a single special remote. Will probably wait to see if someone has a use case before supporting GETINFO, GETAVAILABILITY, CLAIMURL, etc.
Add to external special remote protocol, enabled by the DELEGATE
extension:
DELEGATE type=whatever ephemeral=yes|no [params]
Which can be used as a response to TRANSFER, REMOVE, CHECKPRESENT, TRANSFEREXPORT, CHECKPRESENTEXPORT, REMOVEEXPORT, REMOVEEXPORTDIRECTORY, RENAMEEXPORT
This initializes a delegate special remote in a private namespace, and uses it to perform the operation.
Subsequent uses of DELEGATE with the same configuration avoid the overhead of reinitialization.
With ephemeral=yes, the delegate is automatically removed when the external special remote program shuts down (unless another one is using it.) With ephemeral=no, the delegate remains initialized for use next time.
Is a non-ephemeral aspect visible/accessible outside the context of the special remote that set it up? Would it appear as a regular special remote for a CLI user, as if they ran initremote?
I think it would be best for it not to be visible to the user. Since these
remotes can still set their own git configs though, they will necessarily
show up in git remote list. (Any git config remote.foo.bar setting is
enough for that.) It would be possible for git-annex to not treat them as
valid remotes when used outside of the aspect context though.
Easiest would be to set annex-ignore on them.
It would be possible to point GIT_CONFIG at a different config file
when setting up and using the ephemeral special remote. That would have
the problem though that if the special remote looks at some user-set
git configs, it wouldn't see them. An example that comes to mind
that a special remote would be expected to see is the
"credential.helper" configuration. Maybe git-annex could merge .git/config
into the ephemeral remote's version when using it? Seems complex and
potentially slow though.
(BTW, Even ephemeral aspects will be user-visible while git-annex is running.)
At which stage would INITASPECT (have to) be used? The PREPARE stage, I guess.
I think it could be used at any point.
How expensive could INITASPECT be? Would it (immediately) trigger init/prepare of the aspect-remote?
As expensive as git-annex initremote initially, but subsequenty close to
a noop when the remote configuration includes emphemeral=no
Also, calling it repeatedly in the same session with the same configuration should be a noop after the first time. So you could call it immediately before USEASPECT.
That does suggest a simplification: Rather than having a separate INITASPECT command:
USEASPECT type=whatever ephemeral=yes|no [params]
Neat, this avoids needing to name the aspect! And avoids any problem with the aspect name having been used before with a different config.
It also means that any failure to initialize will necessarily make the USEASPECT response be an error message, so error handling takes care of itself.
git-annex would still need a remote name internally; it could eg hash the configuration to get a name.
I'm inclined to go with this simplification.
Do I understand correctly that it would be possible to set the active aspect on a per-key and per-operation basis?
It's per-operation. If you want different aspects for different types of keys it would be up to you to pick between them.