Recent comments posted to this site:

comment 1

Now that external special remotes can support importtree, it does seem like all that is missing to have this in an external special remote is a way to set thirdPartyPopulated.

Comment by joey
comment 1

rclone would be another use case, some of its remotes support server-side generation of md5, sha1, or other hashes.

Comment by joey
comment 1

I feel like I had asked similar question in the past but failed to find it or an answer ;-)

Indeed it feels like it would be great to be able to just declare to always do only "unlocked" mode operations in a given git-annex repo, and thus potentially keeping it always in the "unlocked" mode (even when on system with symlinks support), but without requiring any adjusted/ "synchronization" altogether.

Comment by yarikoptic
comment 4

I have updated export and import appendix to document this new interface. I decided to not try to support a combined exporttree=yes and importtree=yes for external special remotes because noone has ever stepped forward to implement an external special remote that can safely do that. I know it will be much easier and safer to implement importtree only external special remotes.

The external special remote interface for this still needs to be implemented. Work has begun in the importonly_external branch.)

Comment by joey
comment 9

This is now possible to implement, since git-annex has an interface for importtree-only special remotes.

To list, it could avoid checksums and use:

rsync -a --dry-run --out-format='%l|%M|%L|%n' $repo empty-directory

The '%l' field is the file size and combined with the '%M' field's mtime, would constitute the ContentIdentifier. Rsync uses a hardcoded mtime format like "2026/07/13-12:49:50" which git-annex could parse and convert to seconds-from-epoch to save a little bit of space.

The '%L' field will be non-empty when it's a symlink, and those would be skipped.

The '%n' field is the filename, which may be prefixed with a common top-level directory or module name. Note that rsync escapes some characters, eg \#012 is \n.


Size+mtime as a ContentIdentifier is good enough, it only would prevent git-annex from noticing and updating the import when:

  1. Two files with the same size and mtime got swapped with one-another.
  2. A file is modified in place without updating the mtime or changing its size.
Comment by joey
comment 3

This is implemented at the haskell level. But the external special remote protocol doesn't support it yet.

Comment by joey
comment 2

I think it would actually be better to modify Types.Remote and add a new interface specifically for this, than to try to jam it into an existing interface it was not designed for.

Comment by joey