In general, git-annex repositories that are "synchronized" (e.g. with the git-annex-sync command, whatever the backend) have a global namespace. Repositories will eventually converge to have very exactly the same content, generally using git's push/pull/merge mechanisms.
What if we do not wish to exactly have the same content across all repositories, but still want to share some objects?
An example use case here is content (e.g.
sharing, without having to deliberately collaborate over a globally
consistent set of objects in the
master branch. Think of a
decentralized conference proceedings repository where each
conference could add their own content to a conference-specific
repository, while at the same time allowing a unified view in another,
more centralized repository, or allowing users to pick and choose
which conference they would want content from.
While each repository could have its own distinct branch, all
repositories will see all those branches and this may affect content
retention, as git-annex may consider files to be "in use" because they
are on some remote branch, for example. Furthermore, I consider git
branching to be a rather advanced topic in git usage. While git-annex
uses those mechanisms (e.g. the
those are generally hidden from the user until something goes
wrong. Therefore I looked into providing a more straightforward
approach to this problem for my users and myself.
In my use case, I have the following repositories:
- repoA: my own curated media collection
- repoB: a third-party media collection
I do not wish for my local curated collection (repoA) to be completely synchronized with the third-party collection (repoB). This is because we may have different tastes and retention policies: while I archive everything, there are certain media I am not interested in. On the other hand repoB might keep only (say) the last month of media and disard older content but have a more varied collection, which only a subset is interesting to me. Yet I still want to access some of that content!
So I did the following to add the third party repository:
git remote add repoB example.net:repoB git annex sync --no-push repoB git annex get --from=repoB
This works well: I get the files from repoB locally. Of course, if repoB expires some files, this will be impacted locally, but I can always revert those choices without conflict, because I do not push those back.
The downside of the
--no-push option in git-annex-sync is that
it needs to be made explicit at each invocation of the
command. Furthermore, this option is not supported by the assistant,
which will happily sync the master branch to all remotes by default.
An alternative is to manually fetch and merge content:
git fetch repoB git annex merge repoB git reset HEAD^ # revert any possible changes upstream we don't want git commit
Needless to say this quickly becomes quite messy, but it's the amazing
level of control git and git-annex provides, which obviously comes
with its price in complexity. Such a method will also be ignored by
the assistant and further
To make sure those principles are respected in the assistant or a
git annex sync that may mistakenly be ran in that repository,
I need some special setting. There are the options I considered, in
.gitconfig or git-annex's config options:
assistantwill not sync content to the repository, but explicit
get --from=repoBwill still work.
assistant?) will not sync the git repository with the remote
remote.<name>.push=nothing: git won't push by default, unless branches are explicitly given, which may actually be the case for git-annex, so unlikely to work.
remote.<name>.pushurl=/dev/null: will completely disable any push functionality to that remote. any sync will yield the following error:
fatal: '/dev/null' does not appear to be a git repository [...] git-annex: sync: 1 failed
remote.<name>.pushurl=.: will push to the local repo instead. crude hack and may confuse the hell out of git-annex, but at least doesn't yield errors.
A similar approach to hacking the
pushurl is to make
read-only to the user. This however, may trigger the activation of
annex-ignore by git-annex and will otherwise yield the same warnings
Right now, I am using
annex-sync = false in
.git/config. I have
also configured the repository to be in the "manual" standard
group which will avoid copying
files into that repository:
$ git annex group repoB manual group repoB ok (recording state in git...) $ git annex wanted repoB standard wanted repoB ok (recording state in git...)
This is roughly equivalent to setting
annex-ignore = true, yet it
allows for more flexibility. I could, for example, create custom
content expressions to sync certain folders automatically.
A disadvantage of the
annex-sync settings is that it affects both
ways (push and pull), not just push, which is what I am interested
in. Although it could be argued that restricting both is fine here
because we want to manually review changes when we pull changes from
those remotes anyways.
The best approach may be to have git-annex respect the
remote.<name>.push=nothing setting. Another approach would be to add
that would match the
sync --[no-]push --[no-]pull flags.
Note that this is similar in concept to Bittorrent-like features, although here we assumes you already have some transport to share anything you need, yet still have to address the question of semi-synchronized git repositories in some way.
I would obviously welcome additional comments and questions on this approach. -- anarcat