I'm having troubles trying to setup something like this:
- Setup a git-annex locally on my laptop with multiple special remotes.
- These are mainly free cloud services mounted with git-annex-remote-rclone
- Set the number of copies to 2 (so I can have some redundancy on my annexed files, and spread two different copies into two random special remotes)
- Use my local annex folder, but due to the (potentially) large size of it, be able to "git annex drop some/files" while maintaining the numcopies:2 setting
- Set it up so that once I copy/create a file in the annex folder, automatically replicated twice (since numcopies:2) to any of my special remotes.
So far, I have setup my local repo as repository group: client, and all my special remotes as "incremental backup" (since I do not wanted them to each hold a full backup of my whole annex, since it might be much larger than the available space provided by the cloud provider, Box for example).
I had numcopies set to 2, but this only copies one version of each file to another special remote (I assume the other copy is the one in my local repo)
Setting numcopies to 3, copies to two special remotes (plus my local copy) but "git annex drop some/file" does not work, complaining that only two other copies are available, so it is not working as expected.
I'm sure I'm missing something obvious here but how would I set this up so it works as automatically/transparent as possible?
Maybe I could utilize the "/archive" folder and move files there, but i hate destroying my directory structure by moving files there, and I would rather "drop" the files in place, and "get" them when i need them...
A big thanks to the collective git-annex brain out there...
git annex untrust here
will tell git-annex not to count the local copy towards
numcopies
.You should not need to untrust any repository to do this kind of thing. Only use untrust if you really expect that a repository could lose data at any time. So while Ilya's suggestion will technically work, I don't think it's a good idea.
By setting the special remotes to "incremental backup", you've told git-annex you only want one copy of the file spread amoung all of them whenever possible. So you've shot yourself in the foot there if your goal is to have more than one copy of the file amoung the special remotes.
The easiest thing to do is pick some of the special remotes to be "archive" and keep the rest "incremental backup", and each file will be stored twice, once in each of those two different types of repositories.
There is also a paragraph in standard groups that explains how to configure archive repositories to keep more than one copy, and following this could let you use only archive repositories and have them keep as many copies as you eventually want:
Note that if you want to archive multiple copies (not a bad idea!), you can set
git-annex groupwanted archive
to a version of the above preferred content expression with a larger number of copies than 1. Then make the archive repositories have a preferred content expression of "groupwanted" in order to use your modified version.Ie: