Hi,
Git-annex is really awesome. It has made my life really easier when having to move files around.
Yet, I have been struggling with a use case that I cannot get working with git annex.
In short, my request is: could it be possible to have --want-get and --want-drop accept a repository as argument to match the preferred content of that repository instead of here?
Now, let me explain why I need this:a
All my files are stored into a NAS accessible via a local network.
I have an annex in my desktop computer. Using preferred content (via "git annex wanted") and "git annex get|drop --auto", I am able to almost automatically handle what files are put into my computer. What I do is to "git annex wanted" to indicate what I want to be here and launch a home made script that basically does "git annex get --auto" and "git annex drop --auto".
Let's say I have a android phone to which I connect via ssh over adb. It contains a git repository but few files are in it. It has no wifi and so no access to the network, meaning no access to the NAS.
The links between annexes then looks like:
NAS <-> Computer <-> Phone
When I want to put a file into my phone, I generally launch "git annex get file" from my computer (then I get the file from the NAS) and "git annex copy --to phone file".
I want to be able to automatise this a bit by playing with preferred content (like I do with my computer). This means that I want to launch "git annex wanted" to edit the preferred content of the phone annex and then "git annex get --auto" and launch "git annex copy --auto --to phone". This way, when I am not in front of my computer, I can still from my phone run "git annex wanted here 'preferred content'" and hope for my synchronisation scripts (run in my computer) to put the good files into my phone.
Obviously those commands won't work since the git annex get --auto command will only get what my computer wants, not what my phone wants.
The intuitive (IMHO) way to do would be to launch:
git annex get --want-get phone
git annex copy --auto --to phone
git annex drop --auto
With "--want-get repository" meaning, "Matches files that the preferred content settings for the repository make it want to get.".
For the time being, I succeed in doing this with
OLD_WANTED=$(git annex wanted here)
git annex wanted here $(git annex wanted phone)
git annex copy --auto --to phone
git annex wanted ${OLD_WANTED}
git annex drop --auto
This is complicated and adds two extra commits in the git-annex branch (one for each setting of git annex wanted) each time I call the script.
What do you think?
Thanks for reading.
Well, I suppose this is doable, but the way this scenario is generally handled is to make a transfer repository (which your desktop is serving as here) have a preferred content expression that makes it want files that the client repositories (phones) want, until the files have reached the clients:
In your case you could have:
Where
$desktop_preferred_contentis whatever files the desktop actually wants on its own, and$phone_preferred_contentis a copy of the preferred content setting for the phone.To make this work, you also need to put your phone in the phones group.
git annex sync --contenton the desktop and it will automatically get files, copy the right ones to the phone, and drop any then-unwanted files.I agree with the fact that it answers my initial question.
Nevertheless, I guess it is really annoying to have to change the preferred content in two locations each time I change want to modify what is on my phone. Indeed, I quite often change what should be really present on my phone, depending on several factors (my mood, the time I will have in transports and the phase of the moon).
The reason why I use "git annex wanted" is that it is straightforward: I just launch : "git annex wanted phone include=some/file" and that's all.
With the solution you propose, I would have to each time additionally launch "git annex wanted here '(not inallgroup=phones and (include=some/file)) or ($desktop_preferred_content)'"
Where I probably have to previously launch "git annex wanted here" to remember what is the preferred content of my computer (and put it in $desktop_preferred_content).
Another option would be to run "git annex vicfg" and edit both fields manually, but IMHO this appears also to be too complicated relatively to the use case.
About your second comment, I really enjoy the idea if "git annex sync --content", but it is really long in big repositories. I guess it is because I cannot restrict the command to a directory like I do with get, drop, move and copy.
Besides, the use case wants to get the files whenever they are and copy them only to the phone. With git annex sync --content, the files are also put in the other repositories.
For the time being, I prefer falling back to explicit commands that are much faster when I know a directory to sync "git annex get --auto directory && git annex copy --auto --to phone directory".
For example, I just tried "git annex sync --content phone" and I killed it after 5 minutes and nothing was copied yet. With the set of two commands above, the synchronization of directory (get --to phone directory + drop --from here directory) took about 3 minutes.
I realized that the directory restriction described earlier is not clear. Let me explain it.
Say I have a big repository of files with the following structure.
A/... B/... C/... D/D1/... D/D2/... D/D3/...
Imagine that each of A, B, C and D contains a lot of files.
Now imagine that I have often put in the preferred content of my phone files in one of D1, D2 or D3. (for instance, include=D/D1/*)
I implicitly know that I can restrict the command to D, and I can rely on preferred content to know what file from D I have to put into my phone.
Then, I can run $ git annex get --auto D $ git annex copy --auto --to phone D $ git annex drop --auto D
This in my use case takes 3 minutes.
When I run $ git annex sync --content phone
git annex goes recursively through A, B and C and that takes a long time (much more than 3 minutes). This time is really wasted since I know I only want to sync files from D.
I cannot test the behavior of "git annex sync --content", but if I restrict the sync to phone, will it take the files from the NAS to put them on the phone? If I don't precise phone in the command line, it will try to sync with other repositories not in sync that I don't want to be in sync for the time being.