Please describe the problem.
I have a git annex repository at ~/doc
which has a remote stored on a second drive at /media/backup/doc.git
, which is a bare git annex repo. The webapp is run from ~/doc
. Changes are made to /media/backup/doc.git
by syncing from other computers, but then those changes to the remote are not automatically propagated to ~/doc
by the webapp.
The use case that lead to this situation is syncing several computers to a large hard drive on a server, utilizing a bare repo as a full backup, and then wishing to have a client repo in the home folder on the server. So other computers sync their repos to /media/backup/doc.git
and then any changes should be propagated to ~/doc
on the server.
Changes made to ~/doc
are synced to the backup repository on the backup hard drive, and those changes are almost immediately synced to the other machines by the webapp running on those respective machines. Those computers are configured to connect to /media/backup/doc.git
on the server using SSH only (and git-annex-shell on the server), and do not use XMPP.
What version of git-annex are you using? On what operating system?
The amd64 pre-built tarball, git-annex version: 6.20160613-g35dbe35, running on Ubuntu 16.04.
Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Migrated all of my syncing to git annex about 10 months ago and haven't had any significant trouble
Can you please provide the following info? For curiosity's sake.
When you are viewing the repos in the web app, on the right hand side for each repo is a drop down menu labelled "actions". Please click there and go to the option labelled "Edit". Click this option, and on the resulting page there should be a field labelled "Repository group". Can you tell us the "Repository group" for the repos you mentioned? What is the "Repository group" for the backup on the secondary drive, the repo in your home folder and the remote clients? My hunch is that these group types are not set up to reflect the configuration you want.
full backup
and the repo at~/doc
isclient
.And the other repositories (which are syncing correctly), they are also marked as client? Clients should sync with each other whenever they can connect to each other, but perhaps your client repositories cannot connect to each other?
Here is a comment from Joey about a similar setup. He says you might need to set up an XMPP account for the various clients to message each other when they have new changes? But that was a long time ago. And anyways it seems like the webapp would be doing this automatically (yet in your post you say the computers do not use XMPP?).
I think everyone would benefit from more info still. For example, what does the webapp look like when you are syncing this repo? Are all the clients and the backup on the same page? Is each client only combined to the backup itself and no other repos? What kind of remote repos are the other computers?
Hope you don't mind the questions, I'm also curious about this issue myself. Thanks for bringing it up.
Good questions. The setup is essentially a star topology where the central node is the full-backup, bare git repository on the external drive on the server. It's not encrypted, if that matters. Since it's a bare repo, nothing actually changes in that node except when changes are pushed from elsewhere.
Every other node is configured as a client and is connected to that repo on the server via SSH. When a change is made to one of the client repos, syncing is happening automatically and nearly instantaneously--not just to the server repo, but the changes are very quickly propagated to the other SSH-connected clients. (I think this is supposed to happen for SSH repos now without using XMPP. In the release notes for version 5.20140421: "This release begins to deprecate XMPP support. In particular, if you use the assistant with a ssh remote that has this version of git-annex installed, you don't need XMPP any longer to get immediate syncing of changes.")
But then there's this one other node in the star topology---the client repo (non-bare) that is actually on the same system as the bare, backup repo. When changes are made to the client, they are synced immediately to the backup repo, and those changes are quickly synced to the other client repos on other computers connected by SSH. The problem is changes in the other direction; when a client repo on a remote system is changed, the updates are synced to the backup repo on the server, but no farther. The webapp running on the server never (or only very slowly) syncs those changes to that client repo that is on the same system. The webapp on the server is running with that local client repo as the "main" repo (I mean, the upper right hand corner says "Repository: ~/doc", which is the client repo). All of the status messages are green and say that the repos are synced. You can, of course, force a sync from there, and and then the changes are noticed and propagated to the client.
I did some testing and it seems like repositories with non-bare remotes on the same filesystem are synced immediately using inotify or something. Maybe this just doesn't happen for bare repos? That's the variable that seems to make syncing not happen automatically, in my limited testing. I can test more to isolate exactly when it happens, but I was hoping Joey would know off the top of his head if this should be working or not.
This has nothing to do with repository groups; it's the git branch not getting synced by the assistant when master is updated in the bare repository.
This is easy enough to repoduce.
The assistant relies on
git annex remotedaemon
to notice changes in remotes, and the remotedaemon only supports remotes accessed via ssh, not on a local drive.The assistant has some special case handling for repositories on removable drives, which lets those be plugged in and syncs with them. Removable drives are also why the remotedaemon does not watch repositories on a local drive. If the drive is removable, the remotedaemon would keep it always busy, by having directories open for inotify, and so prevent a clean removal of the drive.
So, I don't think anything can be done about this, at least in the default configuration. One workaround it using a ssh url for the remote will make the remotedaemon watch it for changes.
I could add a non-default configuration setting that makes a remote on a local drive be watched for changes.
But, it's not clear to me why you'd want to have the changes be pushed in to a repository on a removable drive, rather than pushing/syncing with the repository where the assistant is running. The assistant would then sync changes it received on to the removable drive. That seems like a better setup, will still work when the drive is removed for a while, and is already supported.