I have a server on a network (SRV1), several clients (CLI-N) on another and a archive repo on AWS Glacier. CLIs can access SRV1, but SRV1 cannot access CLIs network. I'd like to use git-annex to synchronize files between clients and archival on Glacier.
My current setup:
SRV1contains:transferrepository (T1)- glacier
archiverepo (G1)
CLI-01contains:user filesrepo (U1)transferrepository (T1)
CLI-02contains:user filesrepo (U1)transferrepository (T1)
CLI-N(...)OBS:
SRV1usesaws toolsandglacier-clito access Glacier- I would rather not to have to install
aws tools&glacier-clion client machines CLIs use ssh/rsync with secret key to accessSRV1
I would like that T1would serve as a gateway for a) synchronizing user files amongst clients; a.1) including moving and dropping files moved to archive by other clients; b) send archive files to G1; and c) get files from G1 and deliver to clients when content is moved out of archive.
As is seem close to working, but when a CLI pulls a file out of archive the recovery processes (c) fails. I expected transfer to fetch the file and then CLI would sync eventually, but does not seem to happen.
Is this even possible with this setup? If not, what changes would you recommend for the scenario above to work?
Transfer repositories unfortunately do not behave that way. You can setup G1 to get content sitting in T1 since T1 will store content until G1 has it. But, in terms of the other direction, doing a
git annex getfrom T1 won't ask SRV1 to get a file from G1 (I don't think).The easiest way I can think of to achieve what you are looking for would be to create your own special remote on SRV1 (lets call it P1) that proxies the requests the way you need. Using hook you could create a special remote that when asked for a file, ssh-es into SRV1 and then runs a script on SRV1 that does a
git annex geton G1. Then, a few hours later, you can do a get again to get the file.I think a reasonable workaround, that ensures some measure of safety would be to just create some read-only credentials for your Glacier. Then add your Glacier repo to all the clients, so you can at least do gets from your clients.
—Andrew
thanks for the input. I will rethink this a bit, might share if come up with something interesting.
kind regards.