I was looking for a good way to archive files to Glacier, and git-annex seems like an excellent approach.
After struggling for a while with the glacier-cli project, I just don't feel that it is well maintained.
- I have to manually symlink it in place of its dependency's glacier binary.
- There are patches to boto that are required for it to work, which are have never been merged. And it seems from the errors I am getting that similar edits are now required in many more places.
- It is locked into supporting only an older version of boto.
That just seems like not a great place to be starting out with an archiving project.
I was wondering if you thought it might be feasible to get rid of that dependency with something else, perhaps this haskell aws project that seems to be actively maintained?
https://github.com/aristidb/aws
There's been an open feature request on the haskell aws for 5 years to support glacier https://github.com/aristidb/aws/issues/81 at this point I am doubtful anything will ever happen there.
The most likely approach seems to be to use the amazonka library, which supports S3 with a Glacier lifecycle. git-annex's S3 remote could be rewritten to use that.
I think it should be possible to set up a S3 bucket and configure it with some web tool to have a Glacier lifecycle, and then use the existing git-annex S3 support to access it. Except for the problem documented at ?strong> Restore s3 files moved to Glacier.
Switching to amazonka would have the benefit of supporting SSL traffic for S3, which is a feature I would enjoy (see git-annex forum thread No SSL traffic for S3?). I have no experience with the amazonka library though, so I can't comment on it.
—Andrew
@abimelech, why don't you work up a branch that makes the glacier special remote be implemented with amazonka. If you want to start with a separate utility that git-annex runs, that would be ok, but I think that the goal should be for git-annex to end up using amazonka itself.
Once that happens, switching the S3 special remote from aws to amazonka would be a natural next step, which I'd be inclined to work on myself. Seems easier to take this path than an immediate leap to amazonka for everything.
I've never used git-annex, so I'm not particularly enthusiastic about developing for it. If someone could describe what glacier actions are needed I could maybe code something up. In any case, wouldn't we want to wait until a version of amazonka with functioning glacier support is released? Unless you'd want to add my git fork of the repo as a dependency...