Here's how to create a Amazon S3 special remote that can be read by anyone who gets a clone of your git-annex repository, without them needing Amazon AWS credentials.
If you want to publish files to S3 so they can be accessed without using git-annex, see publishing your files to the public.
Note: Bear in mind that Amazon will charge the owner of the bucket for public downloads from that bucket.
create public remote
First, export your Amazon AWS credentials:
# export AWS_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
# export AWS_SECRET_ACCESS_KEY="s3kr1t"
Now, create the remote:
# git annex initremote pubs3 type=S3 encryption=none public=yes
initremote pubs3 (checking bucket) (creating bucket in US) ok
The public=yes makes git-annex set a S3 ACL so the files in the bucket are publically readable. For git-annex to be able to download files from that bucket without needing your AWS credentials, you then need to tell it the url of the bucket. Find that url, and run:
# git annex enableremote pubs3 publicurl=...
In the above example, no encryption was used, but it will also work if you enable encryption=shared. Then files will be encrypted on S3, and anyone with a clone of the git repository will be able to download and decrypt them.
It's also ok to enable chunking when setting up the remote.
Now, copy some files to the remote, in the usual way, and push your git repository to someplace where someone else can access it.
use public remote
Once the S3 remote is set up, anyone who can clone the git repositry can get files from the remote, without needing any Amazon AWS credentials.
Start by checking out the git repository.
In the checkout, enable the S3 remote:
# git annex enableremote pubs3
enableremote pubs3 ok
Now, git-annex can be used as usual to download files from that remote.
sharing urls
You can also share urls to files stored in a public S3 remote to people
who are not using git-annex. To find the url, use git annex whereis
.
See S3 for details about configuring S3 remotes.
I'm setting up a repository using the following commands:
I then add something and export it to s3 using:
I then go on to a different computer and clone the repository and run:
and receive this message:
failed git-annex: get: 1 failed
It is my understanding that setting the remote to "public=yes" should allow me to download the files from the remote repository without entering the credentials again. Am I missing something?
Thanks!
It seems to me that you need to tell git-annex the publicurl to use to download files from this S3 remote. That's the url to the top of the S3 bucket. Run `git annex enableremote publics3 publicurl=...' and push, and it should then become available in every clone.
The documentation for publicurl says it's enabled by default when public=yes, but AFAICS it's never actually done that. And I'm not sure I want to hardcode those urls into git-annex. I've updated the documentation instead.