Backblaze announced that they now have S3 storage for buckets created after May 4th.
My current attempt to use it as an S3 remote has me with the following: export AWS_ACCESS_KEY_ID=keyID export AWS_SECRET_ACCESS_KEY=applicationID git-annex initremote backblaze type=S3 host=s3.us-west-000.backblazeb2.com (the full s3 endpoint) embedcreds=yes encryption=hybrid keyid=CA3D8351 bucket= protocol=https
and the output:
initremote backblaze (encryption setup) (to gpg keys: 41AAE7DCCA3D8351) (checking bucket...) (creating bucket in US...) git-annex: S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = ""}, s3ErrorCode = "AuthorizationHeaderMalformed", s3ErrorMessage = "Authorization header does not contain a Credential", s3ErrorResource = Nothing, s3ErrorHostId = Nothing, s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} failed git-annex: initremote: 1 failed
It may be that they are only supporting a newer version of the S3 protocol that does authorization differently. The S3 library git-annex uses defaults to V2 not V4.
I thought that library didn't properly support V4, but it seems it does, although I don't know if it works in all cases. So, I've added an initremote option for S3, signature=v4 .. give it a try.
I rebuilt git-annex with the new signature v4 patch and I got it to work.
I had to create a new application key that wasn't tied to a bucket or else I would get an 'access denied' error. I didn't set any prefix on the files uploaded.
AWS_ACCESS_KEY_ID=keyID AWS_SECRET_ACCESS_KEY=applicationKey git-annex initremote backblaze type=S3 host=S3-endpoint bucket=bucketID protocol=https signature=v4
This created a new bucket with the same bucketID as the one I previously created and intended to use, and with the name equal to the bucketID
I suspect if I had done bucket=bucketName and not bucketID I could've used a bucket specific key but now that it's set up it's not worth it to me to try to change it.
Thanks for verifying it works. I've updated using Backblaze B2 to mention this alternative, hopefully I got the config right. (Is the endpoint always the same?)
I don't know what the financial costs of the API calls work out to, but it's true that setting a remote to a higher annex-cost makes git-annex avoid using it when possible. And also true that trusting a remote will avoid git-annex making API calls to verify that it still contains content.
Bear in mind that trusting a repo can easily lead to data loss in situations where you remove content from it, which is documented in git-annex-trust. Indeed, recently git-annex has started warning about this when you try to trust a remote. If you are sure you'll never remove content from the remote, or will only ever access this remore from a single repository, you might be able to convince yourself you'll avoid situations where trusting it would lose data.