The external special remote protocol allows the following responses to TRANSFER RETRIEVE {key} {file}:
TRANSFER-SUCCESS RETRIEVE {key}TRANSFER-FAILURE RETRIEVE {key} {message}
I propose a third response: TRANSFER-REDIRECT-URL RETRIEVE {key} {url}
This will permit the following use cases:
1) Make a request against an authentication server that provides a short-lived access token to the same or a different server. The authentication server does not need to relay the data. 2) Deterministically calculate a remote URL (or local path) without reimplementing HTTP fetch logic, taking advantage of the testing and security hardening of the git-annex implementation.
This seems like a good design to me. It will need a protocol extension to indicate when a git-annex version supports it.
It occured to me that when
git-annex p2phttpis used and is proxying to a special remote that uses this feature, it would be possible to forward the redirect to the http client, so the server would not need to download the object itself.A neat optimisation potential, although implementing it would cut across several things in a way I'm unsure how to do cleanly.
That did make me wonder though, if the redirect url would always be safe to share with the client, without granting the client any abilities beyond a one-time download. And I think that's too big an assumption to make for this optionisation. Someone could choose to redirect to an url containing eg, http basic auth, which would be fine when using it all locally, but not in this proxy situation. So there would need to be an additional configuration to enable the proxy optimisation.
One problem with this design is that there may be HTTP headers that are needed for authorization, rather than putting authentication in the url.
I think we may have talked about this at the hackfest, and came down on the side of simplicity, supporting only an url. Can't quite remember.
It might also be possible to redirect to an url when storing an object. There it is more likely that a custom http verb would be needed, rather than PUT.
I think that protocol design should leave these possibilities open to be implemented later. So, I'm going with this:
Which leaves open the possibility for later things like:
TRANSFEREXPORT, in the "simple export interface" also uses TRANSFER-SUCCESS/TRANSFER-FAILURE, and should also support this extension.