ATM git-annex heavily relies on credentials needed by special remote(s) (S3, WEBDAV) to be passed via environment variables.
- main shortcoming: those environment variables (e.g. AWS_ACCESS_KEY_ID) do not allow for different set of credentials to be provided to be used e.g. by multiple special remotes of the same
typeand would require multiple git-annex invocations to get through all of them - user experience:
- there is no UI to prompt user to enter needed/desired secrets. That also relates to software relying on git-annex, such as datalad, as it somehow needs to know/discover that underlying call to
git-annexapparently needs some credentials to be passed via env vars - users then (typically) need to hardcode values of those secrets verbatim in their scripts (instead of using some more secure credentials store)
- there is no UI to prompt user to enter needed/desired secrets. That also relates to software relying on git-annex, such as datalad, as it somehow needs to know/discover that underlying call to
Given those (and possibly other) shortcomings, I thought it would have been great if git-annex supported some "credentials" storage/prompting/querying mechanism which would address main shortcoming and improve user experience. May be git-annex could use git-credential mechanism. If not, then may be have some protocol created to allow for external git-annex-credential-* helpers?
For reference, here is a PR in DataLad https://github.com/datalad/datalad/pull/5796 to provide git-credential-datalad helper so git could query our (datalad) credentials store.
But git-annex does allow for multiple special remotes of the same type that need different creds. At least for S3, when you run
git-annex initremoteit copies the env var values into a file, so you only need to have the env var set when running initremote, and all subsequent use of the remote will use the value it cached (unless you keep the env vars set, then it does use the env var).And there is a UI to prompt the user for the creds:
Perhaps initremote of S3/glacier/webdav remotes could query git credential when the env vars are not set. Those are the only 3 built-in remotes that use the env var creds system actually.
Fundamentally, what kind of authentication information a special remote needs depends on the service it's talking to, and there is a very large scope of possibilities, not only the simple username+password that git credential is designed for. Even S3 has a third possible credential that is sometimes needed and would not fit into git credential (
AWS_SESSION_TOKEN). Other remotes such a borg don't even use git-annex to handle credentials, but run a program that prompts for whatever it needs, or rely on other stored data like ssh keys. A remote could just as well need a hardware token, or one time password, or biometric id, whatever. So git-annex has to leave authentication up to the remotes to prompt for in whatever way makes sense for them, aside from the simple username+password case.Since
git-annex initremotedisplays an error message when it needs creds, does datalad not forward that error message to the user?There seems to be a significant disconnect between what you're describing and what I've seen of and know of creds usage in git-annex. Gonna need more information.