May be it is already available and thus not really todo (feel welcome to close then or I will), and/or I have asked before, but I failed to find it. Sorry if duplicating the issue.
In some scenarios we do know ahead of time url + size + filename (thus extension) + corresponding checksum for the content. We would like to implement addurl which would not even talk to the remote server but just "mint" corresponding path to the key under .git/annex/objects and registerurl to it. But I failed to find a convenience annex command which would allow us to get the full key (accounting for E the way git annex does), and full path to it (thus adding DIRHASH which special remotes could "query", and which is appropriate given the repository settings).
I made subject prefixed with calckey since it seems like the closest hit but it does need an actual file.
edit1: I was pointed to forum/Is_it_possible_adding_files_to_git_annex_bypassing40some_parts_of41the_git_annex_tools/#comment-a80b2371012dda93ad7fb4926af91b48 which seems to be the closest hit, but relies on assuming how extension is extracted by annex, which might be incorrect for some files.
Any extension you pick to use would work fine; it's not a value that needs to be calculated in a long-term reproducible way, and git-annex has refined the extensions it picks repeatedly.
I'm wary about exposing a way for git-annex to calculate the extension. It risks a slippery slope to exposing other internal details of how other backends (eg URL) construct parts of their keys. Especially with the external backends existing now.
This does generate the path to the annex object:
Except, hmm, when the repo is tuned with annex.tune.objecthash1=true it does not take that into account. And if the repo is tuned with annex.tune.objecthashlower=true that needs to be
${hashdirlower}instead.I think it would be a good idea to add something to examinekey that simplifies this, eg
${objectpath}that would exand to the full path. Taking the tuning edge cases into account and also future-proofing the interface.Another approach would be to generate a SHA256 key, make the annex link, and then use
git-annex migrate --backend=SHA256Eto convert the annex link.And hmm, that does point toward a way to use the existing backend API cleanly. Why bother with writing out the symlinks only to have migrate convert them? The fastMigrate API has well-defined inputs, so we can just expose it.
Eg:
--batch would need to input the filename as well as the key when used with --migrate-to-backend.
Woohoo -- thank you Joey!
Here is my complete example which seems to work quite neatly. One aspect which you might (or not -- shouldn't matter to me but bash people might appreciate): could "${filename}" (if provided to examinekey) be passed into
formatas well? that would allow for easy looping like in this example, but on more than one key at a time.Unrelated, just curious -- what is the best way to "chain" multiple batch processes in bash, like here
examinekeyandregisterurl?