Recent comments posted to this site:
- Remove comment
I've written up a draft interface for programs used by a compute special remote: compute special remote interface
It could be argued that git-annex should recurse into submodules. Oddly, I don't remember that anyone has ever tried to make that argument. If they did it was a long time ago. It may be that datalad has relieved enough of the pressure in that area that it's not bothering many people.
Anyway, I wouldn't want to tie compute special remotes to changing git-annex in that way, but I also wouldn't want to rule out adding useful stuff to git-annex just because it breaches the submodule boundary in a way that's new to git-annex.
Thinking about a command like this:
git-annex addcomputed foo --to ffmpeg-cut \
--input source=submodule/input.mov \
--value starttime=15:00 \
--value endtime=30:00
That would need to look inside the submodule to find the input key.
When getting the key later, it can't rely on the tree still containing the
same submodules at the same locations. git mv submodule foo
would break
the computation.
I think that can be dealt with by having it fall back to checking location logs of all submodules, to find the submodule that knows about a key.
Deleting a submodule would still break the computation, and that seems difficult to avoid. Seems acceptable.
@m.risse earlier you said that it would be bad to
Silently use the old version of "data.grib", creating a mismatch between "data.nc" and "data.grib"
That's what I was getting at when I said:
But if you already have data.nc file present in a repository, it does not get updated immediately when you update the source "data.grib" file.
So just using files from HEAD for the computation is not sufficient to avoid this kind of mismatch. The user will need some workflow to deal with it.
Eg, they could recompute data.nc whenever data.grib is updated, and so make a commit that updates both files together. But if they're doing that, why does the computation need to use files from HEAD? Recomputing data.nc could just as well pin the new key of data.grib.
Please take a look at iroh. It started as an IPFS implementation in rust, realized that IPFS is slow and overengineered and now pivoted to providing p2p connections with quic.
- Peers/nodes/endpoints use ed25519 keys as identities.
- The iroh project hosts relay servers for initial NAT hole punching and as connection fallbacks.
- So far there are 4 initial discovery implementations: DNS, Local (mDNS), Pkarr or Bittorrents Mainline DHT
I'm waiting for their FOSDEM talk. But there is also a good presentation on YT: A tour of iroh.
Nothing. --fast happens to be parsed as a global option so it's
accepted with every command, but it does not change the usual behavior of
git-annex get
at all.
Commands like git-annex copy
that implement a different behavior for --fast have it documented
in their individual man pages.