Currently, the git-annex branch is not checked out, but is accessed as needed with commands like git-cat. Could git-annex work faster if it kept the git-annex branch checked out? Especially if one could designate a fast location (like a ramdisk) for keeping the checked-out copy. Maybe git-worktree could be used to tie the separate checkout to the repository.
Interesting idea, though a ramdisk seems like overkill; filesystem caching tends to be quite good.
It would only speed up branch reads, not writes.
I don't think it would be much faster to read individual files from the filesystem than reading the data directly out of the git repository. git pack files are designed for fast random access. On top of that, the git cat-file --batch interface adds extra context switches and buffer copies and parsing overhead; something like libgit2 could potentially be quite a bit faster.
Note that git-annex repository v0 did keep the git-annex data in the regular git tree. My notes during the conversion from that to the branch include: "After all that, git-annex turned out to be nearly as fast as before when it was simply reading files from the filesystem, and actually faster in some cases." Although I didn't record more detailed benchmark information. Would be interesting to dig up git-annex 0.20110610 and benchmark it against the current version.