Hi,
I've been experimenting with combining ikiwiki with git-annex and it seems to work. Thought I'd post my process. I've commented on the ikiwiki website as well but perhaps it'd be of interest to git-annex folks.
I have very little understanding of any of the tools involved and have just attempted to make it work using my limited knowledge. I don't use the web interface for ikiwiki which simplifies things.
The website in question just went online and is currently an archive of architectural photographs and the site relies heavily on the ikiwiki osm and album plugins.
Setting things up
To start with I set up the wiki on the server and git clone to into $wrkdir on my laptop. I then initialize a git-annex repo in the $srcdir on the server. Leaving the $gitdir untouched. The $scrdir git-annex repo has to be in direct mode. Before doing any syncing I add annex-ignore = true and annex-sync = false to .git/config in the origin repo ($gitdir): this is to prevent polluting $gitdir with git-annex data. The same process is repeated in the $wrkdir on the laptop.
Pushing and syncing
With this setup I can then git add remote $srcdir, git add $file and git-push mdwn files and other lightweight data from the laptop. While git annex-add, git-annex sync and git-annex copy --to $srcdir jpg's and other heavy files. All pure git commands work as expected with ikiwiki and the website recompiles etc.
Snags
I'm frequently left with (non-dangling) symlinks in the $srcdir despite the annex repo being in direct mode. When this happens git-annex fsck sorts things out.
Uploading image files does require a bit of manual work. But as this is done less frequently it's not much of an issue for me. I'm guessing that by doing things it the correct order (whatever that might be) I could avoid some of the manual work.
The thing to keep in mind is to never git-add the typechanged annexed files in the $srcdir. In general I never use git commands in the $srcdir.
The main problem is the symlinks though as they demand a manual git-annex fsck. I have no idea what causes the symlinks in a direct mode repo.
Any comments?
I understand you do not use the web interface - but what if you did? would it commit all those files into git?
Could we add the git-annex files to a .gitignore file?
I have very poor understanding of what ikiwiki actually does behind the scenes including how it uses the $srcdir and $gitdir.
The only way I could see the web interface working would be to use the git-annex content expressions and having the assistant running on the server. That still doesn't prevent large files from being checked into git though? That all depends on which order ikiwiki can be made to do things.
I might be able to test on a local wiki but that would have to wait a while.
Interesting experiment.
I don't know why you don't want to let git-annex sync its data to $gitdir.
The symlinks could be occuring because of a bug in direct mode. (I have fixed many past bugs that caused that.) But just as likely it's because ikiwiki will run
git pullin the srcdir.I think it would make more sense to use the underlay plugin and keep your annexed repository in a separate underlay. This would guarantee ikiwiki doesn't run git commands in there, and would ensure that nothing done with the web interface could mess with the annex.
@Joey
Well neither do I!
It seemed to be the way to avoid duplicating data while still having the images picked up by the ikiwiki album plugin. Wouldn't the files in the $gitdir end up duplicated in $srcdir?
When you mention it I've had similar problems with my vfat usb annex repos. Using the post-receive merge hook to make files visible for non git-annex devices about town. Nothing I can reliably recreate but I will keep my eye out for bugs.
Yep that would be ideal. For my usecase the album plugin is vital and I can't understand how album would pick up and deal with images in an underlay dir. This is a bit OT for this site though most of my questions are ikiwiki related.
Turns out using the underlay was a piece of cake! Just mirror the folder structure of the repo in your underlaydir put your jpegs there and off you go. Images are picked up by the album plugin and it all just works. No need to coax git-annex into doing odd stuff.
Thought I'd post this for other ikiwikers.
So how does this actually work in practice? Is the underlay directory a completely disconnected repository?
Can we have step by step instructions on how to set this up?
@anarcat
I described the ikiwiki album setup on the ikiwiki website as there are no git-annex tricks anymore. Just standard behaviour.
Yes they are completely separate repos. Really the underlay dir could be managed in any way you like svn, rsync, ftp etc. I didn't expect the album plugin to pick everything up but apparently it does! Perhaps underlays are completely integrated and appear as 'normal' to all of ikiwiki?
Yes, I believe that is the way the underlay works - the files are just picked up.
As for the ikiwiki forum post - maybe you should move it to the "tips" section?
Thanks for the documentation!!