Gitolite is a git repository
manager. Here's how to add git-annex support to gitolite, so you can
git annex copy
files to a gitolite repository, and git annex get
files from it.
A nice feature of using gitolite with git-annex is that users can be given
read-only access to a repository, and this allows them to git annex get
file contents, but not change anything.
First, you need new enough versions:
- the current
master
branch of gitolite works with git-annex (tested 2014-04-19), but v3.5.3 and earlier v3.x require use of thegit-annex
branch. - gitolite 2.2 also works -- this version contains a git-annex-shell ADC and supports "ua" ADCs.
- git-annex 3.20111016 or newer needs to be installed on the gitolite server. Don't install an older version, it wouldn't be secure!
Instructions for gitolite master
branch
To setup gitolite to work with git-annex, you can follow the instructions on the gitolite website,
and just add 'git-annex-shell ua',
to the ENABLE list in ~/.gitolite.rc
.
Here are more detailed instructions:
1: Create a git
user
sudo adduser \ --system \ --shell /bin/bash \ --gecos 'git version control' \ --group \ --disabled-password \ --home /home/git git
2: Copy a public SSH key for the user you want to be the gitolite administrator.
In the instructions below, I placed the key in a file named /home/git/me.pub
.
3: Clone and install gitolite
First switch to the git
user (e.g. sudo su - git
) and then run:
cd git clone https://github.com/sitaramc/gitolite.git mkdir -p bin ./gitolite/install -ln
4: Add ~/bin
to PATH
Make sure that ~/bin
is in the PATH
, since that's where gitolite installed its binary. Do something like this:
echo 'export PATH=/home/git/bin:$PATH' >> .profile export PATH=/home/git/bin:$PATH
5: Configure gitolite
Edit ~/.gitolite.rc
to enable the git-annex-shell command.
Find the ENABLE
list and add this line in there somewhere:
'git-annex-shell ua',
Now run gitolite's setup:
gitolite setup -pk me.pub rm me.pub
Instructions for gitolite 2.2
And here's how to set it up. The examples are for gitolite as installed on Debian with apt-get, but the changes described can be made to any gitolite installation, just with different paths.
Set $GL_ADC_PATH
in .gitolite.rc
, if you have not already done so.
echo '$GL_ADC_PATH = "/usr/local/lib/gitolite/adc/";' >>~gitolite/.gitolite.rc
Make the ADC directory, and a "ua" subdirectory.
mkdir -p /usr/local/lib/gitolite/adc/ua
Install the git-annex-shell ADC into the "ua" subdirectory from the gitolite repository.
cd /usr/local/lib/gitolite/adc/ua/ cp gitolite/contrib/adc/git-annex-shell .
Now all gitolite repositories can be used with git-annex just as any ssh remote normally would be used. For example:
# git clone gitolite@localhost:testing Cloning into testing... Receiving objects: 100% (18/18), done. # cd testing # git annex init init ok # cp /etc/passwd my-cool-big-file # git annex add my-cool-big-file add my-cool-big-file ok (Recording state in git...) # git commit -m added [master d36c8b4] added 1 files changed, 1 insertions(+), 0 deletions(-) create mode 120000 my-cool-big-file # git push --all Counting objects: 17, done. Delta compression using up to 2 threads. Compressing objects: 100% (12/12), done. Writing objects: 100% (14/14), 1.39 KiB, done. Total 14 (delta 0), reused 1 (delta 0) To gitolite@localhost:testing c552a38..db4653e git-annex -> git-annex 29cd204..d36c8b4 master -> master # git annex copy --to origin copy my-cool-big-file (checking origin...) (to origin...) WORM-s2502-m1318875140--my-cool-big-file 2502 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/1) sent 2606 bytes received 31 bytes 1758.00 bytes/sec total size is 2502 speedup is 0.95 ok
Troubleshooting
I got an error like this when setting up gitolite after setting up a local git repo and git annex:
git-annex-shell: First run: git-annex init Command ssh ["git@git.example.com","git-annex-shell 'configlist' '/~/myrepo.git'"] failed; exit code 1
because I forgot to "git push --all" after adding the new gitolite remote.
Looks like you are missing a closing double quote on the line:
echo '$GL_ADC_PATH = "/usr/local/lib/gitolite/adc/;' >>~gitolite/.gitolite.rc
right after /;
I got this working by the way - great stuff.
Well a repo url like
gitolite@localhost:testing
puts it in the gitolite user's /~/testingThis worked when I added the gitolite stuff, anyway.. Let's see if it still does:
Still seems right, the ADC's regexp will match this the git-annex shell command.
I guess there is some path rewriting going in in gitolite proper because if try a url of the form ssh://git@localhost/testing, then it still works with gitolite, but fails with the ADC because the repo is passed as /testing:
What I have to ask Sitaram and or find in the docs is if this is a bug or a feature in gitolite. I can see how the leading slash would get swallowed up by this line
in gl-auth-command, but I guess that isn't the whole story.
I confirmed with Sitaram that this is intentional, if probably under-documented. Since the ADC strips the leading /~/ in assigning $start anyway, I guess something like the following will work
That patch seems ok, it doesn't seem to allow through any repo locations that were blocked before.
So, it has my blessing.. but the ADC is in gitolite and will need to be patched there.
After some debugging printing, here is my current understanding.
urls of the form git@host:~repo or ssh://git@host
urls of the form git@host:/repo or ssh://git@host/repo
urls of the form git@host:repo
So the weird case is the last one where git and git-annex are sending different things over the wire. I don't know if you have other motivations for doing the url normalization on the client side, but it isn't needed for gitolite, and in some sense complicates things a little. On the other hand, now that I see what is going on, it isn't a big deal to just strip the leading /~ off in the adc. It does lead to the odd situation of some URLs working for git-annex but not git.
/~/
comes from. I don't anticipate this changing on the git-annex side.Hi,
I noticed using the git-annex branch of gitolite v3 that the same URL with ".git" at the end would not work in git-annex. For example my test repository was
git@git2.mildred.fr:u/mildred/Annex.git
but it didn't work until I converted it togit@git2.mildred.fr:u/mildred/Annex
On the server, the repository is in
repositories/u/mildred/Annex.git
If I try a copy with git-annex for example, I would get:
(test is the name of my remote and titi is my file)
Note, in my gitolite conf, I have:
The steps to activate git-annex integration have changed/simplified for v3.
1) during install, be sure to use the 'git-annex' branch, rather than master[fn:1].
2) to enable git-annex-shell, open ~/.gitolite.rc and insert 'git-annex-shell' => 'ua' into the hash list in the COMMANDS array.[fn:2]
'git-annex-shell' => 'ua',
[fn:1] We'd like to have this feature-branch merged to master, so please send Sitaram feedback, positive and negative, based on your experiences. [fn:2] There is no GL_ADC_PATH and no "ua" subdirectory here, and nothing to "install"; the command now comes with gitolite.
@khaije
Could you paste your config file? Here is mine: http://paste.debian.net/44856/ I don't have any COMMANDS array. Could you elaborate your modifications please?
Thanks.
@François
The proper array in .gitolite.rc seems to be the "ENABLE" array, which it appears is parsed into the COMMANDS array in src/lib/Gitolite/Rc.pm
@Khaije
Using 'git-annex-shell' => 'ua' doesn't seem to work for me. The program still fails in src/gitolite-shell around line 163 (gitolite repo version b1d3c05):
When I insert $rc{COMMANDS}{ words[0] } into the _die message, it shows up as "1" instead of "ua" as I was expecting.
When I manually set $rc{COMMANDS}{ words[0] } to 'ua' slightly earlier in the script, the git-annex-shell command gets run but it seems to fail to parse the result of the configlist command properly because then I get
I am actually about to give up on the notion of using git-annex and gitolite together. Maybe. I am interested to know if anyone else is having similar problems.
We have reached the same point as the previous poster from 25 days ago. $ git annex copy --to origin FATAL: suspicious characters loitering about 'git-annex-shell 'configlist' '/~/testing''
Remote origin does not have git-annex installed; setting remote.origin.annex-ignore git-annex: cannot determine uuid for origin
Anyone actually have this working?
my gitolite.rc is available at https://gist.github.com/khaije1/7609848
For whatever reason I've found this to be very simple to get working so I'd guess there's a missing ingredient somewhere. The combination of gitolite and git-annex is valuable to me so I'll add documents to the url above in hopes it will assist some people with getting the same value.
Adding:
'git-annex-shell' =>1,
To the .gitolite.rc file resulted in the "FATAL: suspicous characters loitering about 'git-annex-shell 'configlist' '/~/testing''...
Gitolite source code (https://github.com/sitaramc/gitolite/commit/b1d3c0571409b7c6279fc6a77253c3bc262ab425#diff-79a3701e9e2cee0ea1316451c21a3fec) requires this entry:
'git-annex-shell ua'
The latest commit in the gitolite repository "git-annex support, finally in master!" looks really promissing. I'm currently using ubuntu trusty with updated gitolite3 package and the configuration provided by Khaije. One line needs to be changed: 'git-annex-shell' => 'ua', instead of 'git-annex-shell' =>1,
However, one little detail is still open for me. I need to call the server side "git annex init" on the server (sudo su gitolite3; cd ~/repositories/testing.git && git annex init). I cannot find a way to initialize the server from client side. E.g. git annex init && git push --all is not enough. The man page describes git annex initremote for other server types but not for gitolite remotes. Wouldn't we need something similar for gitolite as well? Or is there a better solution which I do not recognize?
@Adrian, that's good news. It would be helpful if someone could update the top of this page to document how to use git-annex with the new version of gitolite. This is a wiki..
You're not supposed to need to use initremote when dealing with normal git remotes. It seems that something got lost that automatically initialize the remote repository in this situation. I've fixed it so that it will be set up when
git-annex-shell configlist
is run.(@wayne, this fixes the problem you reported too..)
master
branch of gitolite today on my server, and it works fine! Updated the wiki with the commands I used to setup gitolite.Hi!
I've made a fresh install of Gitolite 3.6.1 and got this error when trying to sync :
FATAL: bad git-annex-shell command: git-annex-shell 'configlist' '/nicolas.git' at /home/git/bin/commands/git-annex-shell line 25, line 1.
I find out that I need to add the following lines to the gitolite.rc in the server side.
'git-annex-shell ua'
The signal repository works with gitolite as expected. However, the mirroring feature is not working for slaves. When I do
git annex copy --to origin
The master server store the annex file correctly. The file managed by annex is not syncing to the slave mirrors at all.