tips/using gitolite with git-annexgit-annexhttp://git-annex.branchable.com/tips/using_gitolite_with_git-annex/git-annexikiwiki2018-04-27T21:06:42Zcomment 1http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_1_9a2a2a8eac9af97e0c984ad105763a73/openid [www.openid.albertlash.com]2013-11-27T22:47:37Z2011-12-24T06:08:45Z
<p>Looks like you are missing a closing double quote on the line:</p>
<p>echo '$GL_ADC_PATH = "/usr/local/lib/gitolite/adc/;' >>~gitolite/.gitolite.rc</p>
<p>right after /;</p>
<p>I got this working by the way - great stuff.</p>
comment 2http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_2_d8efea4ab9576555fadbb47666ecefa9/joey2013-11-27T22:47:37Z2011-12-24T16:54:31Z
I've fixed the typo (anyone can edit pages in this wiki FWIW.)
repo name conventions?http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_3_807035f38509ccb9f93f1929ecd37417/bremner2013-11-27T22:47:37Z2011-12-30T21:41:13Z
I'm confused by the fact that the git-annex-shell adc rejects any repo names that don't start with /~/ since none of my repos start that way. It seems work ok if I just delete /\~ from the front of the regex, but I feel like I must be missing something.
comment 4http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_4_eb81f824aadc97f098379c5f7e4fba4c/joey2013-11-27T22:47:37Z2011-12-31T00:29:45Z
<p>Well a repo url like <code>gitolite@localhost:testing</code> puts it in the gitolite user's /~/testing</p>
<p>This worked when I added the gitolite stuff, anyway.. Let's see if it still does:</p>
<pre>
joey@gnu:~/tmp>mkdir g
joey@gnu:~/tmp>cd g
joey@gnu:~/tmp/g>git init
Initialized empty Git repository in /home/joey/tmp/g/.git/
joey@gnu:~/tmp/g>git annex init
init ok
joey@gnu:~/tmp/g>git remote add test 'gitolite@localhost:testing'
joey@gnu:~/tmp/g>touch foo
joey@gnu:~/tmp/g>git annex add foo
add foo (checksum...) ok
(Recording state in git...)
joey@gnu:~/tmp/g>git annex copy foo --to test --debug
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","ls-files","--cached","-z","--","foo"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","check-attr","annex.numcopies","-z","--stdin"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","show-ref","--hash","refs/heads/git-annex"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","show-ref","git-annex"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","cat-file","--batch"]
Running: ssh ["-4","gitolite@localhost","git-annex-shell 'configlist' '/~/testing'"]
</pre>
<p>Still seems right, the ADC's regexp will match this the git-annex shell command.</p>
gitolite gets different paths for different urlshttp://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_5_f688309532d2993630e9e72e87fb9c46/bremner2013-11-27T22:47:37Z2011-12-31T01:50:49Z
<p>I guess there is some path rewriting going in in gitolite proper because if try a url of the form
ssh://git@localhost/testing, then it still works with gitolite, but fails with the ADC because
the repo is passed as /testing:</p>
<pre>
Running: ssh ["git@host","git-annex-shell 'configlist' '/recommend'"]
Running: ssh ["git@host","git-annex-shell 'configlist' '/recommend'"]
</pre>
<p>What I have to ask Sitaram and or find in the docs is if this is a bug or a feature in gitolite. I can see how the leading slash would get swallowed up by this line</p>
<pre>
$repo = "'$REPO_BASE/$repo.git'"
</pre>
<p>in gl-auth-command, but I guess that isn't the whole story.</p>
ssh://gitolite-host/repo-name is supposed to workhttp://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_6_3e203e010a4df5bf03899f867718adc5/bremner2013-11-27T22:47:37Z2011-12-31T03:34:17Z
<p>I confirmed with Sitaram that this is intentional, if probably under-documented.
Since the ADC strips the leading /~/ in assigning $start anyway, I guess something like the following will work</p>
<pre>
diff --git a/contrib/adc/git-annex-shell b/contrib/adc/git-annex-shell
index 7f9f5b8..523dfed 100755
--- a/contrib/adc/git-annex-shell
+++ b/contrib/adc/git-annex-shell
@@ -28,7 +28,7 @@ my $cmd=$ENV{SSH_ORIGINAL_COMMAND};
# the second parameter.
# Further parameters are not validated here (see below).
die "bad git-annex-shell command: $cmd"
- unless $cmd =~ m#^(git-annex-shell '\w+' ')/\~/([0-9a-zA-Z][0-9a-zA-Z._\@/+-
+ unless $cmd =~ m#^(git-annex-shell '\w+' ')/(?:\~\/)?([0-9a-zA-Z][0-9a-zA-Z.
my $start = $1;
my $repo = $2;
my $end = $3;
</pre>
comment 7http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_7_f8fd08b6ab47378ad88c87348057220d/joey2013-11-27T22:47:37Z2011-12-31T18:32:28Z
<p>That patch seems ok, it doesn't seem to allow through any repo locations that were blocked before.</p>
<p>So, it has my blessing.. but the ADC is in gitolite and will need to be patched there.</p>
afaict git annex normalizes urls on the client side.http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_8_8249772c142117f88e37975d058aa936/bremner2013-11-27T22:47:37Z2011-12-31T22:29:38Z
<p>After some debugging printing, here is my current understanding.</p>
<ul>
<li><p>urls of the form git@host:~repo or ssh://git@host</p>
<ul>
<li>git sends commands like "git-receive-pack '~/repo'</li>
<li>gitolite converts these to $REPO_BASE/~/repo which fails. ~/repo would also fail fwiw.</li>
<li>git-annex sends seems /~/repo, which works</li>
</ul>
</li>
<li><p>urls of the form git@host:/repo or ssh://git@host/repo</p>
<ul>
<li>git sends "git-receive-pack '/db/cs3383'"</li>
<li>gitolite converts this to $REPO_BASE/repo which works</li>
<li>git annex sends "git-annex-shell 'inannex' '/repo' ..." which works, but only with the patch above.</li>
</ul>
</li>
<li><p>urls of the form git@host:repo</p>
<ul>
<li>git sends "git-receive-pack 'repo'</li>
<li>gitolite converts this to $REPO_BASE/repo, which works</li>
<li>git-annex sends "git-annex-shell 'inannex' '/~/db/cs3383'...", which also works for git-annex-shell.</li>
</ul>
</li>
</ul>
<p>So the weird case is the last one where git and git-annex are sending different things over the wire.
I don't know if you have other motivations for doing the url normalization on the client side, but it isn't needed for gitolite, and in some sense complicates things a little. On the other hand, now that I see what is going on, it isn't a big deal to just strip the leading /~ off in the adc. It does lead to the odd situation of some URLs working for git-annex but not git.</p>
comment 9http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_9_28418635a6ed7231b89e02211cd3c236/joey2013-11-27T22:47:37Z2012-01-02T16:27:55Z
Ah right. git-annex normalizes all git ssh style user@host:dir to valid uris, which is where the <code>/~/</code> comes from. I don't anticipate this changing on the git-annex side.
git-annex no longer supported by gitolite g3http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_10_8767bc8014b459a3cd76f275fd4fa8d6/npouillard2013-11-27T22:47:37Z2013-03-25T12:47:21Z
See http://gitolite.com/gitolite/dev-status.html for some details.
Problems with URL ending with ".git"http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_11_00715e0b47f09130e0e536e29f7b9258/mildred2013-11-27T22:47:37Z2013-05-24T12:15:16Z
<p>Hi,</p>
<p>I noticed using the git-annex branch of gitolite v3 that the same URL with ".git" at the end would not work in git-annex. For example my test repository was <code>git@git2.mildred.fr:u/mildred/Annex.git</code> but it didn't work until I converted it to <code>git@git2.mildred.fr:u/mildred/Annex</code></p>
<p>On the server, the repository is in <code>repositories/u/mildred/Annex.git</code></p>
<p>If I try a copy with git-annex for example, I would get:</p>
<pre><code>$ git annex copy titi --to test
copy titi (checking test...) FATAL: u/mildred/Annex.git mildred DENIED
(unable to check test) failed
git-annex: copy: 1 failed
</code></pre>
<p>(test is the name of my remote and titi is my file)</p>
<p>Note, in my gitolite conf, I have:</p>
<pre><code>repo u/CREATOR/[a-zA-Z0-9].*
C = @all
RW+D = CREATOR
RW = WRITERS
R = READERS
</code></pre>
comment 12http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_12_7027ce60265b8f24c8ab54553e544068/Kavin2013-11-27T22:47:37Z2013-07-25T03:20:15Z
latest code of gitolite does not support git-annex ? I could not find a way to make it work ?
git-annex with gitolite FTWhttp://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_13_75218b7409c0e281cb01c9b2791e8cdf/Khaije2013-11-27T22:47:37Z2013-08-13T15:13:07Z
<p>The steps to activate git-annex integration have changed/simplified for v3.</p>
<p>1) during install, be sure to use the 'git-annex' branch, rather than master[fn:1].</p>
<p>2) to enable git-annex-shell, open ~/.gitolite.rc and insert 'git-annex-shell' => 'ua' into the hash list in the COMMANDS array.[fn:2]</p>
<h1>'git-annex-shell' => 'ua',</h1>
<p>[fn:1] We'd like to have this feature-branch merged to master, so please send Sitaram feedback, positive and negative, based on your experiences.
[fn:2] There is no GL_ADC_PATH and no "ua" subdirectory here, and nothing to "install"; the command now comes with gitolite.</p>
comment 14http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_14_7d4d4515218d1259d32be3baeb5ee56e/François2013-11-27T22:47:37Z2013-09-22T18:30:45Z
<p>@khaije</p>
<p>Could you paste your config file? Here is mine: http://paste.debian.net/44856/
I don't have any COMMANDS array. Could you elaborate your modifications please?</p>
<p>Thanks.</p>
git-annex with gitolite-rchttp://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_15_dc6f21b5a3d5931c8d949a9753411b9e/wayne2013-11-27T22:47:37Z2013-10-19T17:03:32Z
<p>@François</p>
<p>The proper array in .gitolite.rc seems to be the "ENABLE" array, which it appears is parsed into the COMMANDS array in src/lib/Gitolite/Rc.pm</p>
<p>@Khaije</p>
<p>Using 'git-annex-shell' => 'ua' doesn't seem to work for me. The program still fails in src/gitolite-shell around line 163 (gitolite repo version b1d3c05):</p>
<pre><code>_die "suspicious characters loitering about '$soc'"
if $rc{COMMANDS}{ words[0] } ne 'ua' and $soc !~ $REMOTE_COMMAND_PATT;
</code></pre>
<p>When I insert $rc{COMMANDS}{ words[0] } into the _die message, it shows up as "1" instead of "ua" as I was expecting.</p>
<p>When I manually set $rc{COMMANDS}{ words[0] } to 'ua' slightly earlier in the script, the git-annex-shell command gets run but it seems to fail to parse the result of the configlist command properly because then I get</p>
<pre><code>Failed to get annex.uuid configuration of repository origin
Instead, got: "annex.uuid=\ncore.gcrypt=\n"
</code></pre>
<p>I am actually about to give up on the notion of using git-annex and gitolite together. Maybe. I am interested to know if anyone else is having similar problems.</p>
Any further info regarding gitolite support?http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_16_8e5039e6655fc80dc863b6cdf44ef02a/Douglas2013-11-27T22:47:37Z2013-11-14T03:20:31Z
<p>We have reached the same point as the previous poster from 25 days ago.
$ git annex copy --to origin
FATAL: suspicious characters loitering about 'git-annex-shell 'configlist' '/~/testing''</p>
<p> Remote origin does not have git-annex installed; setting remote.origin.annex-ignore
git-annex: cannot determine uuid for origin</p>
<p>Anyone actually have this working?</p>
comment 17http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_17_9c40e1da8bb44f7207e802377f5cf923/Khaije2013-11-27T22:47:37Z2013-11-23T02:14:12Z
<p>my gitolite.rc is available at https://gist.github.com/khaije1/7609848</p>
<p>For whatever reason I've found this to be very simple to get working so I'd guess there's a missing ingredient somewhere. The combination of gitolite and git-annex is valuable to me so I'll add documents to the url above in hopes it will assist some people with getting the same value.</p>
Using Gitolite 3.2http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_18_f1a180d5cf65b7a870a13ddb4f76d00d/Laura2014-01-17T20:14:16Z2014-01-17T20:14:16Z
<p>Adding:</p>
<p> 'git-annex-shell' =>1,</p>
<p>To the .gitolite.rc file resulted in the "FATAL: suspicous characters loitering about 'git-annex-shell 'configlist' '/~/testing''...</p>
<p>Gitolite source code (https://github.com/sitaramc/gitolite/commit/b1d3c0571409b7c6279fc6a77253c3bc262ab425#diff-79a3701e9e2cee0ea1316451c21a3fec) requires this entry:</p>
<p>'git-annex-shell ua'</p>
How can the git annex init command be called on the server?http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_19_33c19097b6f2b48dfe09ec4c8d952d06/Adrian2014-03-25T06:58:56Z2014-03-25T06:58:56Z
<p>The latest commit in the gitolite repository "git-annex support, finally in master!" looks really promissing. I'm currently using ubuntu trusty with updated gitolite3 package and the configuration provided by Khaije. One line needs to be changed: 'git-annex-shell' => 'ua', instead of 'git-annex-shell' =>1,</p>
<p>However, one little detail is still open for me. I need to call the server side "git annex init" on the server (sudo su gitolite3; cd ~/repositories/testing.git && git annex init). I cannot find a way to initialize the server from client side. E.g. git annex init && git push --all is not enough. The man page describes git annex initremote for other server types but not for gitolite remotes. Wouldn't we need something similar for gitolite as well? Or is there a better solution which I do not recognize?</p>
comment 20http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_20_c82af00db3dd74ee9bfe12668e76e57b/joeyh.name2014-03-26T18:24:30Z2014-03-26T18:24:30Z
<p>@Adrian, that's good news. It would be helpful if someone could update the top of this page to document how to use git-annex with the new version of gitolite.
This is a wiki..</p>
<p>You're not supposed to need to use initremote when dealing with normal git remotes. It seems that something got lost that automatically initialize the remote repository in this situation. I've fixed it so that it will be set up when <code>git-annex-shell configlist</code> is run.</p>
<p>(@wayne, this fixes the problem you reported too..)</p>
Worked for me, updated wikihttp://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_21_d2feaaf22d41413048dabf706d1b267e/Ellis2014-04-19T09:41:11Z2014-04-19T09:41:11Z
I tried the <code>master</code> branch of gitolite today on my server, and it works fine! Updated the wiki with the commands I used to setup gitolite.
Using Gitolite 3.6.1http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_22_90ead0e1fe8615fe11f6f1bd7dcd7e84/Nicolas2014-09-11T00:11:32Z2014-09-11T00:11:32Z
<p>Hi!</p>
<p>I've made a fresh install of Gitolite 3.6.1 and got this error when trying to sync :</p>
<p>FATAL: bad git-annex-shell command: git-annex-shell 'configlist' '/nicolas.git' at /home/git/bin/commands/git-annex-shell line 25, line 1.</p>
Problem solved!http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_23_70dcb7e7ffdd14351adaf4c40ee7fdd0/risca2015-03-17T09:37:04Z2015-03-17T09:37:04Z
<div class="highlight-hs"><pre class="hl">I was experiencing the same issue <span class="hl opt">(</span>FATAL<span class="hl opt">:</span> bad git<span class="hl opt">-</span>annex<span class="hl opt">-</span>shell command<span class="hl opt">:</span> git<span class="hl opt">-</span>annex<span class="hl opt">-</span>shell <span class="hl opt">'</span>configlist<span class="hl opt">' '/</span>myrepo<span class="hl opt">'</span> at <span class="hl opt">/</span>usr<span class="hl opt">/</span>share<span class="hl opt">/</span>gitolite3<span class="hl opt">/</span>commands<span class="hl opt">/</span>git<span class="hl opt">-</span>annex<span class="hl opt">-</span>shell line <span class="hl num">25</span><span class="hl opt">, <</span>DATA<span class="hl opt">></span> line <span class="hl num">1</span><span class="hl opt">.)</span>
I solved with the following patch<span class="hl opt">:</span>
<span class="hl slc">--- /usr/share/gitolite3/commands/git-annex-shell 2015-03-17 10:26:46.114680811 +0100</span>
<span class="hl opt">+++</span> orig<span class="hl opt">.</span>git<span class="hl opt">-</span>annex<span class="hl opt">-</span>shell <span class="hl num">2015</span><span class="hl opt">-</span><span class="hl num">03</span><span class="hl opt">-</span><span class="hl num">17 10</span><span class="hl opt">:</span><span class="hl num">25</span><span class="hl opt">:</span><span class="hl num">16.074680925</span> <span class="hl opt">+</span><span class="hl num">0100</span>
<span class="hl opt">@@ -</span><span class="hl num">23</span><span class="hl opt">,</span><span class="hl num">7</span> <span class="hl opt">+</span><span class="hl num">23</span><span class="hl opt">,</span><span class="hl num">7</span> <span class="hl opt">@@</span> my <span class="hl opt">$</span>cmd <span class="hl opt">= $</span>ENV<span class="hl opt">{</span>SSH_ORIGINAL_COMMAND<span class="hl opt">};</span>
<span class="hl opt">#</span> the second parameter<span class="hl opt">.</span>
<span class="hl opt">#</span> Further parameters are <span class="hl kwc">not</span> validated here <span class="hl opt">(</span>see below<span class="hl opt">).</span>
die <span class="hl str">"bad git-annex-shell command: $cmd"</span>
<span class="hl opt">-</span> <span class="hl kwc">unless</span> <span class="hl opt">$</span>cmd <span class="hl opt">=</span>~ m<span class="hl opt">#</span>^<span class="hl opt">(</span>git<span class="hl opt">-</span>annex<span class="hl opt">-</span>shell <span class="hl opt">'\</span>w<span class="hl opt">+' ')/(</span>?<span class="hl opt">:\</span>~<span class="hl opt">\/)</span>?<span class="hl opt">([</span><span class="hl num">0</span><span class="hl opt">-</span><span class="hl num">9</span>a<span class="hl opt">-</span>zA<span class="hl opt">-</span>Z<span class="hl opt">][</span><span class="hl num">0</span><span class="hl opt">-</span><span class="hl num">9</span>a<span class="hl opt">-</span>zA<span class="hl opt">-</span>Z<span class="hl opt">.</span>_<span class="hl opt">\@/+-]*)('( .*</span>|<span class="hl opt">))$#;</span>
<span class="hl opt">+</span> <span class="hl kwc">unless</span> <span class="hl opt">$</span>cmd <span class="hl opt">=</span>~ m<span class="hl opt">#</span>^<span class="hl opt">(</span>git<span class="hl opt">-</span>annex<span class="hl opt">-</span>shell <span class="hl opt">'\</span>w<span class="hl opt">+' ')/\</span>~<span class="hl opt">/([</span><span class="hl num">0</span><span class="hl opt">-</span><span class="hl num">9</span>a<span class="hl opt">-</span>zA<span class="hl opt">-</span>Z<span class="hl opt">][</span><span class="hl num">0</span><span class="hl opt">-</span><span class="hl num">9</span>a<span class="hl opt">-</span>zA<span class="hl opt">-</span>Z<span class="hl opt">.</span>_<span class="hl opt">\@/+-]*)('( .*</span>|<span class="hl opt">))$#;</span>
my <span class="hl opt">$</span>start <span class="hl opt">= $</span><span class="hl num">1</span><span class="hl opt">;</span>
my <span class="hl opt">$</span>repo <span class="hl opt">= $</span><span class="hl num">2</span><span class="hl opt">;</span>
my <span class="hl opt">$</span>end <span class="hl opt">= $</span><span class="hl num">3</span><span class="hl opt">;</span>
References<span class="hl opt">:</span> http<span class="hl opt">://</span>git<span class="hl opt">-</span>annex<span class="hl opt">.</span>branchable<span class="hl opt">.</span>com<span class="hl opt">/</span>tips<span class="hl opt">/</span>using_gitolite_with_git<span class="hl opt">-</span>annex<span class="hl opt">/#</span>comment<span class="hl opt">-</span><span class="hl num">16</span>ee81f0d9e973ecb2fbb9d4a1fff261
It was shown <span class="hl num">3</span> years ago<span class="hl opt">!</span> I don<span class="hl opt">'</span>t know why the bug it<span class="hl opt">'</span>s still on<span class="hl opt">.</span> I<span class="hl opt">'</span>m using debian with<span class="hl opt">:</span>
<span class="hl opt">-</span> git<span class="hl opt">-</span>annex<span class="hl opt">:</span>
Installed<span class="hl opt">:</span> <span class="hl num">5.20141024</span>~bpo70<span class="hl opt">+</span><span class="hl num">1</span>
<span class="hl opt">-</span> gitolite3<span class="hl opt">:</span>
Installed<span class="hl opt">:</span> <span class="hl num">3.6</span><span class="hl opt">-</span><span class="hl num">1</span>~bpo70<span class="hl opt">+</span><span class="hl num">1</span>
</pre></div>
Using gitolite 3.6.6 mirror not working with annexhttp://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_24_a62f59295b3912dc38def5e493bdb55a/git-annex2016-09-28T18:12:56Z2016-09-28T18:12:56Z
<p>I find out that I need to add the following lines to the gitolite.rc
in the server side.</p>
<p>'git-annex-shell ua'</p>
<p>The signal repository works with gitolite as expected.
However, the mirroring feature is not working for slaves.
When I do</p>
<p>git annex copy --to origin</p>
<p>The master server store the annex file correctly.
The file managed by annex is not syncing to the slave
mirrors at all.</p>
Where does gitolite stores the annex objects contents?http://git-annex.branchable.com/tips/using_gitolite_with_git-annex/comment_25_08b3b257b3dbd32cb807e29f512ea9c9/davicastro2018-04-27T21:06:42Z2018-04-27T21:06:42Z
Hi. Does anyone knows where does gitolite stores the annex objects' contents? (I got it to work, and git annex is working fine, but I want to "see the annex files" in the central gitolite server. And it puzzles me I can't find them! It definitively is not in the repo.git bare repository under the gitolite admin repositores folder)