initremote type=git fresh "regression"

Our test started to fail on 20220610 daily tests, with smth like

2022-06-10T05:51:16.5860150Z >               **results,
2022-06-10T05:51:16.5860330Z             )
2022-06-10T05:51:16.5861310Z E           datalad.runner.exception.CommandError: CommandError: 'git -c diff.ignoreSubmodules=none annex initremote fresh-sr type=git location=http://127.0.0.1:50821/ autoenable=true -c annex.dotfiles=true' failed with exitcode 1 under /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/datalad_temp_test_as_common_datasourceerpclil1 [out: 'initremote fresh-sr 
2022-06-10T05:51:16.5861910Z E           
2022-06-10T05:51:16.5862300Z E           failed'] [err: 'http://127.0.0.1:50821//config download failed: File not found
2022-06-10T05:51:16.5862730Z E           git-annex: git repository does not have an annex uuid
2022-06-10T05:51:16.5863040Z E           initremote: 1 failed']
2022-06-10T05:51:16.5863180Z 
2022-06-10T05:51:16.5863540Z ../../../../hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/datalad/runner/runner.py:205: CommandError

most likely due to 10.20220525-64-g14584e7a3 "initremote type=git probe uuid" .

yet to unravel details/change in behavior, but since changelog entry says only about "Improve handling of type=git special remotes" I would not expect negative change in the behavior. Joey -- anything immediate comes to mind?

notabug --Joey

RSS Atom

comment 1

the "heart" of the issue is that in initremote we provide URL which is without /.git suffix. That used to work just fine and our test testing git annex get on the clone accomplished the mission.

Comment by yarikoptic — Mon Jun 13 23:52:48 2022

Remove comment

comment 2

The change that broke your test uses the exact same url uuid detection code that git-annex usually uses. So if you were using an url without .git before, it seems like it would have failed to detect a uuid after enabling the special remote.

If that reasoning does not hold, I'm going to need a test case to understand why. Unfortunately, your test case is hard for me to understand what it's doing.

I can't even get git to clone a non-bare git repo served over plain http without using .git in the url, so don't understand how your test case is doing that.

When I manually make a git remote with a http url without .git (which git cannot pull from), I see the git-annex uuid detection code failing, as I would expect:

git remote add test http://localhost/~joey/tmp/foo/
git config annex.security.allowed-ip-addresses all
git-annex info

  Remote test not usable by git-annex; setting annex-ignore

  http://localhost/~joey/tmp/foo//config download failed: Not Found

And the initremote behavior is consistent with that:

git-annex initremote bar type=git location=http://localhost/~joey/tmp/foo/
initremote bar
  http://localhost/~joey/tmp/foo//config download failed: Not Found

git-annex: git repository does not have an annex uuid
failed

(It's worth noting that the change was made to handle another datalad use case, ?strong>git is not working for unkn reason. I'd consider that use case unusual, and marginally worth supporting. I could of course revert the change, but then I'd have to close that bug as wontfix.)

Comment by joey — Tue Jun 14 15:51:57 2022

Remove comment

hold it... ;)

Thank you Joey! I will try to gain better insight into our dance in that test of ours. I am also curious how it worked before that it doesn't work now. FWIW with datalad clone we do try URL as is, and then also with /.git to make users' life easier and so they could just e.g. datalad clone https://datasets.datalad.org.

Comment by yarikoptic — Tue Jun 14 18:47:24 2022

Remove comment

mystery resolved

Ok, resolved mystery for myself:

So if you were using an url without .git before, it seems like it would have failed to detect a uuid after enabling the special remote.

yes -- if initremote failed to detect uuid it

didn't exit with non-0 (i.e. initremote call succeeded, our code passed through)
produced a "funky" looking remotes.log entry without UUID:

(git-annex)lena:~/.tmp/datalad_temp_test_as_common_datasourcek5ozqwqd[dl-test-branch]git-annex
$> git show git-annex:remote.log
 autoenable=true location=http://127.0.0.1:41311/ name=fresh-sr type=git timestamp=1656702874.867345813s
0392395d-c1a2-4791-ba6a-e932f1392c34 autoenable=true location=http://127.0.0.1:41311/.git name=mike2 type=git timestamp=165670282

and our test was working before because apparently we had that type=git remote already added with the correct (with .git) url (as you can see above) so as long as initremote didn't crash, we were "happy" a little more info could be found in our datalad issue

Comment by yarikoptic — Fri Jul 1 19:19:30 2022

Remove comment

comment 5

Aha! That funky remote.log was caused by a historical bad type in git-annex: UUID has a NoUUID value, and when it probed for the UUID, it did not check if it got NoUUID, which serializes to "". Really fixing that is hard, I've tried a couple of times.

Anyway, the new behavior is not buggy, so closing.

Comment by joey — Thu Jul 7 16:10:12 2022

Remove comment

Add a comment