Please describe the problem.
I have git-credential-oauth configured to ease http authentication against Forgejo instances:
[credential]
helper = cache --timeout 21600
helper = oauth
When I am using git annex push to push to a non-existing repository on a Forgejo-aneksajo instance it doesn't utilize that credential helper though, and instead asks for username and password (see log below). The same also happens for git annex sync. Once oauth authorization has happened and an access token is cached (i.e. after the git push in the log) git-annex does use it properly.
What steps will reproduce the problem?
See log below, combined with the git-credential-oauth configuration from above.
What version of git-annex are you using? On what operating system?
git-annex version: 10.20250828-gfe7ecf505146342fe8df2430a0bcaf5f02d89a80
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Servant Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.24.1 bloomfilter-2.0.1.2 crypton-0.34 DAV-1.3.4 feed-1.3.2.1 ghc-9.6.6 http-client-0.7.17 persistent-sqlite-2.13.3.0 torrent-10000.1.3 uuid-1.3.15 yesod-1.6.2.1
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL GITBUNDLE GITMANIFEST VURL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg rclone hook external compute mask
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
Please provide any additional information below.
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
$ datalad create test1
create(ok): /home/icg149/Playground/test1 (dataset)
$ cd test1
$ git remote add origin https://atris.fz-juelich.de/m.risse/test1.git
$ git annex push origin
Username for 'https://atris.fz-juelich.de': ^C
[ble: exit 130]
$ git push origin main
Please complete authentication in your browser...
https://atris.fz-juelich.de/login/oauth/authorize?client_id=a4792ccc-144e-407e-86c9-5e7d8d9c3269&code_challenge=uEYAd0rzQY4JG0yOkDNMUNEBHqIQInrvdMqOIFL3AWI&code_challenge_method=S256&redirect_uri=http%3A%2F%2F127.0.0.1%3A42305&response_type=code&state=9UFx41eIbP3Qn9PWh_5eGaE3UDWMNdWBKE6_nwIo_DM
Enumerating objects: 6, done.
Counting objects: 100% (6/6), done.
Delta compression using up to 8 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 521 bytes | 521.00 KiB/s, done.
Total 6 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
To https://atris.fz-juelich.de/m.risse/test1.git
* [new branch] main -> main
$ git annex push origin
push origin
Everything up-to-date
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), 426 bytes | 426.00 KiB/s, done.
Total 5 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
remote:
remote: Create a new pull request for 'synced/main':
remote: https://atris.fz-juelich.de/m.risse/test1/compare/main...synced/main
remote:
remote:
remote: Create a new pull request for 'synced/git-annex':
remote: https://atris.fz-juelich.de/m.risse/test1/compare/main...synced/git-annex
remote:
To https://atris.fz-juelich.de/m.risse/test1.git
* [new branch] main -> synced/main
* [new branch] git-annex -> synced/git-annex
ok
$
# End of transcript or log.
git-annex is actually using git credential here. That's where the "Username for" prompt comes from.
Looks like the url you gave 404's. But git-annex is hitting
https://atris.fz-juelich.de/m.risse/test1.git/configand getting a 401 Unauthorized for that. Which is why it is using git credential. But I'm not sure if I'm seeing now the same now as what you would have seen.@matrs Any chance you could give me access to reproduce this using your server so I could look into that?
If the server sent back 404 for the /config hit, then the early UUID discovery would not prompt with git credential.
Then, to make "push to create" work smoothly,
git-annex push, after pushing the git branches, could regenerate the remote list. So if the branch push created the git repo, any annex uuid that the new repo has would be discovered at that point.The remote list regeneration would only need to be done when there are git remotes that don't have a UUID yet.
The assistant would also need to be made to do that.
Looks like the 401 Unauthorized happens for all non-existent repos when accessing
/config.Eg:
A bug in Forgejo?
The chicken-and-egg problem you are describing is actually something msz has already encountered and reported, but that issue is fixed: Forgejo-aneksajo also creates the repository for requests to /config, and will git-annex-init it if the request comes from a git-annex user agent and the user has write permissions. More about that here:
So that's not it... I've investigated a bit and I think I led you astray with the comment about a "non-existing repository". I am also seeing the issue with a pre-created repository, and even with a pre-created and git-annex-init'ialized repository.
The issue is actually that for ATRIS I rely on git-credential-oauth's "Gitea-like-Server" discovery here: https://github.com/hickford/git-credential-oauth/blob/f01271d94c70b9280c19f489f90c05e9aba0d757/main.go#L206
When doing a
git push origin mainthe git-credential-oauth helper actually receives this request:while with
git annex pushit is just this:Git-credential-oauth recognizes that it is talking to a Gitea/Forgejo server based on this
wwwauth[]=Basic realm="Gitea"data. Without it and in the absence of a more specific configuration for the server it doesn't try to handle it and falls back to the standard http credential handling of git. I am not sure where these capability and wwwauth fields are coming from, but I think git-annex should somehow do the same as git here...I've gotten at the data git sends to the credential helper with this trivial script:
and configuring it as my credential helper.
I have to say, I like this pattern of processes communicating over simple line-based protocols
git pushseems to first make a GET request for something like/m.risse/test-push-oauth2.git/info/refs?service=git-receive-pack, which responds with a 401 andwww-authenticate: Basic realm="Gitea"among the headers. Git then seems to pass this information on to the git-credential-helper.git annex pushlikewise receives a 401 response from the/configendpoint with the same www-authenticate header, so it could pass it on to the credential helper too.I am not sure where the
capabilitys are coming from...Hmm, then
git-annex pullwill create a repository. Which is going further than "push to create".I do think my idea in comment #2 would be better than how you implemented that. But it's also not directly relevant to this bug report.
I did open support push to create.
The www-authenticate header is also sent when the request for
/configis a 401. So git-annex can use that to set the wwwauth field.The capability fields are indicating capabilities of git. I checked and git-credential-oauth does not rely on those capabilities.
(Wildly, git-credential-oauth is looking for "GitLab", "GitHub", and "Gitea" in order to sniff what backend it's authenticating to, and that's all it uses the wwwauth for.)