Recent comments posted to this site:

crypton 2.0.0 known issue on Intel CPUs (fixed in crypton 2.0.1, but Homebrew Formula blocks 2.0.1)

Apologies for the many comments to get here, but I suspect I've found the cause at last, in the Crypton 2.0.1 release notes:

fix(cpu): stop reading Intel's SDBG bit as AMD's XOP, which crashed SHA-512 and ChaCha20 on Broadwell and later.

which links to this bug report on Crypton 2.0.0 with similar crashes, that starts with:

Following the 2.0.0 bump I get SIGILL on an elder machine (i7-6700K / Skylake).

Where on macOS, signal 4 is ILL:

ewen@basadi:~$ kill -l 4
ILL
ewen@basadi:~$ 

so the symptoms match.

And the affected macOS Intel system (Intel Mac Mini, from 2020) I suspect is roughly the same CPU generation to be affected.

So in theory rather than:

ewen@basadi:~$ git annex version | grep crypton
dependency versions: aws-0.25.3 bloomfilter-2.0.1.3 crypton-2.0.0 DAV-1.3.4 feed-1.3.2.1 ghc-9.14.1 http-client-0.7.19 torrent-10000.1.3 uuid-1.3.16.1 yesod-1.6.2.3
ewen@basadi:~$ 

I actually need the build to be against crypton-2.0.1 on macOS Intel.... but the Homebrew Formula explicitly blocks that for another reason:

      # https://github.com/psibi/crypton-conduit/issues/5
      "--constraint=crypton<2.0.1",

where https://github.com/psibi/crypton-conduit/issues/5 was one of the issues I noted above as a new change in the Homebrew Formula, which suggests that git-annex won't build (on macOS) with crypton 2.0.1.

However I think they're confused about what changed in crypton 2.0.1 (which is basically a one line fix) and are probably thinking of crypton 2.1.0 (which is a much larger change), and I've said so in the crypton-conduit issue report, where they noted the conflict.

After forcing Homebrew locally to build with:

cabal v2-install --allow-newer=base,template-haskell --constraint=crypton<2.1.0 --constraint=magic<2 --constraint=QuickCheck<2.17 --constraint=persistent-sqlite +systemlib +use-pkgconfig

which pulls in crypton 2.0.1:

ewen@basadi:~$ git annex version | egrep 'crypton|git-annex version'
git-annex version: 10.20261006
dependency versions: aws-0.25.3 bloomfilter-2.0.1.3 crypton-2.0.1 DAV-1.3.4 feed-1.3.2.1 ghc-9.14.1 http-client-0.7.19 torrent-10000.1.3 uuid-1.3.16.1 yesod-1.6.2.3
ewen@basadi:~$ 

I find that git annex addurl works again:

ewen@basadi:/tmp/test-annex$ git annex addurl --pathdepth -1 'https://download.2600.com/mediadownload/www.2600.com/offthewall/mp3files/2026/off_the_wall__20261006-128.mp3'
addurl https://download.2600.com/mediadownload/www.2600.com/offthewall/mp3files/2026/off_the_wall__20261006-128.mp3 
(to off_the_wall__20261006-128.mp3) ok
(recording state in git...)
ewen@basadi:/tmp/test-annex$

and I assume that git annex importfeed will probably work again too.

Ewen

PS: The working Homebrew bottle install on macOS ARM also has crypton-2.0.0 as the build dependency, but since it seems to be an Intel CPU specific bug in Crypton 2.0.0, that wouldn't matter on ARM.

Comment by ewen —
git-annex depends on ram now (10.20261006)

Ah, on closer inspection of the changes it looks like git-annex itself now depends on ram (from stack.yaml):

[...]
resolver: lts-24.52
extra-deps:
- aws-0.25.2
- file-io-0.2.0
- blake3-0.3
- magic-1.1
- ram-0.22.0

which is presumably at least part of the reason why HomeBrew went with the alternative approach of depending on an unmerged PR of yesod-static.

Unfortunately the Homebrew git-annex Formula PR (for 10.20261005) -- https://github.com/Homebrew/homebrew-core/pull/315642/files -- doesn't give any commentary on why the changes were made, and the working source branch is naturally deleted.

But building git-annex 10.20261005 / 10.20261006 with the old ram < 0 build constraint from the Homebrew Formula seems unlikely to work.

Ewen

Comment by ewen —
Homebrew Formula adopts yesod-static PR 1916 AFAICT

Looking closer, as best I can tell Homebrew have effectively proactively adopted the PR with their recent Formula change (above), since the URL:

https://github.com/yesodweb/yesod/archive/23f8d636842023c7cde36109ea24258df0d5ecd6.tar.gz

seems to correspond to this commit:

https://github.com/leftaroundabout/yesod/commit/23f8d636842023c7cde36109ea24258df0d5ecd6

at the tip of the source branch of the PR (unmerged yesod PR #1916). Even though the download is via the upstream gitrepo.

For the record, I'd report this to Homebrew but they (a) officially don't support macOS Intel any longer ("Tier 3" as of last month), and (b) officially don't support local builds).

It is however seeming like I should possibly try building 10.20261006 without this pinning to a PR version of yesod-static, as the more I look at that change in the Homebrew Formula, the more surprising it seems.

Ewen

Comment by ewen —
yesod-static context changes

From the yesod-static issue (linked above):

I think the problem is that crypton-1.1.4 now uses ram instead of memory, which is fundamentally incompatible with the current version of yesod-static.

it seems like all of those yesod-static / ram / crypton changes are related, but that the problems people encountered were build issues rather than a runtime issue. (And the unmerged yesod-static PR #1916, seems to selectively use ram or memory depending on the crypton version; notably it is unmerged for 4+ months.)

But it does at least seem notable that Homebrew have changed how they're addressing those build conflicts.

Ewen

Comment by ewen —
Homebrew git-annex Formula

For ease of reference this is the Homebrew Formula for building git-annex:

https://github.com/Homebrew/homebrew-core/blob/main/Formula/g/git-annex.rb

and the only commit I see since 10.20060901 with non-trivial changes (eg, more than just source URL/sha256, bottle sha256) is:

https://github.com/Homebrew/homebrew-core/commit/b39e1cef1d46210c79163e52fbb2335864c88d37

which looks like it's trying to add two constraints on:

  • yesod-static (specifically 1.6.1.4)
  • crypton (specifically <2.0.1)

and remove one other constraint.

  • ram (<0)

Those changes reference:

(with the ram constraint apparently being removed in favour of a different constraint).

Are those changes around yesod-static or crypton likely to be a cause of issues with URL downloads?

Ewen

Comment by ewen —
comment 8

If the server does have git-annex installed, the git post-receive hook usually runs git-annex, which handles updating the git-annex branch from synced/git-annex as a side effect of what it's really itended to do.

I guess that is not happening in the forgejo repository for whatever reason.

Anyway, I reproduced this behavior with a ssh remote that has git-annex installed, but no git hooks. And made git-annex sync do the force push with lease.

Comment by joey —
comment 3

Thank you for confirming the LLM use and for not doing it again.

And yeah, putting time into filing a good bug report can pay off. But if you have limited time and file a not so good bug report, that's fine. I can often read between the lines, and just seeing what you did and what git-annex output is often enough. And if it isn't, someone else can come along and improve the bug report. So I would encourage you to do what you can and not feel you have to reach for AI to fill in the rest.

Comment by joey —
ai
sorry about using Claude to assist in this and missing your feedback on that, next time I'll use my own wording. I've encountered a few more bugs but currently lack the time to file them without ai assistance, but I also won't waste your time triaging ai slop
Comment by zommuter —
comment 8

It's fine to force-push your git-annex branch to the server in this situation.

Comment by joey —
comment 5

Your log shows that you did not have refs/remotes/hub-con/git-annex at the point where git-annex sync would have merged it:

may be because it was not yet created by forgejo+aneksjo or I have not fetched yet... oh, well

If your local git-annex branch contains a git-annex forget transition, git-annex necessarily cannot merge unrelated remote git-annex branches into it, because they may contain exactly the historical information that was requested to be forgotten. Instead, the new information is extracted from the remote git-annex branch and put into the local git-annex branch.

oh, what how that magical addition has happened! I still have transition information

❯ git ls-tree git-annex | grep transitions
100644 blob 891a754697d1602294fe8619878aeb9bb22b1586    transitions.log
❯ git show git-annex:transitions.log
ForgetGitHistory 1789701492s
ForgetDeadRemotes 1789701492s

but when should it get gone -- any step I could do to fascilitate besides just going and killing that file? Or it to remain there forever to be sure not pick up from "old history"? (here new forgejo though was newer AFAIK)

Comment by yarikoptic —