Thanks for the link to the macOS standalone build workflow -- as you say it looks like stack-botan.yaml with magicmime enabled, and using Homebrew to supply some of the dependencies. That's a useful reference for the future.

Based on what I found yesterday I'm fairly sure both of:

• Intel CPU (of a relevant vintage)
• built with exactly Crypton 2.0.0 (ie, using stack.yaml, allowing Crypton 2.x.x and a restriction preventing Crypton 2.0.1 / Crypton 2.1.0)

are required to trigger the build that crashes.

Since Crypton 2.0.1 came out fairly soon after Crypton 2.0.0 and has the fix for the Intel CPU feature detection, I suspect that's an uncommon build configuration. Likely only affecting Homebrew macOS Intel ("Tier 3") and Homebrew Linux Intel ("Tier 2" from memory), due to their (apparently mistaken) choice to block Crypton 2.0.1 in the dependency resolver.

So I think this issue could be closed now, and just serve as future reference for other possible crash isues.

I also agree with you it's quite hard to avoid dependencies-of-dependencies (especially recursively or where fairly key dependencies use the library/versions one would prefer to avoid). Hopefully Crypon 2.x.x settles down a bit more now the clear "major version" breaking changes are done, because it looks hard to completely avoid in a network-related project.

Ewen

What won't work for such a directory special remote is cloning it from the url that git-annex outputs when pushing to it. Eg:

joey@darkstar:~/tmp/bb/repo>git push local1 master
Full remote url: annex::23158cb0-7aa5-43ea-9ca3-c8254e9aa1f0?encryption=none&type=directory
Everything up-to-date
joey@darkstar:~/tmp/bb/repo>..
cd -- ..
joey@darkstar:~/tmp/bb>git clone 'annex::23158cb0-7aa5-43ea-9ca3-c8254e9aa1f0?encryption=none&type=directory'
Cloning into '23158cb0-7aa5-43ea-9ca3-c8254e9aa1f0?encryption=none&type=directory'...
git-remote-annex: Specify directory=

fatal: remote helper 'annex' aborted session
- exit 128
joey@darkstar:~/tmp/bb>git clone 'annex::23158cb0-7aa5-43ea-9ca3-c8254e9aa1f0?encryption=none&type=directory&directory=specialdirectoryremote'
Cloning into '23158cb0-7aa5-43ea-9ca3-c8254e9aa1f0?encryption=none&type=directory&directory=specialdirectoryremote'...

Working around that is possible, but you have to add the directory parameter to the url:

joey@darkstar:~/tmp/bb>git clone 'annex::23158cb0-7aa5-43ea-9ca3-c8254e9aa1f0?encryption=none&type=directory&directory=specialdirectoryremote'
Cloning into '23158cb0-7aa5-43ea-9ca3-c8254e9aa1f0?encryption=none&type=directory&directory=specialdirectoryremote'...

This poor fit is why I say that a bare git repository is better.

This is not macOS specific. The directory special remote requires a directory= parameter be given to initremote. That is to support things like a directory special remote that points to a USB drive that moves between several computers. The mount point of the drive can be different on each computer, and so each place the directory special remote is enabled it needs the directory= to be specified.

I don't really see any point in using git-remote-annex with a directory special remote. You can just as well use a bare git repository.

But anyway, if you pass the directory= parameter to enableremote, it does work.

BTW, it's not necessary to run enableremote after initremote. You can just run initremote --with-url.

termux now includes a git-annex package, and that avoids the proot hack that the git-annex.linux standalone build uses to run on android.

So, I think that would be the solution to this problem.

Since using the termux package is now the recommended way to install git-annex on android, and whatever the problem with proot is unlikely to get resolved, I'm going to close this.

It's deeply unfortunate that crypton has stated being developed by pointing an LLM at it and hoping that the (not well understood) results work. The consequences can be seen in the crypton changelog.

git-annex is migrating away from crypton to the extent it can, but unfortunately it's used for https libraries and I have not found a way around that.

PS, the build environment used for the MacOs standalone build is here but is basically just the stack-botan.yaml. That pulls in crypton-1.0.6 currently.

Apologies for the many comments to get here, but I suspect I've found the cause at last, in the Crypton 2.0.1 release notes:

fix(cpu): stop reading Intel's SDBG bit as AMD's XOP, which crashed SHA-512 and ChaCha20 on Broadwell and later.

which links to this bug report on Crypton 2.0.0 with similar crashes, that starts with:

Following the 2.0.0 bump I get SIGILL on an elder machine (i7-6700K / Skylake).

Where on macOS, signal 4 is ILL:

ewen@basadi:~$ kill -l 4
ILL
ewen@basadi:~$ 

so the symptoms match.

And the affected macOS Intel system (Intel Mac Mini, from 2020) I suspect is roughly the same CPU generation to be affected.

So in theory rather than:

ewen@basadi:~$ git annex version | grep crypton
dependency versions: aws-0.25.3 bloomfilter-2.0.1.3 crypton-2.0.0 DAV-1.3.4 feed-1.3.2.1 ghc-9.14.1 http-client-0.7.19 torrent-10000.1.3 uuid-1.3.16.1 yesod-1.6.2.3
ewen@basadi:~$ 

I actually need the build to be against crypton-2.0.1 on macOS Intel.... but the Homebrew Formula explicitly blocks that for another reason:

      # https://github.com/psibi/crypton-conduit/issues/5
      "--constraint=crypton<2.0.1",

where https://github.com/psibi/crypton-conduit/issues/5 was one of the issues I noted above as a new change in the Homebrew Formula, which suggests that git-annex won't build (on macOS) with crypton 2.0.1.

However I think they're confused about what changed in crypton 2.0.1 (which is basically a one line fix) and are probably thinking of crypton 2.1.0 (which is a much larger change), and I've said so in the crypton-conduit issue report, where they noted the conflict.

After forcing Homebrew locally to build with:

cabal v2-install --allow-newer=base,template-haskell --constraint=crypton<2.1.0 --constraint=magic<2 --constraint=QuickCheck<2.17 --constraint=persistent-sqlite +systemlib +use-pkgconfig

which pulls in crypton 2.0.1:

ewen@basadi:~$ git annex version | egrep 'crypton|git-annex version'
git-annex version: 10.20261006
dependency versions: aws-0.25.3 bloomfilter-2.0.1.3 crypton-2.0.1 DAV-1.3.4 feed-1.3.2.1 ghc-9.14.1 http-client-0.7.19 torrent-10000.1.3 uuid-1.3.16.1 yesod-1.6.2.3
ewen@basadi:~$ 

I find that git annex addurl works again:

ewen@basadi:/tmp/test-annex$ git annex addurl --pathdepth -1 'https://download.2600.com/mediadownload/www.2600.com/offthewall/mp3files/2026/off_the_wall__20261006-128.mp3'
addurl https://download.2600.com/mediadownload/www.2600.com/offthewall/mp3files/2026/off_the_wall__20261006-128.mp3 
(to off_the_wall__20261006-128.mp3) ok
(recording state in git...)
ewen@basadi:/tmp/test-annex$

and I assume that git annex importfeed will probably work again too.

Ewen

PS: The working Homebrew bottle install on macOS ARM also has crypton-2.0.0 as the build dependency, but since it seems to be an Intel CPU specific bug in Crypton 2.0.0, that wouldn't matter on ARM.

Ah, on closer inspection of the changes it looks like git-annex itself now depends on ram (from stack.yaml):

[...]
resolver: lts-24.52
extra-deps:
- aws-0.25.2
- file-io-0.2.0
- blake3-0.3
- magic-1.1
- ram-0.22.0

which is presumably at least part of the reason why HomeBrew went with the alternative approach of depending on an unmerged PR of yesod-static.

Unfortunately the Homebrew git-annex Formula PR (for 10.20261005) -- https://github.com/Homebrew/homebrew-core/pull/315642/files -- doesn't give any commentary on why the changes were made, and the working source branch is naturally deleted.

But building git-annex 10.20261005 / 10.20261006 with the old ram < 0 build constraint from the Homebrew Formula seems unlikely to work.

Ewen

Looking closer, as best I can tell Homebrew have effectively proactively adopted the PR with their recent Formula change (above), since the URL:

https://github.com/yesodweb/yesod/archive/23f8d636842023c7cde36109ea24258df0d5ecd6.tar.gz

seems to correspond to this commit:

https://github.com/leftaroundabout/yesod/commit/23f8d636842023c7cde36109ea24258df0d5ecd6

at the tip of the source branch of the PR (unmerged yesod PR #1916). Even though the download is via the upstream gitrepo.

For the record, I'd report this to Homebrew but they (a) officially don't support macOS Intel any longer ("Tier 3" as of last month), and (b) officially don't support local builds).

It is however seeming like I should possibly try building 10.20261006 without this pinning to a PR version of yesod-static, as the more I look at that change in the Homebrew Formula, the more surprising it seems.

Ewen

From the yesod-static issue (linked above):

I think the problem is that crypton-1.1.4 now uses ram instead of memory, which is fundamentally incompatible with the current version of yesod-static.

it seems like all of those yesod-static / ram / crypton changes are related, but that the problems people encountered were build issues rather than a runtime issue. (And the unmerged yesod-static PR #1916, seems to selectively use ram or memory depending on the crypton version; notably it is unmerged for 4+ months.)

But it does at least seem notable that Homebrew have changed how they're addressing those build conflicts.

Ewen