Please describe the problem.

On the current macOS HomeBrew build of the current git-annex (10.20240227), it appears that the build dependencies have dragged in the latest Haskell tls package. Which now defaults supportedExtendedMainSecret to RequireEMS (previously it seems to have been AllowEMS; see eg darcs bug report of similar error).

The result of this is that some podcast feeds, from webservers which do not support EMS, fail with an error, eg:

importfeed https://risky.biz/feeds/risky-business
  download failed: HandshakeFailed (Error_Protocol "peer does not support Extended Main Secret" HandshakeFailure)

  warning: downloading the feed failed (feed: https://risky.biz/feeds/risky-business)
ok

(And presumably this will also affect some non-podcast HTTPS downloads; I found it in a podcast download context.)

I believe this "Extended Main Secret" is also known as "Extended Master Secret", aka RFC 7627, which was written up in 2015. So I can understand why ~9 years later the Haskell tls library is defaulting to insisting on EMS in a new major version. Unfortunately not all webservers, especially podcast feed webservers, have caught up with this.

As best I can tell git annex is getting this tls dependency via http-client which uses http-client-tls, and http-client-tls appears to just have a tls (>=1.2) dependency, which is presumably how tls-2.0.0 / tls-2.0.1 got dragged in, with these new defaults.

I'm unclear if git-annex is in a position to pass AllowEMS to the TLS library (and thus restore to the old default). But at least in the short term it might be worth considering doing that if possible.

What steps will reproduce the problem?

Currently I have three podcast feeds (two from the same webserver) which fail:

git annex importfeed https://risky.biz/feeds/risky-business

git annex importfeed https://risky.biz/feeds/risky-business-news

git annex importfeed https://www.thecultureoftech.com/index.php/feed/podcast/ 

(Given the irony that the first two are are an InfoSec podcast, I have also reported this missing EMS extension support to them as well, so it may get fixed before you try it.)

It looks like I've also had one media file download fail repeatedly for the same reason (but the podcast feed itself downloads okay):

git annex addurl https://traffic.omny.fm/d/clips/53b6fe2a-4ef6-4356-ae92-a61500df6da0/40b3f537-c161-4823-ae44-af3a007e121b/b2682900-b36c-447b-812d-b1290049fea8/audio.mp3

What version of git-annex are you using? On what operating system?

git annex 10.20240227, on macOS Ventura (13.6.3). With git annex installed from HomeBrew.

ewen@basadi:~$ git annex version
git-annex version: 10.20240227
build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.24.1 bloomfilter-2.0.1.2 crypton-0.34 DAV-1.3.4 feed-1.3.2.1 ghc-9.6.3 http-client-0.7.16 persistent-sqlite-2.13.3.0 torrent-10000.1.3 uuid-1.3.15 yesod-1.6.2.1
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: darwin x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
ewen@basadi:~$ 

Please provide any additional information below.

ewen@basadi:~/Music/podcasts$ git annex importfeed https://www.thecultureoftech.com/index.php/feed/podcast/ 
importfeed gathering known urls ok
importfeed https://www.thecultureoftech.com/index.php/feed/podcast/ 
  download failed: HandshakeFailed (Error_Protocol "peer does not support Extended Main Secret" HandshakeFailure)

  warning: downloading the feed failed (feed: https://www.thecultureoftech.com/index.php/feed/podcast/)
ok
ewen@basadi:~/Music/podcasts$ 
ewen@basadi:~/Music/podcasts$ git annex addurl https://traffic.omny.fm/d/clips/53b6fe2a-4ef6-4356-ae92-a61500df6da0/40b3f537-c161-4823-ae44-af3a007e121b/b2682900-b36c-447b-812d-b1290049fea8/audio.mp3
addurl https://traffic.omny.fm/d/clips/53b6fe2a-4ef6-4356-ae92-a61500df6da0/40b3f537-c161-4823-ae44-af3a007e121b/b2682900-b36c-447b-812d-b1290049fea8/audio.mp3 
git-annex: HttpExceptionRequest Request {
  host                 = "traffic.omny.fm"
  port                 = 443
  secure               = True
  requestHeaders       = [("Accept-Encoding",""),("User-Agent","git-annex/10.20240227")]
  path                 = "/d/clips/53b6fe2a-4ef6-4356-ae92-a61500df6da0/40b3f537-c161-4823-ae44-af3a007e121b/b2682900-b36c-447b-812d-b1290049fea8/audio.mp3"
  queryString          = ""
  method               = "HEAD"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
  proxySecureMode      = ProxySecureWithConnect
}
 (InternalException (HandshakeFailed (Error_Protocol "peer does not support Extended Main Secret" HandshakeFailure)))
failed
addurl: 1 failed
ewen@basadi:~/Music/podcasts$ 

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Absolutely, I've been using git-annex as my podcatcher (among other reasons) for about a decade at this point. Thanks for developing it!