DISCUSS: use clang 20

[h-vetinari]

Clang has a 6 month release cadence, and the last few years we've updated to an even release in the fall (#6571, #4991, #3261, #2010).

In particular, development on LLVM 19 has stopped completely. The same is true for LLVM 20 actually, but at least it contains 6 months more of features & fixes. Upstream is currently doing point-releases for 21.x, which will stop with the release of v22 early next year. We can then do a bump to GCC 15 & Clang 21 in the spring again (after the release of GCC 16 resp. the corresponding 15.x point-release).

However, due to two ABI breaks in LLVM v20.1, we might want to skip this release

Also, one of the two ABI breaks might need a more comprehensive rebuild anyway (hopefully not "full rebuild..."). This PR is for discussion purposes, and remains a draft until these points are resolved one way or another.

To-Do list based on docs

• Update all respective feedstocks (listing only those for activation; they wouldn't pass if the implementation wasn't complete)
  • LLVM v20.1.8 clang-compiler-activation-feedstock#165 (osx)
  • add clang v20.1 ctng-compiler-activation-feedstock#148 (linux; not part of global pinning)
  • add clang v20.1 clang-win-activation-feedstock#51 (win; not part of the global pinning)
  • build for clang v20.1 r_clang_activation-feedstock#9 (clang for r-feedstocks; not part of global pinning)
• Tested that compilers work
  • WIP: Test new compilers scipy-feedstock#250
• Checked release notes for ABI-relevant changes
  • Clang potentially breaking changes
    • Minor changes for overflow handling
  • Clang & libcxx ABI changes

    • [likely problematic] [ABI] mangleCXXCtorVTable mangles an incorrect name llvm/llvm-project#108015:

      Fixed the Itanium mangling of the construction vtable name. This change will introduce incompatibilities with code compiled by Clang 19 and earlier versions, unless the -fclang-abi-compat=19 option is used.

      I don't know what a "construction vtable" is, but vtables are very common, so we might have to do a bigger rebuild, or permanently add -fclang-abi-compat=19 to the compiler flags, which also isn't a very appealing idea.

    • [potentially problematic] [libc++] Vendor communication: upcoming ABI break in various container types llvm/llvm-project#154985:
      I don't remember seeing any issues related to this when we released libcxx v20.1; OTOH, we split the libcxx headers from the library about a year ago, and clangxx now always brings a matching libcxx-devel. This could mean that using libcxx 20 headers at compile-time and libcxx 21 at runtime would surface the ABI break (and inversely, that we've never encountered the ABI break because the libcxx 20 headers were never in use at scale).

      In the referenced issue, the LLVM maintainers were discussing the possibility of an LLVM v20.2 release, i.e. one including a backport of llvm/llvm-project@f5e687d, but that seems to have fizzled out and realistically won't happen.

    • Other minor changes of the kind we had ignored in 19, 18, 17, 16, 15 & 14

Previously: #7421 #6571 #4890 #4991 #4215 #3731 #3290 #3261 #2802 #2010

CC @conda-forge/core

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

[h-vetinari]

Any thoughts on this @isuruf @xhochy @mgorny?

[mgorny]

I don't know what a "construction vtable" is, but vtables are very common, so we might have to do a bigger rebuild, or permanently add -fclang-abi-compat=19 to the compiler flags, which also isn't a very appealing idea.

Me neither, but I've just tested the included example, and confirmed that the Clang-19 mangling was different than GCC, so another point worth considering is that the new behavior aligns it with GCC.

---

Are you aware if more ABI breaks are coming in 21? Perhaps it would then make sense to skip it indeed and do a single rebuild for both. Watching from the sidelines, I didn't notice any "killer features" in Clang 20, but I wasn't watching very carefully.

[h-vetinari]

Are you aware if more ABI breaks are coming in 21?

Clang 21 fixes the libcxx ABI break at least. I guess we'll have to deal with the vtable one either way.

The only other ABI change mentioned in the v21 release notes is: llvm/llvm-project#120670, which shouldn't affect us AFAICT, since (outside of a handful microarch-enabled feedstocks), we're generally not using AVX.

[mgorny]

Yeah. Technically we could also backport the fix to 20.x, but then we'd be incompatible with vanilla libc++.

[h-vetinari]

but then we'd be incompatible with vanilla libc++.

OTOH, we'd be compatible with libcxx 19 and 21 again... 🤔

But clearly not appealing to preempt upstream on this, especially if they would have bumped the minor version for it. That's why I thought we might want to skip this release...

[h-vetinari]

@isuruf, any thoughts about the "construction vtable" issue here? If it's not a break-the-world event, then I think we can just do a regular update to clang 21 together with GCC 15 next spring.

[isuruf]

Let's wait for clang 21. I don't think this is going to be a big issue

[h-vetinari]

Let's wait for clang 21. I don't think this is going to be a big issue

Thanks for your opinion. I agree.