{compiler}[GCCcore/14.3.0] LLVM 21.1.2

[Crivella]

Added EC file for LLVM 21.1.0

Requires:

• Changes to LLVM easyblock required for LLVM 21 easybuild-easyblocks#3902

[github-actions]

Updated software LLVM-21.1.2-GCCcore-14.3.0.eb

Diff against LLVM-20.1.8-GCCcore-14.3.0.eb

easybuild/easyconfigs/l/LLVM/LLVM-20.1.8-GCCcore-14.3.0.eb

diff --git a/easybuild/easyconfigs/l/LLVM/LLVM-20.1.8-GCCcore-14.3.0.eb b/easybuild/easyconfigs/l/LLVM/LLVM-21.1.2-GCCcore-14.3.0.eb
index 0b77ae2681..f3cfca92c9 100644
--- a/easybuild/easyconfigs/l/LLVM/LLVM-20.1.8-GCCcore-14.3.0.eb
+++ b/easybuild/easyconfigs/l/LLVM/LLVM-21.1.2-GCCcore-14.3.0.eb
@@ -1,5 +1,5 @@
 name = 'LLVM'
-version = '20.1.8'
+version = '21.1.2'
 
 
 homepage = "https://llvm.org/"
@@ -16,7 +16,7 @@ to use LLVM as an optimizer and code generator.
 toolchain = {'name': 'GCCcore', 'version': '14.3.0'}
 toolchainopts = {
     # 'cstd': 'gnu++11',
-    'pic': True
+    'pic': True,
 }
 
 source_urls = ['https://github.com/llvm/llvm-project/releases/download/llvmorg-%(version)s/']
@@ -27,14 +27,12 @@ patches = [
     'LLVM-18.1.8_envintest.patch',
     'LLVM-19.1.7_libomptarget_tests.patch',
     'LLVM-19.1.7_clang_rpathwrap_test.patch',
-    'LLVM-20.1.x_improved-CUDA-13-support.patch',
 ]
 checksums = [
-    {'llvm-project-20.1.8.src.tar.xz': '6898f963c8e938981e6c4a302e83ec5beb4630147c7311183cf61069af16333d'},
+    {'llvm-project-21.1.2.src.tar.xz': '1a417d1c8faf8d93e73fec1cbb76d393ed3218974c2283c7bac9672d3d47c54b'},
     {'LLVM-18.1.8_envintest.patch': '8e25dfab8a29a860717b4bd2d8cdd0e795433766d7fffbda32d06a2bde47058d'},
     {'LLVM-19.1.7_libomptarget_tests.patch': '79a67c118d034cfb74e255696369150c73432d2b422f4834efacb26f7904edbf'},
     {'LLVM-19.1.7_clang_rpathwrap_test.patch': '5ee6a87ec8ff1c8b736ffe0513aa2098bd2b83a1ffc647a1ad2cf966f567e8a1'},
-    {'LLVM-20.1.x_improved-CUDA-13-support.patch': '78762218c8f34b25533f909533076f195136232b7fca1672af602182dfcdc5bf'},
 ]
 
 builddependencies = [
@@ -88,6 +86,8 @@ test_suite_ignore_patterns = [
     # Those 2 might fail with "tzdb: the path '/etc/localtime' is not a symlink"
     "std/time/time.zone/time.zone.db/time.zone.db.tzdb",
     "std/time/time.zone/time.zone.db/time.zone.db.access",
+    # Since v21.x several benchmark tests are timing out
+    "llvm-libc++-shared.cfg.in :: benchmarks/",
 ]
 
 moduleclass = 'compiler'

Diff against LLVM-20.1.7-GCCcore-14.2.0.eb

easybuild/easyconfigs/l/LLVM/LLVM-20.1.7-GCCcore-14.2.0.eb

diff --git a/easybuild/easyconfigs/l/LLVM/LLVM-20.1.7-GCCcore-14.2.0.eb b/easybuild/easyconfigs/l/LLVM/LLVM-21.1.2-GCCcore-14.3.0.eb
index 981cc37366..f3cfca92c9 100644
--- a/easybuild/easyconfigs/l/LLVM/LLVM-20.1.7-GCCcore-14.2.0.eb
+++ b/easybuild/easyconfigs/l/LLVM/LLVM-21.1.2-GCCcore-14.3.0.eb
@@ -1,20 +1,22 @@
 name = 'LLVM'
-version = '20.1.7'
+version = '21.1.2'
 
 
 homepage = "https://llvm.org/"
-description = """The LLVM Core libraries provide a modern source- and target-independent
- optimizer, along with code generation support for many popular CPUs
- (as well as some less common ones!) These libraries are built around a well
- specified code representation known as the LLVM intermediate representation
- ("LLVM IR"). The LLVM Core libraries are well documented, and it is
- particularly easy to invent your own language (or port an existing compiler)
- to use LLVM as an optimizer and code generator."""
+description = """
+The LLVM Core libraries provide a modern source- and target-independent
+optimizer, along with code generation support for many popular CPUs
+(as well as some less common ones!) These libraries are built around a well
+specified code representation known as the LLVM intermediate representation
+("LLVM IR"). The LLVM Core libraries are well documented, and it is
+particularly easy to invent your own language (or port an existing compiler)
+to use LLVM as an optimizer and code generator.
+"""
 
-toolchain = {'name': 'GCCcore', 'version': '14.2.0'}
+toolchain = {'name': 'GCCcore', 'version': '14.3.0'}
 toolchainopts = {
     # 'cstd': 'gnu++11',
-    'pic': True
+    'pic': True,
 }
 
 source_urls = ['https://github.com/llvm/llvm-project/releases/download/llvmorg-%(version)s/']
@@ -25,31 +27,30 @@ patches = [
     'LLVM-18.1.8_envintest.patch',
     'LLVM-19.1.7_libomptarget_tests.patch',
     'LLVM-19.1.7_clang_rpathwrap_test.patch',
-    'LLVM-20.1.x_improved-CUDA-13-support.patch',
 ]
 checksums = [
-    {'llvm-project-20.1.7.src.tar.xz': 'cd8fd55d97ad3e360b1d5aaf98388d1f70dfffb7df36beee478be3b839ff9008'},
+    {'llvm-project-21.1.2.src.tar.xz': '1a417d1c8faf8d93e73fec1cbb76d393ed3218974c2283c7bac9672d3d47c54b'},
     {'LLVM-18.1.8_envintest.patch': '8e25dfab8a29a860717b4bd2d8cdd0e795433766d7fffbda32d06a2bde47058d'},
     {'LLVM-19.1.7_libomptarget_tests.patch': '79a67c118d034cfb74e255696369150c73432d2b422f4834efacb26f7904edbf'},
     {'LLVM-19.1.7_clang_rpathwrap_test.patch': '5ee6a87ec8ff1c8b736ffe0513aa2098bd2b83a1ffc647a1ad2cf966f567e8a1'},
-    {'LLVM-20.1.x_improved-CUDA-13-support.patch': '78762218c8f34b25533f909533076f195136232b7fca1672af602182dfcdc5bf'},
 ]
 
 builddependencies = [
-    ('binutils', '2.42'),
-    ('Python', '3.13.1'),
-    ('CMake', '3.31.3'),
+    ('binutils', '2.44'),
+    ('Python', '3.13.5'),
+    ('CMake', '4.0.3'),
     ('psutil', '7.0.0'),  # Needed to enable test timeout in lit
     ('lit', '18.1.8'),
-    ('git', '2.49.0'),
+    ('git', '2.50.1'),
 ]
 
 dependencies = [
-    ('libxml2', '2.13.4'),
+    ('libffi', '3.5.1'),
+    ('libxml2', '2.14.3'),
     ('ncurses', '6.5'),
     ('zlib', '1.3.1'),
-    ('Z3', '4.13.4'),
-    ('zstd', '1.5.6'),
+    ('Z3', '4.15.1'),
+    ('zstd', '1.5.7'),
 ]
 
 build_shared_libs = True
@@ -81,6 +82,12 @@ test_suite_ignore_patterns = [
     "Driver/atomic.f90",
     "Driver/gcc-toolchain-install-dir.f90",
     "api_tests/test_ompd_get_icv_from_scope.c",
+    "std/time/time.syn/formatter",
+    # Those 2 might fail with "tzdb: the path '/etc/localtime' is not a symlink"
+    "std/time/time.zone/time.zone.db/time.zone.db.tzdb",
+    "std/time/time.zone/time.zone.db/time.zone.db.access",
+    # Since v21.x several benchmark tests are timing out
+    "llvm-libc++-shared.cfg.in :: benchmarks/",
 ]
 
 moduleclass = 'compiler'

Diff against LLVM-20.1.5-GCCcore-13.2.0.eb

easybuild/easyconfigs/l/LLVM/LLVM-20.1.5-GCCcore-13.2.0.eb

diff --git a/easybuild/easyconfigs/l/LLVM/LLVM-20.1.5-GCCcore-13.2.0.eb b/easybuild/easyconfigs/l/LLVM/LLVM-21.1.2-GCCcore-14.3.0.eb
index eb265a3129..f3cfca92c9 100644
--- a/easybuild/easyconfigs/l/LLVM/LLVM-20.1.5-GCCcore-13.2.0.eb
+++ b/easybuild/easyconfigs/l/LLVM/LLVM-21.1.2-GCCcore-14.3.0.eb
@@ -1,20 +1,22 @@
 name = 'LLVM'
-version = '20.1.5'
+version = '21.1.2'
 
 
 homepage = "https://llvm.org/"
-description = """The LLVM Core libraries provide a modern source- and target-independent
- optimizer, along with code generation support for many popular CPUs
- (as well as some less common ones!) These libraries are built around a well
- specified code representation known as the LLVM intermediate representation
- ("LLVM IR"). The LLVM Core libraries are well documented, and it is
- particularly easy to invent your own language (or port an existing compiler)
- to use LLVM as an optimizer and code generator."""
-
-toolchain = {'name': 'GCCcore', 'version': '13.2.0'}
+description = """
+The LLVM Core libraries provide a modern source- and target-independent
+optimizer, along with code generation support for many popular CPUs
+(as well as some less common ones!) These libraries are built around a well
+specified code representation known as the LLVM intermediate representation
+("LLVM IR"). The LLVM Core libraries are well documented, and it is
+particularly easy to invent your own language (or port an existing compiler)
+to use LLVM as an optimizer and code generator.
+"""
+
+toolchain = {'name': 'GCCcore', 'version': '14.3.0'}
 toolchainopts = {
     # 'cstd': 'gnu++11',
-    'pic': True
+    'pic': True,
 }
 
 source_urls = ['https://github.com/llvm/llvm-project/releases/download/llvmorg-%(version)s/']
@@ -26,77 +28,29 @@ patches = [
     'LLVM-19.1.7_libomptarget_tests.patch',
     'LLVM-19.1.7_clang_rpathwrap_test.patch',
 ]
-# riscv_debug_complex_1.patch: Flang's LLVM CodeGen backend (in version 20.1.5) does not support returning
-# complex numbers of certain precisions (like complex(16) or complex(32)) on RISC-V target.
-# This is a known limitation in current Flang/LLVM support.
-
-# riscv_execution_engine.patch: Skip the test on RISC-V, which is appropriate since JIT execution via MLIR's
-# ExecutionEngine is not reliably supported on this architecture in this LLVM version.
-
-# riscv_MLIR.patch: Disable problematic MLIR Runner tests on RISC-V.
-# Failures on RISC-V are due to missing soft-float helper symbols (like __addsf3),
-# which is a known issue when running LLVM's JIT on platforms like RISC-V.
-
-# riscv_xray_check.patch: RISC-V has looser execution ordering; relax CHECK-NEXT to CHECK
-
-# riscv_fuzzer.patch: Mark as UNSUPPORTED for RISC-V tests that use libFuzzer + -fsanitize=address,fuzzer
-
-# riscv_ompt.patch: Mark as UNSUPPORTED for RISC-V some ompt tests that are known to be fragile
-# see (https://github.com/llvm/llvm-project/issues/35661#issuecomment-980993177)
-
 checksums = [
-    {'llvm-project-20.1.5.src.tar.xz': 'a069565cd1c6aee48ee0f36de300635b5781f355d7b3c96a28062d50d575fa3e'},
+    {'llvm-project-21.1.2.src.tar.xz': '1a417d1c8faf8d93e73fec1cbb76d393ed3218974c2283c7bac9672d3d47c54b'},
     {'LLVM-18.1.8_envintest.patch': '8e25dfab8a29a860717b4bd2d8cdd0e795433766d7fffbda32d06a2bde47058d'},
     {'LLVM-19.1.7_libomptarget_tests.patch': '79a67c118d034cfb74e255696369150c73432d2b422f4834efacb26f7904edbf'},
     {'LLVM-19.1.7_clang_rpathwrap_test.patch': '5ee6a87ec8ff1c8b736ffe0513aa2098bd2b83a1ffc647a1ad2cf966f567e8a1'},
 ]
 
-if ARCH == 'riscv64':
-    patches += [
-        'LLVM-20.1.5_riscv_optimization_remark.patch',
-        'LLVM-20.1.5_riscv_fast_math.patch',
-        'LLVM-20.1.5_riscv_debug_complex_1.patch',
-        'LLVM-20.1.5_riscv_execution_engine.patch',
-        'LLVM-20.1.5_riscv_MLIR.patch',
-        'LLVM-20.1.5_riscv_xray_check.patch',
-        'LLVM-20.1.5_riscv_fuzzer.patch',
-        'LLVM-20.1.5_riscv_ompt.patch',
-    ]
-    checksums += [
-        {'LLVM-20.1.5_riscv_optimization_remark.patch':
-         'e414a15e0fc48049a6ec75835cd032c8db6fb5c6d2825f0c604969c192aa60e0'},
-        {'LLVM-20.1.5_riscv_fast_math.patch':
-         '8f5383024c089a8220247aef1a021f85ea5c58c93196ebf80d8227f9c4097eb7'},
-        {'LLVM-20.1.5_riscv_debug_complex_1.patch':
-         '42a04437a0b74c8cf8aeef585f138b85416ea55d3ad5464762e766d476010590'},
-        {'LLVM-20.1.5_riscv_execution_engine.patch':
-         '7d795920e98c2c65df30ed01cc7e9251d69ae25e2674c251bc283ed44b53d02f'},
-        {'LLVM-20.1.5_riscv_MLIR.patch':
-         '2348ee5db243f94d7a8877aee1a4332585e3bd58976af76c4bed01972d7793ca'},
-        {'LLVM-20.1.5_riscv_xray_check.patch':
-         '5ca1e5730a3ee7cc1914959d64330f213a58f9818d91dbba346db63c77b47606'},
-        {'LLVM-20.1.5_riscv_fuzzer.patch':
-         '0d28e1acc96ab10daa2f696db01163c81f8a027a6ab8ccd5a770c6b7e91a4c34'},
-        {'LLVM-20.1.5_riscv_ompt.patch':
-         '9ed82093831777cee08629cb68b7e47d2cace3e0b950381b002f9bfd01bae1f6'},
-    ]
-
 builddependencies = [
-    ('binutils', '2.40'),
-    ('Python', '3.11.5'),
-    ('CMake', '3.27.6'),
-    ('psutil', '6.1.0'),  # Needed to enable test timeout in lit
-    ('lit', '18.1.7'),
-    ('git', '2.42.0'),
+    ('binutils', '2.44'),
+    ('Python', '3.13.5'),
+    ('CMake', '4.0.3'),
+    ('psutil', '7.0.0'),  # Needed to enable test timeout in lit
+    ('lit', '18.1.8'),
+    ('git', '2.50.1'),
 ]
 
 dependencies = [
-    ('libffi', '3.4.4'),
-    ('libxml2', '2.11.5'),
-    ('ncurses', '6.4'),
-    ('zlib', '1.2.13'),
-    ('Z3', '4.13.0'),
-    ('zstd', '1.5.5'),
+    ('libffi', '3.5.1'),
+    ('libxml2', '2.14.3'),
+    ('ncurses', '6.5'),
+    ('zlib', '1.3.1'),
+    ('Z3', '4.15.1'),
+    ('zstd', '1.5.7'),
 ]
 
 build_shared_libs = True
@@ -111,13 +65,6 @@ build_bolt = True
 build_openmp = True
 build_openmp_tools = True
 build_openmp_offload = True
-
-# openmp offload is not supported for riscv64 (https://clang.llvm.org/docs/OffloadingDesign.html)
-# There is a Bug in (at least) testing phase of BOLT for RISC-V. See https://github.com/llvm/llvm-project/issues/134245
-if ARCH == 'riscv64':
-    build_openmp_offload = False
-    build_bolt = False
-
 use_polly = True
 
 python_bindings = True
@@ -135,6 +82,12 @@ test_suite_ignore_patterns = [
     "Driver/atomic.f90",
     "Driver/gcc-toolchain-install-dir.f90",
     "api_tests/test_ompd_get_icv_from_scope.c",
+    "std/time/time.syn/formatter",
+    # Those 2 might fail with "tzdb: the path '/etc/localtime' is not a symlink"
+    "std/time/time.zone/time.zone.db/time.zone.db.tzdb",
+    "std/time/time.zone/time.zone.db/time.zone.db.access",
+    # Since v21.x several benchmark tests are timing out
+    "llvm-libc++-shared.cfg.in :: benchmarks/",
 ]
 
 moduleclass = 'compiler'

[Thyre]

Generally, I think having the new LLVM in a toolchain is great.

If we consider creating an LLVM toolchain with that, we need to keep in mind that there are packages using an older LLVM (e.g. Mesa or numba), where some are in GCCcore. I would also really recommend waiting for the .3 / .4 release before defining the toolchain, since the first few minor releases might be a bit more unstable.

[Crivella]

Yeah for now i just want to see what changes we need to get 21 to compile.
For sure it would beneficial to have a TC based on 21 considering the regression observer in GROMACS with LLVM 20 and arm64 + SVE

• https://gitlab.com/gromacs/gromacs/-/issues/5390#note_2702855160
• {numlib,compiler,toolchain,mpi}[lfoss/2023b] Added all ECs required to compile up to the lfoss and lmpflf toolchains #23051 (comment)

But i think it would also be beneficial to have one based on LLVM 20 if we can get away with only one version of LLVM for Mesa/Numba (maybe even Rust?)

Not sure if we would want 2025a to be with LLVM 20 and 2025b with 21 or if we should wait a 2026a for 21

[Thyre]

Not sure if we would want 2025a to be with LLVM 20 and 2025b with 21 or if we should wait a 2026a for 21

I would tend towards using the last patch release for each toolchain. This would mean 19.1.7 for 2025a, 20.1.8 for 2025b, and so on. This would be similar to GCC, where we never go with the latest release, but a more mature version. It also grants us better compatibility with CUDA and such.

[Crivella]

Test report by @Crivella
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3902
FAILED
Build succeeded (with --ignore-test-failure) for 0 out of 1 (1 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.13
See https://gist.github.com/Crivella/4e4d2423d648eaa86847677599a08c0c for a full test report.

[Crivella]

Still observing the same test errors as reported in llvm/llvm-project#151972

Will have to decide if we want to try and increase the timeout threshold for the tests or just ignore everything with

llvm-libc++-shared.cfg.in :: benchmarks/

Will also have to fix the sanity checks

== FAILED: Installation ended unsuccessfully: Sanity check failed: no file found at 'lib/libFortranRuntime.a' in /home/crivella/.local/easybuild/software/LLVM/21.1.0-GCCcore-14.3.0
no file found at 'lib/libFortranCommon.a' in /home/crivella/.local/easybuild/software/LLVM/21.1.0-GCCcore-14.3.0
no file found at 'lib/x86_64-unknown-linux-gnu/libomptarget-nvptx.bc' in /home/crivella/.local/easybuild/software/LLVM/21.1.0-GCCcore-14.3.0
no file found at 'lib/x86_64-unknown-linux-gnu/libomptarget-amdgpu.bc' in /home/crivella/.local/easybuild/software/LLVM/21.1.0-GCCcore-14.3.0 (took 3 hours 18 mins 53 secs)

Apparently they moved again the location of the BC files

crivella@crivella-desktop:~/.local/easybuild/software/LLVM/21.1.0-GCCcore-14.3.0/lib$ find . -name "*.bc"
./nvptx64-nvidia-cuda/libomptarget-nvptx.bc
./amdgcn-amd-amdhsa/libomptarget-amdgpu.bc

[Crivella]

Test report by @Crivella
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3902
FAILED
Build succeeded (with --ignore-test-failure) for 0 out of 1 (1 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.13
See https://gist.github.com/Crivella/8afd044fb4f73861687ae828e2d8a55f for a full test report.

[Thyre]

Looking at my manual Cronjob build on a machine, I see this in the initial CMake log:

-- Google Benchmark version: v0.0.0, normalized to 0.0.0

So we likely need to add a googlebenchmark build dependency, or disable the feature (which may help with your failed tests).

[Crivella]

Umh I've seen I have that also in our 20.x build logs.
In theory that message should only be included if LLVM_INCLUDE_BENCHMARKS is ON.
We set it to OFF in all stages, but apparently it is not being honored when running make check-all to build the test-suite...

So i guess we have always also been running the third-party/benchmarks even if we wanted them disabled...
In 20.1.7 i have 148 :: benchmarks/ tests being ran, while in 21.1.0 there are 318 so i guess they added something that is failing for us

EDIT

On second inspection it seems the benchmark tests are only being ran for the runtimes.
I think we have to pass LLVM_INCLUDE_BENCHMARKS also to the runtime intermediate step, will try this and see what happens

[Crivella]

Test report by @Crivella
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3902
SUCCESS
Build succeeded (with --ignore-test-failure) for 1 out of 1 (1 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.13
See https://gist.github.com/Crivella/87ec614723f2c1a6b46f14343628a054 for a full test report.

[Crivella]

Now we are set with the sanity checks, but even passing LLVM_INCLUDE_BENCHMARKS=OFF to the runtimes build is still not honored when calling make check-all, also added a comment on llvm/llvm-project#151972 (comment).

Gonna keep looking to see if I am missing a way to ensure this parameter is respected also for the tests of the runtimes...
In case we want this to get in faster i would probably add llvm-libc++-shared.cfg.in :: benchmarks/ as an ingnored patters as previously suggested

[Crivella]

Test report by @Crivella
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3902
SUCCESS
Build succeeded (with --ignore-test-failure) for 1 out of 1 (1 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.13
See https://gist.github.com/Crivella/ad085c632081bfb5b06aefe0b2adb0b2 for a full test report.

[Thyre]

Test report by @Thyre
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3902
SUCCESS
Build succeeded (with --ignore-test-failure) for 1 out of 1 (1 easyconfigs in total)
jrc0900.jureca - Linux Rocky Linux 9.6, AArch64, ARM UNKNOWN, 1 x NVIDIA NVIDIA GH200 480GB, 580.65.06, Python 3.9.21
See https://gist.github.com/Thyre/6ac8785547a7b2eaf87187abfd2b93a0 for a full test report.

[Thyre]

29 tests failed (aside from the GPU ones, which we ignore):

Timed Out Tests (15):
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/lower_bound.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/make_heap.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/min.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/push_heap.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/ranges_make_heap.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/ranges_push_heap.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/set_intersection.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/containers/associative/flat_map.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/containers/associative/flat_multimap.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/containers/associative/unordered_set.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/containers/sequence/vector.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/containers/string.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/format/formatter_float.bench.cpp
********************
Failed Tests (388):
  BOLT-Unit :: Profile/./ProfileTests/failed_to_discover_tests_from_gtest
  Clang :: Analysis/z3/crosscheck-statistics.c
  Flang :: Driver/gcc-toolchain-install-dir.f90
  libFuzzer-aarch64-default-Linux :: stack-overflow-with-asan.test
  libFuzzer-aarch64-libcxx-Linux :: stack-overflow-with-asan.test
  libFuzzer-aarch64-static-libcxx-Linux :: stack-overflow-with-asan.test
[... all the GPU ones ...]
  lldb-shell :: Commands/command-disassemble-riscv32-bytes.s
  lldb-unit :: DAP/./DAPTests/DisconnectRequestHandlerTest/DisconnectTriggersTerminateCommands
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/modifying/unique.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/algorithms/modifying/unique_copy.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/atomic_wait_1_waiter_1_notifier.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/atomic_wait_N_waiter_N_notifier.bench.cpp
  llvm-libc++-shared.cfg.in :: benchmarks/atomic_wait_multi_waiter_1_notifier.bench.cpp
[...]
== 2025-09-17 13:17:02,646 llvm.py:1377 INFO Tests timed out: 15
== 2025-09-17 13:17:02,646 llvm.py:1380 INFO Counting timed out tests as failed tests
== 2025-09-17 13:17:02,646 llvm.py:1392 INFO Ignored 374 failed tests due to ignore patterns
== 2025-09-17 13:17:02,646 build_log.py:434 WARNING Test failure ignored: Too many failed tests: 29 (10 allowed)

[Crivella]

Yeah the problem is always those timed out tests.
I have not had the time to dig more why the benchmarks are being ran for the runtimes even if we set LLVM_INCLUDE_BENCHMARKS=OFF.
Tried setting it OFF also as a runtime args, but it is then ignored when you do make check-all

[Thyre]

No problem 😄
I just wanted to give another test result while digging into llvm/llvm-project#159088

[Crivella]

Forgot the easyblock...

Test report by @Crivella
FAILED
Build succeeded (with --ignore-test-failure) for 0 out of 1 (1 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.13
See https://gist.github.com/Crivella/984cc80e241a30ce624d4df14b841b7c for a full test report.

[Crivella]

Test report by @Crivella
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3902
SUCCESS
Build succeeded (with --ignore-test-failure) for 1 out of 1 (1 easyconfigs in total)
crivella-desktop - Linux Ubuntu 22.04.5 LTS (Jammy Jellyfish), x86_64, 13th Gen Intel(R) Core(TM) i9-13900K (skylake), Python 3.11.13
See https://gist.github.com/Crivella/ad27d3963c604719faa991952e6690c7 for a full test report.

[Crivella]

Closing this in favor of having 21.x on the newest TC

• {tools}[GCCcore/15.2.0] lit v18.1.8, LLVM v21.1.8, psutil v7.2.1, Z3 v4.15.4 #25009