enable `runtest` and `test_cmd` customization in `MesonNinja` easyblock by Thyre · Pull Request #4111 · easybuilders/easybuild-easyblocks

Thyre · 2026-04-11T10:20:01Z

(created using eb --new-pr)

Instead of relying on each EasyConfig to set runtest = ninja test, provide a sane default and enable users of the EasyBlock to just set runtest = True for a sane default. Also enable customization via test_cmd, available in other generic easyblocks.

To further ensure that we're not using too many resources, limit the parallelism via environment variables described in the Meson documentation. Just calling meson test would use all available CPU cores, ignoring our wanted parallelism.

Closes #4087

Co-authored-by: Kenneth Hoste <kenneth.hoste@ugent.be>

boegel · 2026-06-20T12:55:56Z

+        test_cmd = self.cfg.get('test_cmd') or DEFAULT_TEST_CMD
+        runtest = self.cfg['runtest']


@Thyre We need to be a bit more careful here I think, to avoid introducing a backwards-incompatible change...

For an exiting easyconfig file like cairomm-1.16.2-GCC-12.3.0.eb which uses MesonNinja as easyblock and includes this currently:

runtest = 'ninja test'

this (and the logic below) means trouble, because we're essentially prepending "meson " to it, which results in:

== testing... >> running shell command: export MESON_TESTTHREADS=16 && export MESON_NUM_PROCESSES=16 && meson ninja test

Our easyconfig files are being updated so they work nicely with this updated MesonNinja easyblock in easybuilders/easybuild-easyconfigs#25749, but we need to take into account that we don't control all easyconfig files out there.

We should try and come up with an approach where existing easyconfigs that include runtest = 'ninja test' keep working as expected...

test_cmd should only be set to the default when runtest is True.
For all other values, we should keep this empty.

This should keep the old behavior for existing external EasyConfigs.

Co-authored-by: Kenneth Hoste <kenneth.hoste@ugent.be>

Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>

Thyre · 2026-06-20T13:35:12Z

Test report by @Thyre

Overview of tested easyconfigs (in order)

SUCCESS HPCToolkit-2026.0.1-gompi-2026.1.eb

Build succeeded for 1 out of 1 (total: 1 min 38 secs) (1 easyconfigs in total)
Linux - Linux Arch Linux UNKNOWN, x86_64, AMD Ryzen 7 7800X3D 8-Core Processor (zen4), 1 x AMD Navi 48 [Radeon RX 9070/9070 XT/9070 GRE] (device id: 0x7550, gfx: gfx1201, driver: Linuxversion7.0.11-arch1-1(linux@archlinux)(gcc(GCC)16.1.120260430,GNUld(GNUBinutils)2.46.0)#1SMPPREEMPT_DYNAMICTue,02Jun202618:26:58+0000), 1 x AMD Raphael (device id: 0x164e, gfx: gfx1036, driver: Linuxversion7.0.11-arch1-1(linux@archlinux)(gcc(GCC)16.1.120260430,GNUld(GNUBinutils)2.46.0)#1SMPPREEMPT_DYNAMICTue,02Jun202618:26:58+0000), Python 3.14.5
See https://gist.github.com/Thyre/93892b9f145a523e60f929f84dc3e1d5 for a full test report.

Thyre · 2026-06-20T13:43:05Z

Test report by @Thyre

Overview of tested easyconfigs (in order)

SUCCESS GStreamer-1.26.5-GCCcore-14.3.0.eb

Build succeeded for 1 out of 1 (total: 46 secs) (1 easyconfigs in total)
Linux - Linux Arch Linux UNKNOWN, x86_64, AMD Ryzen 7 7800X3D 8-Core Processor (zen4), 1 x AMD Navi 48 [Radeon RX 9070/9070 XT/9070 GRE] (device id: 0x7550, gfx: gfx1201, driver: Linuxversion7.0.11-arch1-1(linux@archlinux)(gcc(GCC)16.1.120260430,GNUld(GNUBinutils)2.46.0)#1SMPPREEMPT_DYNAMICTue,02Jun202618:26:58+0000), 1 x AMD Raphael (device id: 0x164e, gfx: gfx1036, driver: Linuxversion7.0.11-arch1-1(linux@archlinux)(gcc(GCC)16.1.120260430,GNUld(GNUBinutils)2.46.0)#1SMPPREEMPT_DYNAMICTue,02Jun202618:26:58+0000), Python 3.14.5
See https://gist.github.com/Thyre/4716e1085bbfc45a3278a04dc9125ad5 for a full test report.

Thyre · 2026-06-20T13:45:05Z

HPCToolkit (using meson test) and GStreamer (using ninja test) seem to work again.
Will test more thoroughly with the corresponding EasyConfig PR.

One drawback of the current implementation is that MESON_TESTTHREADS and MESON_NUM_PROCESSES are now only set for runtest = True. It's probably worth to generalize this more, so that it's always set if meson shows up in either runtest or test_cmd.

But first, I want to do some more testing.

Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>

Thyre · 2026-06-20T16:36:22Z

@boegelbot please test @ jsc-zen3
EB_ARGS="--installpath /tmp/$USER/ebpr-4111 CrystFEL-0.11.1-foss-2024a.eb cairomm-1.16.2-GCC-11.3.0.eb cairomm-1.16.2-GCC-12.3.0.eb GStreamer-1.26.5-GCCcore-14.3.0.eb GStreamer-1.26.6-GCCcore-14.2.0.eb HPCToolkit-2025.0.1-gompi-2025a-CUDA-12.8.0.eb HPCToolkit-2025.0.1-gompi-2025a.eb HPCToolkit-2025.0.1-gompi-2025b-CUDA-12.9.1.eb HPCToolkit-2025.0.1-gompi-2025b.eb HPCToolkit-2026.0.1-gompi-2026.1.eb HarfBuzz-11.2.1-GCCcore-14.2.0.eb HarfBuzz-11.4.1-GCCcore-14.3.0.eb HarfBuzz-14.2.0-GCCcore-15.2.0.eb HarfBuzz-9.0.0-GCCcore-13.3.0.eb libvips-8.15.2-GCC-12.3.0.eb libvips-8.16.1-GCC-13.2.0.eb libvips-8.16.1-GCC-13.3.0.eb MODFLOW-6.4.4-foss-2023a.eb MODFLOW-6.4.4-foss-2023b.eb numsa-0.2.0-gfbf-2023b.eb numsa-0.2.0-gfbf-2024a.eb PretextMap-0.1.9-GCCcore-12.3.0.eb xraylib-4.2.1-gfbf-2025b.eb xtb-6.5.1-foss-2022a.eb xtb-6.6.0-foss-2022a.eb xtb-6.6.0-intel-2022a.eb xtb-6.6.1-foss-2022b.eb xtb-6.6.1-gfbf-2022b.eb xtb-6.6.1-gfbf-2023a.eb xtb-6.7.0-gfbf-2023a.eb CPCM-X-1.1.0-gfbf-2023b.eb CPCM-X-1.1.0-gfbf-2024a.eb xtb-6.7.1-gfbf-2023b.eb xtb-6.7.1-gfbf-2024a.eb MATLAB-2024b.eb Dynare-6.0-foss-2023b-MATLAB-2024b.eb"

boegelbot · 2026-06-20T16:53:07Z

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=4111 EB_ARGS="--installpath /tmp/$USER/ebpr-4111 CrystFEL-0.11.1-foss-2024a.eb cairomm-1.16.2-GCC-11.3.0.eb cairomm-1.16.2-GCC-12.3.0.eb GStreamer-1.26.5-GCCcore-14.3.0.eb GStreamer-1.26.6-GCCcore-14.2.0.eb HPCToolkit-2025.0.1-gompi-2025a-CUDA-12.8.0.eb HPCToolkit-2025.0.1-gompi-2025a.eb HPCToolkit-2025.0.1-gompi-2025b-CUDA-12.9.1.eb HPCToolkit-2025.0.1-gompi-2025b.eb HPCToolkit-2026.0.1-gompi-2026.1.eb HarfBuzz-11.2.1-GCCcore-14.2.0.eb HarfBuzz-11.4.1-GCCcore-14.3.0.eb HarfBuzz-14.2.0-GCCcore-15.2.0.eb HarfBuzz-9.0.0-GCCcore-13.3.0.eb libvips-8.15.2-GCC-12.3.0.eb libvips-8.16.1-GCC-13.2.0.eb libvips-8.16.1-GCC-13.3.0.eb MODFLOW-6.4.4-foss-2023a.eb MODFLOW-6.4.4-foss-2023b.eb numsa-0.2.0-gfbf-2023b.eb numsa-0.2.0-gfbf-2024a.eb PretextMap-0.1.9-GCCcore-12.3.0.eb xraylib-4.2.1-gfbf-2025b.eb xtb-6.5.1-foss-2022a.eb xtb-6.6.0-foss-2022a.eb xtb-6.6.0-intel-2022a.eb xtb-6.6.1-foss-2022b.eb xtb-6.6.1-gfbf-2022b.eb xtb-6.6.1-gfbf-2023a.eb xtb-6.7.0-gfbf-2023a.eb CPCM-X-1.1.0-gfbf-2023b.eb CPCM-X-1.1.0-gfbf-2024a.eb xtb-6.7.1-gfbf-2023b.eb xtb-6.7.1-gfbf-2024a.eb MATLAB-2024b.eb Dynare-6.0-foss-2023b-MATLAB-2024b.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_4111 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 10891

Test results coming soon (I hope)...

Details

- notification for comment with ID 4759051325 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2026-06-20T19:14:17Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS CrystFEL-0.11.1-foss-2024a.eb
SUCCESS cairomm-1.16.2-GCC-11.3.0.eb
SUCCESS cairomm-1.16.2-GCC-12.3.0.eb
SUCCESS GStreamer-1.26.5-GCCcore-14.3.0.eb
SUCCESS GStreamer-1.26.6-GCCcore-14.2.0.eb
SUCCESS HPCToolkit-2025.0.1-gompi-2025a-CUDA-12.8.0.eb
SUCCESS HPCToolkit-2025.0.1-gompi-2025a.eb
SUCCESS HPCToolkit-2025.0.1-gompi-2025b-CUDA-12.9.1.eb
SUCCESS HPCToolkit-2025.0.1-gompi-2025b.eb
SUCCESS HPCToolkit-2026.0.1-gompi-2026.1.eb
SUCCESS HarfBuzz-11.2.1-GCCcore-14.2.0.eb
SUCCESS HarfBuzz-11.4.1-GCCcore-14.3.0.eb
SUCCESS HarfBuzz-14.2.0-GCCcore-15.2.0.eb
SUCCESS HarfBuzz-9.0.0-GCCcore-13.3.0.eb
SUCCESS libvips-8.15.2-GCC-12.3.0.eb
SUCCESS libvips-8.16.1-GCC-13.2.0.eb
SUCCESS libvips-8.16.1-GCC-13.3.0.eb
SUCCESS MODFLOW-6.4.4-foss-2023a.eb
SUCCESS MODFLOW-6.4.4-foss-2023b.eb
SUCCESS numsa-0.2.0-gfbf-2023b.eb
SUCCESS numsa-0.2.0-gfbf-2024a.eb
SUCCESS PretextMap-0.1.9-GCCcore-12.3.0.eb
SUCCESS xraylib-4.2.1-gfbf-2025b.eb
FAIL xtb-6.5.1-foss-2022a.eb (build issue)
(partial log available at https://gist.github.com/boegelbot/26e2e7878e7fc69a275e12e7997ef3c1)
SUCCESS xtb-6.6.0-foss-2022a.eb
SUCCESS xtb-6.6.0-intel-2022a.eb
SUCCESS xtb-6.6.1-foss-2022b.eb
SUCCESS xtb-6.6.1-gfbf-2022b.eb
SUCCESS xtb-6.6.1-gfbf-2023a.eb
SUCCESS xtb-6.7.0-gfbf-2023a.eb
SUCCESS CPCM-X-1.1.0-gfbf-2023b.eb
SUCCESS CPCM-X-1.1.0-gfbf-2024a.eb
SUCCESS xtb-6.7.1-gfbf-2023b.eb
SUCCESS xtb-6.7.1-gfbf-2024a.eb
FAIL MATLAB-2024b.eb (build issue)
(partial log available at https://gist.github.com/boegelbot/5fe7c4cf9860a54075b035ed81717f59)
FAIL Dynare-6.0-foss-2023b-MATLAB-2024b.eb (build issue)
(partial log available at https://gist.github.com/boegelbot/64392673b4e09824769703ef0335e3d4)

Build succeeded for 33 out of 36 (total: 2 hours 20 mins 42 secs) (36 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.7, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.25
See https://gist.github.com/boegelbot/871ae85f216cd268c2dfc6c68b076b9b for a full test report.

Thyre · 2026-06-20T19:18:44Z

I'd guess that the test failure is unrelated? The last two are still due to MATLAB missing on jsc-zen3.

Thyre · 2026-06-20T21:16:27Z

@boegelbot please test @ jsc-zen3
EB_ARGS="--installpath /tmp/$USER/ebpr-4111 xtb-6.5.1-foss-2022a.eb"

boegelbot · 2026-06-20T21:33:07Z

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=4111 EB_ARGS="--installpath /tmp/$USER/ebpr-4111 xtb-6.5.1-foss-2022a.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_4111 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 10897

Test results coming soon (I hope)...

Details

- notification for comment with ID 4760011115 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2026-06-20T21:39:51Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS xtb-6.5.1-foss-2022a.eb

Build succeeded for 1 out of 1 (total: 6 mins 31 secs) (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.7, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.25
See https://gist.github.com/boegelbot/8e6806d03f7b3535ac1ca7af77d58a67 for a full test report.

Crivella · 2026-08-10T09:24:20Z

+            else:
+                test_cmd = ''


I think we might want to give a warning if we are overriding a test_cmd set in the EC file.

Maybe for another PR but in the configuremake EB we use both test_cmd and runtest, would have to check other EBs but we might want to make this behavior more consistent (maybe even move the general test logic at the framework level)

enable 'runtest' and 'test_cmd' customization in MesonNinja easyblock

17ddc93

Thyre added the enhancement label Apr 11, 2026

Thyre mentioned this pull request Apr 11, 2026

replace meson test and ninja test by runtest = True in easyconfigs using MesonNinja easyblock easybuilders/easybuild-easyconfigs#25749

Open

1 task

boegel changed the title ~~enable 'runtest' and 'test_cmd' customization in MesonNinja easyblock~~ enable runtest and test_cmd customization in MesonNinja easyblock Jun 20, 2026

boegel requested changes Jun 20, 2026

View reviewed changes

boegel added this to the next release (5.3.2?) milestone Jun 20, 2026

boegel added 2 commits June 20, 2026 14:22

minor code style fix in MesonNinja easyblock: fix order

f23608d

Co-authored-by: Kenneth Hoste <kenneth.hoste@ugent.be>

add leading space when appending to pretestopts

4b35424

Co-authored-by: Kenneth Hoste <kenneth.hoste@ugent.be>

boegel requested changes Jun 20, 2026

View reviewed changes

Thyre and others added 2 commits June 20, 2026 15:14

Use update instead of += to append options

7e3d185

Co-authored-by: Kenneth Hoste <kenneth.hoste@ugent.be>

ensure that old easyconfigs can still run with mesonninja changes

0abc2da

Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>

always apply env vars for limiting parallelism

a2e6975

Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>

boegel modified the milestones: next release (5.3.2?), release after release of Aug'26 Aug 7, 2026

Crivella reviewed Aug 10, 2026

View reviewed changes

		test_cmd = self.cfg.get('test_cmd') or DEFAULT_TEST_CMD
		runtest = self.cfg['runtest']

Uh oh!

Conversation

Thyre commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

boegel Jun 20, 2026

Choose a reason for hiding this comment

Uh oh!

Thyre Jun 20, 2026

Choose a reason for hiding this comment

Uh oh!

Thyre commented Jun 20, 2026

Overview of tested easyconfigs (in order)

Uh oh!

Thyre commented Jun 20, 2026

Overview of tested easyconfigs (in order)

Uh oh!

Thyre commented Jun 20, 2026

Uh oh!

Thyre commented Jun 20, 2026

Uh oh!

boegelbot commented Jun 20, 2026

Uh oh!

boegelbot commented Jun 20, 2026

Overview of tested easyconfigs (in order)

Uh oh!

Thyre commented Jun 20, 2026

Uh oh!

Thyre commented Jun 20, 2026

Uh oh!

boegelbot commented Jun 20, 2026

Uh oh!

boegelbot commented Jun 20, 2026

Overview of tested easyconfigs (in order)

Uh oh!

Crivella Aug 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Thyre commented Apr 11, 2026 •

edited

Loading