Skip to content

only retain runtime dependencies of toolchain in Toolchain instance#5048

Merged
boegel merged 2 commits intoeasybuilders:developfrom
Flamefire:fix-runtime-tc-deps
Nov 14, 2025
Merged

only retain runtime dependencies of toolchain in Toolchain instance#5048
boegel merged 2 commits intoeasybuilders:developfrom
Flamefire:fix-runtime-tc-deps

Conversation

@Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Nov 14, 2025

In #5042 tcdeps is actually used to get the software roots. Those are only available for runtime dependencies of the toolchain module. So it will fail for build dependencies which are however included by EasyConfig.toolchain.

Should we load the tcdeps explicitely before loading the toolchain module? The dry-run output seems to suggest that See https://github.com/boegel/easybuild-framework/blob/f7b811b49358e8608230db44835e389c2f6cbaf0/easybuild/tools/toolchain/toolchain.py#L627

edit (by @boegel): this is a fix for failures like:

ERROR: Installation of filevercmp-20191210-GCCcore-13.3.0.eb failed: 'get_software_root software root for M4 was not found in environment'

In easybuilders#5042 `tcdeps` is actually used to get the software roots.
Those are only available for *runtime* dependencies of the toolchain
module. So it will fail for build dependencies which are however
included by EasyConfig.toolchain.
boegel
boegel previously approved these changes Nov 14, 2025
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, fix is spot on I think, nice job @Flamefire

I'll dig a little more, mostly to understand why this doesn't affect all builds, but fix looks sane to me.

Until now, tcdeps was only used to produce dry run output, for nothing else.
This was wrong all along, but nobody really cared or noticed because it only leads to wrong dry run output if the module for the toolchain does not exist yet...

For example, in output of eb bzip2-1.0.8-GCCcore-14.3.0.eb -x:

Loading toolchain module...

module load M4/1.4.20 [SIMULATED]
module load binutils/2.44 [SIMULATED]
module load GCCcore/14.3.0 [SIMULATED]

@bedroge
Copy link
Contributor

bedroge commented Nov 14, 2025

I've tested this with easybuilders/easybuild-easyconfigs#24563. Without the patch from this PR I got:

== FAILED: Installation ended unsuccessfully: get_software_root software root for M4 was not found in environment (took 43 secs)

And with the patch it works fine:

== COMPLETED: Installation ended successfully (took 13 secs)
== Results of the build can be found in the log file(s) /data/apps/software/fermi-lite/20190320-GCCcore-13.3.0/easybuild/easybuild-fermi-lite-20190320-20251114.200828.log

== Build succeeded for 1 out of 1 (total: 13 secs)
== Summary:
   * [SUCCESS] fermi-lite/20190320-GCCcore-13.3.0

@boegel boegel added the bug fix label Nov 14, 2025
@boegel boegel added this to the next release (5.2.0?) milestone Nov 14, 2025
@boegel boegel changed the title Fix error due to access to toolchain build dependencies only retain runtime dependencies of toolchain in Toolchain instance Nov 14, 2025
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel
Copy link
Member

boegel commented Nov 14, 2025

Without the fix, the enhanced test triggers the bug:

$ python3 -O -m test.framework.suite test_exts_deps_build_env
...
======================================================================
ERROR: test_exts_deps_build_env (test.framework.easyblock.EasyBlockTest.test_exts_deps_build_env)
Test whether dependencies are loaded in build environment for extensions.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/work/easybuild-framework/test/framework/easyblock.py", line 3825, in test_exts_deps_build_env
    self.eb_main(args, raise_error=True, do_build=True, verbose=True)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/easybuild-framework/test/framework/utilities.py", line 352, in eb_main
    raise myerr
  File "/Volumes/work/easybuild-framework/test/framework/utilities.py", line 325, in eb_main
    main(args=main_args, logfile=logfile, do_build=do_build, testing=testing, modtool=modtool)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/easybuild-framework/easybuild/main.py", line 778, in main
    do_cleanup = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
                                 hooks, do_build)
  File "/Volumes/work/easybuild-framework/easybuild/main.py", line 596, in process_eb_args
    ecs_with_res = build_and_install_software(
        ordered_ecs, init_session_state, exit_on_failure=exit_on_failure, testing=testing)
  File "/Volumes/work/easybuild-framework/easybuild/main.py", line 209, in build_and_install_software
    raise error
easybuild.tools.build_log.EasyBuildError: "Installation of test.eb failed: 'get_software_root software root for gzip was not found in environment'"

@boegel boegel merged commit b4ef5df into easybuilders:develop Nov 14, 2025
50 of 51 checks passed
@Flamefire
Copy link
Contributor Author

I'll dig a little more, mostly to understand why this doesn't affect all builds, but fix looks sane to me.

I guess many toolchains don't have build dependencies.

Until now, tcdeps was only used to produce dry run output, for nothing else.
This was wrong all along, but nobody really care of noticed because it only leads to wrong dry run output if the module for the toolchain does not exist yet...

Is it wrong though? I guess the toolchain module will automatically load the dependencies, doesn't it?
If that is true then we don't needs this dry-run output.
If not (always) then we need to explicitely load tcdeps in "real-code"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants