Skip to content

Conversation

@boegel
Copy link
Member

@boegel boegel commented May 23, 2025

Bug fix for problem introduced in #3432 (there was a comment indicating that is was necessary, but split_stderr was not enabled in the run_shell_cmd call)

This is important when pip produced deprecation warnings to stderr, for example when installing Python-2.7.18-GCCcore-12.3.0.eb:

$ python -m pip list --isolated --disable-pip-version-check --format json
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
[{"version": "20.3.4", "name": "pip"}, {"version": "44.1.1", "name": "setuptools"}, {"version": "0.37.1", "name": "wheel"}]

Without this fix, there's a horrible crash in sanity check when installing Python-2.7.18-GCCcore-12.3.0.eb:

== Summary:
   * [FAILED]  Python/2.7.18-GCCcore-12.3.0

EasyBuild crashed! Please consider reporting a bug, this should not happen...

Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/easybuild/easybuild-framework/easybuild/main.py", line 829, in <module>
    main_with_hooks()
  File "/easybuild/easybuild-framework/easybuild/main.py", line 815, in main_with_hooks
    main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))
  File "/easybuild/easybuild-framework/easybuild/main.py", line 770, in main
    do_cleanup = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
  File "/easybuild/easybuild-framework/easybuild/main.py", line 592, in process_eb_args
    ecs_with_res = build_and_install_software(
  File "/easybuild/easybuild-framework/easybuild/main.py", line 206, in build_and_install_software
    raise error
  File "/easybuild/easybuild-framework/easybuild/main.py", line 160, in build_and_install_software
    (ec_res['success'], app_log, err_msg, err_code) = build_and_install_one(ec, init_env)
  File "/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 5015, in build_and_install_one
    result = app.run_all_steps(run_test_cases=run_test_cases)
  File "/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4835, in run_all_steps
    self.run_step(step_name, step_methods)
  File "/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4677, in run_step
    step_method(self)()
  File "/easybuild/easybuild-easyblocks/easybuild/easyblocks/p/python.py", line 637, in sanity_check_step
    super().sanity_check_step(custom_paths=custom_paths, custom_commands=custom_commands)
  File "/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 3329, in sanity_check_step
    self._dispatch_sanity_check_step(*args, **kwargs)
  File "/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 3317, in _dispatch_sanity_check_step
    self._sanity_check_step(*args, **kwargs)
  File "/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4302, in _sanity_check_step
    self._sanity_check_step_extensions()
  File "/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4152, in _sanity_check_step_extensions
    res = ext.sanity_check_step()
  File "/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/pythonpackage.py", line 1187, in sanity_check_step
    run_pip_check(python_cmd=python_cmd, unversioned_packages=unversioned_packages)
  File "/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/pythonpackage.py", line 468, in run_pip_check
    pkgs = det_installed_python_packages(names_only=False, python_cmd=python_cmd)
  File "/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/pythonpackage.py", line 429, in det_installed_python_packages
    pkgs = json.loads(res.output.strip())
  File "/usr/lib64/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

@boegel boegel added this to the next release (5.1.0) milestone May 23, 2025
@boegel boegel added the bug fix label May 23, 2025
@boegel
Copy link
Member Author

boegel commented May 23, 2025

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS Python-2.7.18-GCCcore-12.3.0.eb
  • SUCCESS Python-3.13.1-GCCcore-14.2.0.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3539.doduo.os - Linux RHEL 9.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.9.18
See https://gist.github.com/boegel/8acbe164dae57deb54c0c2c69456d6cc for a full test report.

Copy link
Contributor

@lexming lexming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lexming
Copy link
Contributor

lexming commented May 23, 2025

Merging, thanks @boegel !

@lexming lexming merged commit 0ea34a7 into easybuilders:develop May 23, 2025
17 checks passed
@boegel boegel deleted the fix_det_installed_python_packages branch May 23, 2025 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants