add support to Python easyblock for conditionally applying extra patch to fix ctypes when EasyBuild is configured to filter $LD_LIBRARY_PATH

[casparvl]

This change introduces extra easyblock-specific configuration items. They allow us to define patches that should be applied conditionally if EasyBuild is configured to filter LD_LIBRARY_PATH. Since these patches (see Should be used together with: easybuilders/easybuild-easyconfigs#23499) are quite fundamental (change ctypes functions) we prefer them to only be applied in the scenario where they are really needed.

Replaces: #3798
Should be used together with: easybuilders/easybuild-easyconfigs#23499

edit: relevant EESSI support issue: https://gitlab.com/eessi/support/-/issues/154

[boegel]

Code style check still fails:

./easybuild/easyblocks/p/python.py:386:23: F541 f-string is missing placeholders
./easybuild/easyblocks/p/python.py:387:24: F541 f-string is missing placeholders
./easybuild/easyblocks/p/python.py:393:5: E303 too many blank lines (2)

[casparvl]

Ok, sanity check seems to work as intended. Using the patch from easybuilders/easybuild-easyconfigs#23499 I get a successful sanity check, with this in the debugging output:

== 2025-10-15 10:53:41,597 run.py:457 INFO run_shell_cmd: command environment of "python -c 'import ctypes; print(ctypes.CDLL("libpython3.so"))'" will be saved to /scratch-local/casparl.15333717/eb-g6qr2e5w/run-shell-cmd-output/python-utohkjqy
== 2025-10-15 10:53:41,597 run.py:460 INFO run_shell_cmd: Output of "python -c 'import ctypes; print(ctypes.CDLL("libpython3.so"))'" will be logged to /scratch-local/casparl.15333717/eb-g6qr2e5w/run-shell-cmd-output/python-utohkjqy/out.txt
== 2025-10-15 10:53:41,617 run.py:494 INFO Path to bash that will be used to run shell commands: /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/bin/bash
== 2025-10-15 10:53:41,617 run.py:508 INFO Running shell command 'python -c 'import ctypes; print(ctypes.CDLL("libpython3.so"))'' in /tmp/casparl/easybuild/build/Python/3.11.5/GCCcore-13.2.0/pip/pip-23.2.1
== 2025-10-15 10:53:42,135 run.py:622 INFO Shell command completed successfully: python -c 'import ctypes; print(ctypes.CDLL("libpython3.so"))'
== 2025-10-15 10:53:42,135 run.py:624 INFO Output of 'python ...' shell command (stdout + stderr):
<CDLL '/home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Python/3.11.5-GCCcore-13.2.0/lib/../lib64/libpython3.so', handle 11ed940 at 0x1456f50afb10>

== 2025-10-15 10:53:42,138 python.py:748 DEBUG Matching regular expression pattern ^<CDLL '/home/casparl/eessi/versions/2023\.06/software/linux/x86_64/amd/zen2/software/Python/3\.11\.5\-GCCcore\-13\.2\.0.*', handle [a-f0-9]+ at 0x[a-f0-9]+>$ to string <CDLL '/home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Python/3.11.5-GCCcore-13.2.0/lib/../lib64/libpython3.so', handle 11ed940 at 0x1456f50afb10>
== 2025-10-15 10:53:42,138 python.py:752 INFO Call to ctypes.CDLL('libpython3.so') succesfully opened libpython3.so, indicating that the patch that fixes ctypes when EasyBuild is configured to filter LD_LIBRARY_PATH was applied succesfully.

If I use a dummy patch that just inserts some comments, I get:

== FAILED: Installation ended unsuccessfully: Opening of libpython3.so using ctypes.CDLL('libpython3.so') failed. Ctypes requires a patch when
EasyBuild is configured to filter LD_LIBRARY_PATH. Please check if you specified a patch through patch_ctypes_ld_library_path and check the
logs to see if it applied correctly. (took 12 mins 41 secs)
== Results of the build can be found in the log file(s)
/scratch-local/casparl.15371430/eb-h2rz1r18/easybuild-Python-3.11.5-20251015.104150.PXTEX.log
== Running post-easyblock hook...
== Summary:
   * [FAILED]  Python/3.11.5-GCCcore-13.2.0

ERROR: Installation of Python-3.11.5-GCCcore-13.2.0.eb failed: "Opening of libpython3.so using ctypes.CDLL('libpython3.so') failed. Ctypes requires a patch when EasyBuild is configured to filter LD_LIBRARY_PATH. Please check if you specified a patch through patch_ctypes_ld_library_path and check the logs to see if it applied correctly."

[casparvl]

@boegelbot please test @ jsc-zen3
EB_ARGS="Python-3.11.5-GCCcore-13.2.0.eb Python-3.12.3-GCCcore-13.3.0.eb Python-3.13.1-GCCcore-14.2.0.eb"

[boegelbot]

@casparvl: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3860 EB_ARGS="Python-3.11.5-GCCcore-13.2.0.eb Python-3.12.3-GCCcore-13.3.0.eb Python-3.13.1-GCCcore-14.2.0.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3860 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 8415

Test results coming soon (I hope)...

Details

- notification for comment with ID 3407980100 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[boegelbot]

Test report by @boegelbot

Overview of tested easyconfigs (in order)

• FAIL (build issue) Python-3.11.5-GCCcore-13.2.0.eb (partial log available at https://gist.github.com/boegelbot/5a9a4a6bc891002793da112a908e4ccd)
• SUCCESS Python-3.12.3-GCCcore-13.3.0.eb
• SUCCESS Python-3.13.1-GCCcore-14.2.0.eb

Build succeeded for 2 out of 3 (3 easyconfigs in total)
jsczen3c3.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/321093a94c35bb285ea06efca5fe4069 for a full test report.

[casparvl]

3.11.5 failed because I was rebuilding that in easybuilders/easybuild-easyconfigs#23499 as well, so lock file existed. Trying again for good measure...

@boegelbot please test @ jsc-zen3
EB_ARGS="Python-3.11.5-GCCcore-13.2.0.eb"

[boegelbot]

@casparvl: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3860 EB_ARGS="Python-3.11.5-GCCcore-13.2.0.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3860 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 8416

Test results coming soon (I hope)...

Details

- notification for comment with ID 3408456358 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[boegelbot]

Test report by @boegelbot

Overview of tested easyconfigs (in order)

• SUCCESS Python-3.11.5-GCCcore-13.2.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/6cb03b081d28cc424efa14fd78aff697 for a full test report.

[casparvl]

Strangely enough, I got a failure from the sanity check for 3.10.8:

== FAILED: Installation ended unsuccessfully: Opening of libpython3.so using ctypes.CDLL('libpython3.so') failed. Ctypes requires a patch
when EasyBuild is configured to filter LD_LIBRARY_PATH. Please check if you specified a patch through patch_ctypes_ld_library_path and check
the logs to see if it applied correctly. (took 10 mins 25 secs)

But inspecting the files that should have been patched, it seems the patch applied correctly I therefore installed with --skip-sanity-check, and tried to run the sanity check commands manually: they produce the correct output (different output from what was produced as part of the sanity check!). I don't understand why, maybe some different things in the environment of the sanity check are present that do not match the environment if I try after the install is fully completed?

Anyway, to help debug, I'll add a find_library call to the sanity check, and make sure that gets run first. It's good to have as part of the sanity check anyway. And it'll help us pinpoint where in the process things go wrong (e.g. does find_library return something incorrect?).

[casparvl]

I've tried with the find_library call in the sanity check, and get:

== 2025-10-17 21:43:08,529 run.py:622 INFO Shell command completed successfully: python -c 'from ctypes import util; print(util.find_library("libpython3.so"))'
== 2025-10-17 21:43:08,533 run.py:624 INFO Output of 'python ...' shell command (stdout + stderr):
libpython3.so

That is unexpected: we would expect it to return the full path - and as mentioned if I skip the sanity check, finalize the build, and then run that command, it returns the full path just fine.

@boegel any ideas perhaps why this would be different between 3.10.8 and 3.11.5? :\

Edit: Just to add, running e.g.

eb Python-3.10.8-GCCcore-12.2.0-bare.eb --from-pr 23499 --include-easyblocks-from-pr 3860 --rebuild --skip-sanity-check

and then

eb Python-3.10.8-GCCcore-12.2.0-bare.eb --from-pr 23499 --include-easyblocks-from-pr 3860 --rebuild --sanity-check-only

causes the sanity check to pass just fine.

[casparvl]

Ok, the reason turned out to be an issue where the patch would just return the libname if that library was present in the current working directory. That happened to be the case for 3.10.8 apparently, and probably not for 3.11.X. Anyway, fixed in easybuilders/easybuild-easyconfigs@f682e59 .

[boegel]

Test report by @boegel

Overview of tested easyconfigs (in order)

• SUCCESS Python-3.12.3-GCCcore-13.3.0.eb
• FAIL (build issue) Python-3.13.1-GCCcore-14.2.0.eb (partial log available at https://gist.github.com/boegel/02fcd3ef8029c1def4ae39e9373d820d)

Build succeeded for 1 out of 2 (2 easyconfigs in total)
node3541.doduo.os - Linux RHEL 9.6 (Plow), x86_64, AMD EPYC 7552 48-Core Processor, Python 3.13.4
See https://gist.github.com/boegel/e7e852989d768b926bccda5c01503e4a for a full test report.

edit: Failure is due to the patch from easybuilders/easybuild-easyconfigs#24189 not being used, I'll look into that (probably outdated easyconfigs installation on my end)

[boegel]

@casparvl All suggestions tackled in casparvl#5

[boegel]

I'm inclined to simply always run this additional sanity check, regardless of whether or not $LD_LIBRARY_PATH is being filtered.

But that won't work, since the unpatched find_library won't find a hit for libpython3.so (only for python3).

With that in mind, I remove patch_ctypes_ld_library_path from this condition: we want to make sure that if $LD_LIBRARY_PATH is filtered, that we still have a working find_library...

[boegel]

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_ARGS="Python-2.7.18-GCCcore-12.3.0.eb Python-3.10.8-GCCcore-12.2.0.eb Python-3.12.3-GCCcore-13.3.0.eb Python-3.13.5-GCCcore-14.3.0.eb --installpath /tmp/$USER/pr3860"

[boegelbot]

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3860 EB_ARGS="Python-2.7.18-GCCcore-12.3.0.eb Python-3.10.8-GCCcore-12.2.0.eb Python-3.12.3-GCCcore-13.3.0.eb Python-3.13.5-GCCcore-14.3.0.eb --installpath /tmp/$USER/pr3860" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3860 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 8522

Test results coming soon (I hope)...

Details

- notification for comment with ID 3431402779 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

requested changes made

[boegel]

lgtm

[boegelbot]

Test report by @boegelbot

Overview of tested easyconfigs (in order)

• SUCCESS Python-2.7.18-GCCcore-12.3.0.eb
• SUCCESS Python-3.10.8-GCCcore-12.2.0.eb
• SUCCESS Python-3.12.3-GCCcore-13.3.0.eb
• SUCCESS Python-3.13.5-GCCcore-14.3.0.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/2277005a1df352cc1d4b24ba067fb469 for a full test report.

[boegel]

It's better to do this in the constructor, see:

• add patch specified via patch_ctypes_ld_library_path to patches in constructor of Python easyblock #3965