Skip to content

enhance PythonPackage and PythonBundle easyblocks to add support to install dummy Python packages#4063

Merged
boegel merged 8 commits intoeasybuilders:developfrom
lexming:dummy-pip-packages
Apr 8, 2026
Merged

enhance PythonPackage and PythonBundle easyblocks to add support to install dummy Python packages#4063
boegel merged 8 commits intoeasybuilders:developfrom
lexming:dummy-pip-packages

Conversation

@lexming
Copy link
Copy Markdown
Contributor

@lexming lexming commented Feb 12, 2026

This PR adds a new dummy_package option to PythonPackage and PythonBundle that allows to install any python package as a dummy package, which is a fake installation that makes package managers believe that the package is installed.

In practice, this does the following:

  • sources of the dummy python package are not fetched or extracted
  • the usual build/installation of the python package does not happen
  • EB creates a dist-info folder in the site-packages directory with just a minimal METADATA file that contains the name and version of the package

The purpose of dummy packages is to be able to fulfill harcoded requirements in wheels that target python packages repackaging libraries that are already available in EasyBuild. The most notorious example are the CUDA libraries, wheels with support for CUDA need a collection of wheels from Nvidia that just repackage the CUDA libraries. Therefore, with this PR we can depend on the CUDA easyconfig as usual, while still have the fake cuda python packages to not break pip check.

Easyconfig using this feature in easybuilders/easybuild-easyconfigs#25266

@lexming
Copy link
Copy Markdown
Contributor Author

lexming commented Feb 14, 2026

@boegel boegel added this to the release after 5.2.1 milestone Feb 16, 2026
@pavelToman
Copy link
Copy Markdown
Contributor

@boegel boegel changed the title add support to install dummy python packages enhance PythonPackage and PythonBundle easyblocks to add support to install dummy Python packages Mar 25, 2026
retain condition to replace 'python' in exts_filter, only do so when needed
@boegel
Copy link
Copy Markdown
Member

boegel commented Mar 26, 2026

@boegelbot please test @ jsc-zen3
EB_ARGS="--installpath /tmp/$USER/pr4063 Python-3.11.3-GCCcore-12.3.0.eb Python-3.14.2-GCCcore-15.2.0.eb Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb matplotlib-3.10.5-gfbf-2025b.eb tqdm-4.67.1-GCCcore-14.3.0.eb"

@boegelbot
Copy link
Copy Markdown

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=4063 EB_ARGS="--installpath /tmp/$USER/pr4063 Python-3.11.3-GCCcore-12.3.0.eb Python-3.14.2-GCCcore-15.2.0.eb Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb matplotlib-3.10.5-gfbf-2025b.eb tqdm-4.67.1-GCCcore-14.3.0.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_4063 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 10097

Test results coming soon (I hope)...

Details

- notification for comment with ID 4132702682 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegel
Copy link
Copy Markdown
Member

boegel commented Mar 26, 2026

✅ retested use of dummy_package custom easyconfig parameter via:

@boegelbot
Copy link
Copy Markdown

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS Python-3.11.3-GCCcore-12.3.0.eb

  • SUCCESS Python-3.14.2-GCCcore-15.2.0.eb

  • SUCCESS Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb

  • SUCCESS matplotlib-3.10.5-gfbf-2025b.eb

  • SUCCESS tqdm-4.67.1-GCCcore-14.3.0.eb

Build succeeded for 5 out of 5 (total: 3 hours 2 mins 32 secs) (5 easyconfigs in total)
jsczen3c3.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.7, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.25
See https://gist.github.com/boegelbot/13b42af0ff95e230aad86e50d6b06572 for a full test report.

@boegel
Copy link
Copy Markdown
Member

boegel commented Mar 26, 2026

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb

  • SUCCESS SciPy-bundle-2025.07-gfbf-2025b.eb

  • SUCCESS Pillow-11.3.0-GCCcore-14.3.0.eb

  • SUCCESS hypothesis-6.90.0-GCCcore-13.2.0.eb

  • SUCCESS zlib-ng-2.2.4-GCCcore-14.2.0.eb

  • SUCCESS statsmodels-0.14.5-gfbf-2025a.eb

Build succeeded for 6 out of 6 (total: 4 hours 38 mins 59 secs) (6 easyconfigs in total)
node4217.shinx.os - Linux RHEL 9.6, x86_64, AMD EPYC 9654 96-Core Processor (zen4), Python 3.9.21
See https://gist.github.com/boegel/c077bdf9a6b755f84a3f5c51324b5199 for a full test report.

@boegel boegel merged commit 73323f0 into easybuilders:develop Apr 8, 2026
22 checks passed
@lexming lexming deleted the dummy-pip-packages branch April 9, 2026 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants