Unify handling of vendored crates in cargo EasyBlock and add support for non-virtual workspaces #3665

Flamefire · 2025-03-14T16:02:22Z

(created using eb --new-pr)

Followup to #3654 implementing the suggested approach from Flamefire#9 (comment)

It basically removes self.git_vendor_dir and the additional template(s) required for "workspaces" in Git repos to avoid having to duplicate the same logic for "branch" in those too and possibly later for other entries like "tag".

As a result all offline preparation code could be moved to a single function.

What needs to be done for vendored sources:

For every package (folder with a Cargo.toml file) we need a .checksums.json file
~~Virtual~~ workspaces need to be "resolved" to individual packages else cargo errors due to a "virtual manifest"

A "Virtual manifest" is a Cargo.toml file with a [workspace] section but without a [package] section. That has members which are folders containing sub-packages. Each of that needs to be a folder inside what we set as directory = in the $CARGO_HOME/config.toml file.

Previously I extracted git sources to a separate folder, check for workspaces and moved non-workspaces to our regular vendor folder. For workspaces the directory entry for the git repository was set to the folder of the workspace. I.e. the one containing the subfolders. This worked a bit by accident: For the folder set by directory Cargo will search (and require!) direct subfolders. Hence the "outer Cargo.toml" and everything else was irrelevant.
cargo vendor does that: Omit the outer folder and copy only the subfolders to the vendor folder containing all vendored crates.

So that is what I did here:

Extract all crates to our (single) vendor folder
Afterwards iterate over all folders containing a Cargo.toml folder inside that. This catches our TODO case when a source could contain multiple folders with packages
Check each Cargo.toml folder for a virtual workspace. If found move the whole folder out of our vendor folder and copy only the members back
Create the checksum file for the newly copied members or just the single folder

One downside is an extra copy of crates in virtual workspaces because they might contain symlinks to their parent folder. I've seen that for e.g. the LICENSE file. We need to copy the file, not the symlink as cargo vendor does too.

It can get harder: Packages inside a workspace might reference the workspace file. cargo vendor resolves those too, i.e. replace foo.workspace = True by foo = <value of foo in workspace Cargo.toml> in the members. We might need to do that too. Doesn't seem to be necessary for what we have now though.

Those virtual workspaces only apply to git repositories.
Another approach would be:

Extract git sources to a separate git-vendor folder (again) inside a subfolder each (i.e. creating an extra level like subfoo/foo-1.2.3)
For all git sources use the CONFIG_TOML_SOURCE_GIT_WORKSPACE template
For each git source check if it is a virtual workspace. if it is use the full path for the $CARGO_HOME entry, e.g. subfoo/foo-1.2.3 otherwise use the parent folder, i.e. subfoo
As before create checksum files for each folder containing a Cargo.toml file but include the 2nd level for the git-vendor folder, i.e. vendor_dir/*/Cargo.toml, git_vendor_dir/*/Cargo.toml, git_vendor_dir/*/*/Cargo.toml to match the virtual workspaces

This is basically the inverse approach as implemented here: Don't put all (git) sources into the same folder but each in a separate folder.
That has the nice side-effect of avoiding name collisions. Not sure if they are possible at all though.

Edit: Turns out you can have non-virtual workspaces where the parent folder contains a [[package]].
For vendoring purposes all packages need to be subfolders of a (root) vendor folder and contain a .cargo-checksum.json file.

--> Move all members of workspaces out and delete the parent folder if it was only a virtual workspace, but keep if it has a package at the root.

Flamefire · 2025-03-14T16:53:47Z

Test report by @Flamefire

Overview of tested easyconfigs (in order)

SUCCESS uv-0.4.20-GCCcore-13.3.0.eb
SUCCESS polars-0.20.2-gfbf-2023a.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
i7008 - Linux Rocky Linux 8.9 (Green Obsidian), x86_64, AMD EPYC 7702 64-Core Processor (zen2), Python 3.8.17
See https://gist.github.com/Flamefire/761c1589d1c31e5ff774aa37d3760f2a for a full test report.

lexming · 2025-03-15T10:29:47Z

@Flamefire please target 5.0.x, there will be no new changes put into 4.9

Flamefire · 2025-03-15T11:41:04Z

Ok I think I got the merge conflict properly resolved. Used the Browser though but looks fine

boegel · 2025-03-19T11:12:32Z

@Flamefire I changed to target branch in this PR from 5.0.x to develop, you should synchronize your PR branch with current develop branch (which has received a massive update after the release of EasyBuild v5.0.0, see #3670)

Flamefire · 2025-07-22T15:35:21Z

Test report by @Flamefire

Overview of tested easyconfigs (in order)

SUCCESS uv-0.4.20-GCCcore-13.3.0.eb
FAIL (build issue) polars-0.20.2-gfbf-2023a.eb (partial log available at https://gist.github.com/Flamefire/e15ed481e399ab73eaac59beeaced142) ---- Exists on current develop branch too

Build succeeded for 1 out of 2 (2 easyconfigs in total)
n1637 - Linux RHEL 8.9 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (sapphirerapids), Python 3.9.18
See https://gist.github.com/Flamefire/005aa44d649c501f12e0d6238ab0e591 for a full test report.

The Cargo.toml file might contain: ``` [workspace] members = ["components/salsa-macro-rules", "components/salsa-macros"] ``` This means we need to move the 2 subfolders of components and NOT the components folder into the vendor root-directory so they are found. Hence use `basename` for determining the target path which is a no-op when no folder is present.

Seen e.g. for lsp-types in ruff: [[package]] name = "lsp-types" version = "0.95.1" source = "git+https://github.com/astral-sh/lsp-types.git?rev=3512a9f#3512a9f33eadc5402cfab1b8f7340824c8ca1439"

Micket

lgtm, lets get a bunch of testing done on this

boegel · 2025-10-10T17:44:11Z

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_ARGS="--installpath /tmp/$USER/pr3665 alevin-fry-0.9.0-GCCcore-13.2.0.eb bamtofastq-1.4.1-GCCcore-12.3.0.eb bcrypt-4.0.1-GCCcore-12.3.0.eb bcrypt-4.1.3-GCCcore-13.2.0.eb bcrypt-4.3.0-GCCcore-13.3.0.eb bcrypt-4.3.0-GCCcore-14.2.0.eb bcrypt-4.3.0-GCCcore-14.3.0.eb cargo-c-0.10.15-GCCcore-14.3.0.eb cargo-c-0.9.32-GCCcore-13.3.0.eb chopper-0.9.0-GCCcore-12.3.0.eb Clarabel.rs-0.7.1-gfbf-2023a.eb clearml-1.16.5-foss-2023b.eb cramino-0.14.5-GCC-12.3.0.eb cramjam-2.9.0-GCCcore-12.3.0.eb cryptography-41.0.1-GCCcore-12.3.0.eb cryptography-41.0.5-GCCcore-13.2.0.eb cryptography-42.0.8-GCCcore-13.3.0.eb cryptography-44.0.2-GCCcore-14.2.0.eb cryptography-45.0.5-GCCcore-14.3.0.eb DeltaLake-0.15.1-gfbf-2023a.eb DeltaLake-0.25.5-gfbf-2023b.eb DeltaLake-0.25.5-gfbf-2024a.eb elfx86exts-0.6.2-GCC-12.3.0.eb Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb fastparquet-2023.4.0-gfbf-2022b.eb HERRO-0.1.0_20240808-foss-2023a.eb huggingface_hub-0.34.4-GCCcore-13.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb juliaup-1.17.9-GCCcore-12.3.0.eb jupyter-collaboration-2.1.1-GCCcore-13.2.0.eb jupyter-server-2.14.0-GCCcore-13.2.0.eb jupyter-server-2.14.2-GCCcore-13.3.0.eb jupyter-server-2.16.0-GCCcore-14.2.0.eb jupyter-server-2.17.0-GCCcore-14.3.0.eb jupyter-server-2.7.2-GCCcore-12.3.0.eb JupyterLab-4.0.3-GCCcore-12.2.0.eb kyber-0.4.0-GCC-12.3.0.eb lazrs-0.7.0-GCCcore-14.2.0.eb Longshot-0.4.5-GCCcore-11.3.0.eb Longshot-1.0.0-GCCcore-12.3.0.eb Longshot-1.0.0-GCCcore-13.2.0.eb maturin-1.1.0-GCCcore-12.2.0.eb maturin-1.1.0-GCCcore-12.3.0.eb maturin-1.3.1-GCCcore-13.2.0.eb maturin-1.3.2-GCCcore-11.3.0-Rust-1.65.0.eb maturin-1.4.0-GCCcore-12.2.0-Rust-1.75.0.eb maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb maturin-1.5.0-GCCcore-13.2.0-Rust-1.76.0.eb maturin-1.6.0-GCCcore-13.3.0.eb maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb maturin-1.8.3-GCCcore-13.3.0-Rust-1.83.0.eb maturin-1.8.3-GCCcore-14.2.0.eb maturin-1.9.1-GCCcore-14.3.0.eb mfqe-0.5.0-GCC-12.3.0.eb modkit-0.3.3-GCCcore-12.3.0.eb modkit-0.4.1-GCCcore-13.3.0.eb modkit-0.5.0-GCCcore-13.3.0.eb orjson-3.9.15-GCCcore-12.3.0.eb phasius-0.2.0-GCC-12.3.0.eb poetry-1.7.1-GCCcore-12.3.0.eb poetry-1.7.1-GCCcore-13.2.0.eb poetry-1.8.3-GCCcore-13.3.0.eb poetry-2.1.2-GCCcore-14.2.0.eb poetry-2.1.3-GCCcore-14.3.0.eb polars-0.19.19-gfbf-2023a.eb polars-0.20.2-gfbf-2023a.eb polars-1.28.1-gfbf-2023b.eb polars-1.28.1-gfbf-2024a.eb pybigtools-0.1.2-gfbf-2023a.eb pydantic-2.11.7-GCCcore-14.2.0.eb pydantic-2.5.3-GCCcore-12.2.0.eb pydantic-2.5.3-GCCcore-12.3.0.eb pydantic-2.6.4-GCCcore-13.2.0.eb pydantic-2.7.4-GCCcore-13.2.0.eb pydantic-2.9.1-GCCcore-13.3.0.eb pymatgen-2023.12.18-foss-2023a.eb Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb RDP-0.13.5-GCCcore-13.3.0.eb ripunzip-0.4.0.eb rustworkx-0.12.1-foss-2022a.eb rustworkx-0.12.1-foss-2023a.eb rustworkx-0.15.1-gfbf-2023a.eb rustworkx-0.16.0-gfbf-2024a.eb Safetensors-0.3.1-foss-2022a.eb Safetensors-0.4.3-gfbf-2022b.eb Safetensors-0.4.3-gfbf-2023a.eb Safetensors-0.4.4-gfbf-2023b.eb Safetensors-0.6.2-gfbf-2024a.eb scatac_fragment_tools-0.1.0-foss-2023a.eb SKA2-0.3.7-GCCcore-12.2.0.eb skani-0.2.2-GCCcore-12.3.0.eb smafa-0.8.0-GCC-12.3.0.eb SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb Spyder-6.0.1-GCCcore-13.2.0.eb subset-bam-1.1.0-GCCcore-10.3.0.eb tiktoken-0.6.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-13.2.0.eb tiktoken-0.9.0-GCCcore-13.3.0.eb timm-0.6.13-foss-2022a-CUDA-11.7.0.eb timm-0.9.7-foss-2022a-CUDA-11.7.0.eb tlparse-0.3.37-GCCcore-13.3.0.eb tlparse-0.3.5-GCCcore-13.2.0.eb tokenizers-0.13.3-GCCcore-11.3.0.eb tokenizers-0.13.3-GCCcore-12.2.0.eb tokenizers-0.15.2-GCCcore-12.3.0.eb tokenizers-0.19.1-GCCcore-13.2.0.eb tokenizers-0.21.4-GCCcore-13.3.0.eb torch-em-0.7.1-foss-2023a.eb tree-sitter-0.25.3-GCCcore-14.2.0.eb uv-0.2.30-GCCcore-13.3.0.eb vLLM-0.4.0-foss-2023a-CUDA-12.1.1.eb yacrd-1.0.0-foss-2023a.eb"

boegelbot · 2025-10-10T17:53:10Z

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3665 EB_ARGS="--installpath /tmp/$USER/pr3665 alevin-fry-0.9.0-GCCcore-13.2.0.eb bamtofastq-1.4.1-GCCcore-12.3.0.eb bcrypt-4.0.1-GCCcore-12.3.0.eb bcrypt-4.1.3-GCCcore-13.2.0.eb bcrypt-4.3.0-GCCcore-13.3.0.eb bcrypt-4.3.0-GCCcore-14.2.0.eb bcrypt-4.3.0-GCCcore-14.3.0.eb cargo-c-0.10.15-GCCcore-14.3.0.eb cargo-c-0.9.32-GCCcore-13.3.0.eb chopper-0.9.0-GCCcore-12.3.0.eb Clarabel.rs-0.7.1-gfbf-2023a.eb clearml-1.16.5-foss-2023b.eb cramino-0.14.5-GCC-12.3.0.eb cramjam-2.9.0-GCCcore-12.3.0.eb cryptography-41.0.1-GCCcore-12.3.0.eb cryptography-41.0.5-GCCcore-13.2.0.eb cryptography-42.0.8-GCCcore-13.3.0.eb cryptography-44.0.2-GCCcore-14.2.0.eb cryptography-45.0.5-GCCcore-14.3.0.eb DeltaLake-0.15.1-gfbf-2023a.eb DeltaLake-0.25.5-gfbf-2023b.eb DeltaLake-0.25.5-gfbf-2024a.eb elfx86exts-0.6.2-GCC-12.3.0.eb Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb fastparquet-2023.4.0-gfbf-2022b.eb HERRO-0.1.0_20240808-foss-2023a.eb huggingface_hub-0.34.4-GCCcore-13.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb juliaup-1.17.9-GCCcore-12.3.0.eb jupyter-collaboration-2.1.1-GCCcore-13.2.0.eb jupyter-server-2.14.0-GCCcore-13.2.0.eb jupyter-server-2.14.2-GCCcore-13.3.0.eb jupyter-server-2.16.0-GCCcore-14.2.0.eb jupyter-server-2.17.0-GCCcore-14.3.0.eb jupyter-server-2.7.2-GCCcore-12.3.0.eb JupyterLab-4.0.3-GCCcore-12.2.0.eb kyber-0.4.0-GCC-12.3.0.eb lazrs-0.7.0-GCCcore-14.2.0.eb Longshot-0.4.5-GCCcore-11.3.0.eb Longshot-1.0.0-GCCcore-12.3.0.eb Longshot-1.0.0-GCCcore-13.2.0.eb maturin-1.1.0-GCCcore-12.2.0.eb maturin-1.1.0-GCCcore-12.3.0.eb maturin-1.3.1-GCCcore-13.2.0.eb maturin-1.3.2-GCCcore-11.3.0-Rust-1.65.0.eb maturin-1.4.0-GCCcore-12.2.0-Rust-1.75.0.eb maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb maturin-1.5.0-GCCcore-13.2.0-Rust-1.76.0.eb maturin-1.6.0-GCCcore-13.3.0.eb maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb maturin-1.8.3-GCCcore-13.3.0-Rust-1.83.0.eb maturin-1.8.3-GCCcore-14.2.0.eb maturin-1.9.1-GCCcore-14.3.0.eb mfqe-0.5.0-GCC-12.3.0.eb modkit-0.3.3-GCCcore-12.3.0.eb modkit-0.4.1-GCCcore-13.3.0.eb modkit-0.5.0-GCCcore-13.3.0.eb orjson-3.9.15-GCCcore-12.3.0.eb phasius-0.2.0-GCC-12.3.0.eb poetry-1.7.1-GCCcore-12.3.0.eb poetry-1.7.1-GCCcore-13.2.0.eb poetry-1.8.3-GCCcore-13.3.0.eb poetry-2.1.2-GCCcore-14.2.0.eb poetry-2.1.3-GCCcore-14.3.0.eb polars-0.19.19-gfbf-2023a.eb polars-0.20.2-gfbf-2023a.eb polars-1.28.1-gfbf-2023b.eb polars-1.28.1-gfbf-2024a.eb pybigtools-0.1.2-gfbf-2023a.eb pydantic-2.11.7-GCCcore-14.2.0.eb pydantic-2.5.3-GCCcore-12.2.0.eb pydantic-2.5.3-GCCcore-12.3.0.eb pydantic-2.6.4-GCCcore-13.2.0.eb pydantic-2.7.4-GCCcore-13.2.0.eb pydantic-2.9.1-GCCcore-13.3.0.eb pymatgen-2023.12.18-foss-2023a.eb Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb RDP-0.13.5-GCCcore-13.3.0.eb ripunzip-0.4.0.eb rustworkx-0.12.1-foss-2022a.eb rustworkx-0.12.1-foss-2023a.eb rustworkx-0.15.1-gfbf-2023a.eb rustworkx-0.16.0-gfbf-2024a.eb Safetensors-0.3.1-foss-2022a.eb Safetensors-0.4.3-gfbf-2022b.eb Safetensors-0.4.3-gfbf-2023a.eb Safetensors-0.4.4-gfbf-2023b.eb Safetensors-0.6.2-gfbf-2024a.eb scatac_fragment_tools-0.1.0-foss-2023a.eb SKA2-0.3.7-GCCcore-12.2.0.eb skani-0.2.2-GCCcore-12.3.0.eb smafa-0.8.0-GCC-12.3.0.eb SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb Spyder-6.0.1-GCCcore-13.2.0.eb subset-bam-1.1.0-GCCcore-10.3.0.eb tiktoken-0.6.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-13.2.0.eb tiktoken-0.9.0-GCCcore-13.3.0.eb timm-0.6.13-foss-2022a-CUDA-11.7.0.eb timm-0.9.7-foss-2022a-CUDA-11.7.0.eb tlparse-0.3.37-GCCcore-13.3.0.eb tlparse-0.3.5-GCCcore-13.2.0.eb tokenizers-0.13.3-GCCcore-11.3.0.eb tokenizers-0.13.3-GCCcore-12.2.0.eb tokenizers-0.15.2-GCCcore-12.3.0.eb tokenizers-0.19.1-GCCcore-13.2.0.eb tokenizers-0.21.4-GCCcore-13.3.0.eb torch-em-0.7.1-foss-2023a.eb tree-sitter-0.25.3-GCCcore-14.2.0.eb uv-0.2.30-GCCcore-13.3.0.eb vLLM-0.4.0-foss-2023a-CUDA-12.1.1.eb yacrd-1.0.0-foss-2023a.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3665 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 8285

Test results coming soon (I hope)...

Details

- notification for comment with ID 3391484785 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2025-10-11T11:53:59Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS alevin-fry-0.9.0-GCCcore-13.2.0.eb
FAIL (build issue) bamtofastq-1.4.1-GCCcore-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/8221bc06290a2c446e4589fb123bb337)
SUCCESS bcrypt-4.0.1-GCCcore-12.3.0.eb
SUCCESS bcrypt-4.1.3-GCCcore-13.2.0.eb
SUCCESS bcrypt-4.3.0-GCCcore-13.3.0.eb
SUCCESS bcrypt-4.3.0-GCCcore-14.2.0.eb
SUCCESS bcrypt-4.3.0-GCCcore-14.3.0.eb
SUCCESS cargo-c-0.10.15-GCCcore-14.3.0.eb
SUCCESS cargo-c-0.9.32-GCCcore-13.3.0.eb
FAIL (build issue) chopper-0.9.0-GCCcore-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/077915b30284d09df3a38f673e5f62e7)
FAIL (build issue) cramino-0.14.5-GCC-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/45d56255a9c5b337a35a3e65548c0dc5)
SUCCESS cryptography-41.0.1-GCCcore-12.3.0.eb
SUCCESS cryptography-41.0.5-GCCcore-13.2.0.eb
SUCCESS cryptography-42.0.8-GCCcore-13.3.0.eb
SUCCESS elfx86exts-0.6.2-GCC-12.3.0.eb
SUCCESS Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb
SUCCESS HERRO-0.1.0_20240808-foss-2023a.eb
FAIL (build issue) juliaup-1.17.9-GCCcore-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/112f74567c90b4672d895b07728010e0)
FAIL (build issue) kyber-0.4.0-GCC-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/fa7fa636d3ee54b537369f453b6b23fe)
FAIL (build issue) Longshot-0.4.5-GCCcore-11.3.0.eb (partial log available at https://gist.github.com/boegelbot/f7325cdc9731fc0f1df133ff038b2ea2)
FAIL (build issue) Longshot-1.0.0-GCCcore-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/9c77fc1192d4ea19e7e0eb0f398f28b5)
FAIL (build issue) Longshot-1.0.0-GCCcore-13.2.0.eb (partial log available at https://gist.github.com/boegelbot/bb0b0b685aef531db09986f852b98cda)
SUCCESS maturin-1.1.0-GCCcore-12.2.0.eb
SUCCESS maturin-1.1.0-GCCcore-12.3.0.eb
SUCCESS maturin-1.3.1-GCCcore-13.2.0.eb
SUCCESS maturin-1.3.2-GCCcore-11.3.0-Rust-1.65.0.eb
SUCCESS maturin-1.4.0-GCCcore-12.2.0-Rust-1.75.0.eb
SUCCESS maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb
SUCCESS maturin-1.5.0-GCCcore-13.2.0-Rust-1.76.0.eb
SUCCESS maturin-1.6.0-GCCcore-13.3.0.eb
SUCCESS maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb
SUCCESS maturin-1.8.3-GCCcore-13.3.0-Rust-1.83.0.eb
SUCCESS maturin-1.8.3-GCCcore-14.2.0.eb
SUCCESS maturin-1.9.1-GCCcore-14.3.0.eb
SUCCESS mfqe-0.5.0-GCC-12.3.0.eb
FAIL (build issue) modkit-0.3.3-GCCcore-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/1c1a6f689b3de34748d6b5458da97f07)
FAIL (build issue) modkit-0.4.1-GCCcore-13.3.0.eb (partial log available at https://gist.github.com/boegelbot/6390a3041c560cebd96804e44e12ffbd)
FAIL (build issue) modkit-0.5.0-GCCcore-13.3.0.eb (partial log available at https://gist.github.com/boegelbot/061a23f232180c1e62e1439c5a0c4d85)
SUCCESS orjson-3.9.15-GCCcore-12.3.0.eb
FAIL (build issue) phasius-0.2.0-GCC-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/558f224135959b61f49bb9d6c27fdeed)
SUCCESS poetry-1.7.1-GCCcore-12.3.0.eb
SUCCESS poetry-1.7.1-GCCcore-13.2.0.eb
SUCCESS poetry-1.8.3-GCCcore-13.3.0.eb
SUCCESS pybigtools-0.1.2-gfbf-2023a.eb
SUCCESS pydantic-2.11.7-GCCcore-14.2.0.eb
SUCCESS pydantic-2.5.3-GCCcore-12.2.0.eb
SUCCESS pydantic-2.5.3-GCCcore-12.3.0.eb
SUCCESS pydantic-2.6.4-GCCcore-13.2.0.eb
SUCCESS pydantic-2.7.4-GCCcore-13.2.0.eb
SUCCESS pydantic-2.9.1-GCCcore-13.3.0.eb
SUCCESS pymatgen-2023.12.18-foss-2023a.eb
SUCCESS Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb
SUCCESS RDP-0.13.5-GCCcore-13.3.0.eb
SUCCESS ripunzip-0.4.0.eb
SUCCESS rustworkx-0.12.1-foss-2022a.eb
SUCCESS rustworkx-0.12.1-foss-2023a.eb
SUCCESS rustworkx-0.15.1-gfbf-2023a.eb
SUCCESS rustworkx-0.16.0-gfbf-2024a.eb
SUCCESS Safetensors-0.3.1-foss-2022a.eb
SUCCESS Safetensors-0.4.3-gfbf-2022b.eb
SUCCESS Safetensors-0.4.3-gfbf-2023a.eb
SUCCESS Safetensors-0.4.4-gfbf-2023b.eb
SUCCESS Safetensors-0.6.2-gfbf-2024a.eb
SUCCESS SKA2-0.3.7-GCCcore-12.2.0.eb
FAIL (build issue) skani-0.2.2-GCCcore-12.3.0.eb (partial log available at https://gist.github.com/boegelbot/abebff3049a17f520c4712f51ef879c4)
SUCCESS smafa-0.8.0-GCC-12.3.0.eb
FAIL (build issue) subset-bam-1.1.0-GCCcore-10.3.0.eb (partial log available at https://gist.github.com/boegelbot/481b6204b3a9d94f9fc206dd7d694fa2)
SUCCESS tiktoken-0.6.0-GCCcore-12.3.0.eb
SUCCESS tiktoken-0.7.0-GCCcore-12.3.0.eb
SUCCESS tiktoken-0.7.0-GCCcore-13.2.0.eb
SUCCESS tiktoken-0.9.0-GCCcore-13.3.0.eb
SUCCESS timm-0.6.13-foss-2022a-CUDA-11.7.0.eb
SUCCESS timm-0.9.7-foss-2022a-CUDA-11.7.0.eb
SUCCESS tlparse-0.3.37-GCCcore-13.3.0.eb
SUCCESS tlparse-0.3.5-GCCcore-13.2.0.eb
SUCCESS tokenizers-0.13.3-GCCcore-11.3.0.eb
SUCCESS tokenizers-0.13.3-GCCcore-12.2.0.eb
SUCCESS tokenizers-0.15.2-GCCcore-12.3.0.eb
SUCCESS tokenizers-0.19.1-GCCcore-13.2.0.eb
SUCCESS torch-em-0.7.1-foss-2023a.eb
SUCCESS tree-sitter-0.25.3-GCCcore-14.2.0.eb
SUCCESS uv-0.2.30-GCCcore-13.3.0.eb
SUCCESS vLLM-0.4.0-foss-2023a-CUDA-12.1.1.eb
SUCCESS yacrd-1.0.0-foss-2023a.eb
SUCCESS Clarabel.rs-0.7.1-gfbf-2023a.eb
SUCCESS clearml-1.16.5-foss-2023b.eb
SUCCESS cramjam-2.9.0-GCCcore-12.3.0.eb
SUCCESS cryptography-44.0.2-GCCcore-14.2.0.eb
SUCCESS cryptography-45.0.5-GCCcore-14.3.0.eb
SUCCESS DeltaLake-0.15.1-gfbf-2023a.eb
SUCCESS DeltaLake-0.25.5-gfbf-2023b.eb
SUCCESS DeltaLake-0.25.5-gfbf-2024a.eb
SUCCESS fastparquet-2023.4.0-gfbf-2022b.eb
SUCCESS huggingface_hub-0.34.4-GCCcore-13.3.0.eb
SUCCESS jiter-0.4.1-GCCcore-12.3.0.eb
SUCCESS jupyter-collaboration-2.1.1-GCCcore-13.2.0.eb
SUCCESS jupyter-server-2.14.0-GCCcore-13.2.0.eb
SUCCESS jupyter-server-2.14.2-GCCcore-13.3.0.eb
SUCCESS jupyter-server-2.16.0-GCCcore-14.2.0.eb
SUCCESS jupyter-server-2.17.0-GCCcore-14.3.0.eb
SUCCESS jupyter-server-2.7.2-GCCcore-12.3.0.eb
SUCCESS JupyterLab-4.0.3-GCCcore-12.2.0.eb
SUCCESS lazrs-0.7.0-GCCcore-14.2.0.eb
SUCCESS poetry-2.1.2-GCCcore-14.2.0.eb
SUCCESS poetry-2.1.3-GCCcore-14.3.0.eb
SUCCESS polars-0.19.19-gfbf-2023a.eb
SUCCESS polars-0.20.2-gfbf-2023a.eb
SUCCESS polars-1.28.1-gfbf-2023b.eb
SUCCESS polars-1.28.1-gfbf-2024a.eb
SUCCESS Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb
SUCCESS Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb
FAIL (build issue) scatac_fragment_tools-0.1.0-foss-2023a.eb (partial log available at https://gist.github.com/boegelbot/6b8f690a85fd2a1eec5ed814152e747d)
SUCCESS SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb
SUCCESS Spyder-6.0.1-GCCcore-13.2.0.eb
SUCCESS tokenizers-0.21.4-GCCcore-13.3.0.eb

Build succeeded for 100 out of 115 (115 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/3fb1fc5dd12d86c31bee5bf9763d1f9f for a full test report.

boegel · 2025-10-11T12:14:29Z

Similar problem across several failing easyconfigs:

== 2025-10-10 18:07:44,314 cargo.py:372 INFO Found workspace in /tmp/boegelbot/chopper/0.9.0/GCCcore-12.3.0/easybuild_vendor/custom_derive-0.1.7. Members: enum_derive, newtype_derive
== 2025-10-10 18:07:44,314 filetools.py:2682 DEBUG No recursive symlinks in /tmp/boegelbot/chopper/0.9.0/GCCcore-12.3.0/tmp_crate_q30qhqp7/custom_derive-0.1.7/enum_derive
== 2025-10-10 18:07:44,319 build_log.py:226 ERROR EasyBuild encountered an error (at easybuild/easybuild-framework/easybuild/base/exceptions.py:126 in __init__): Failed to copy directory /tmp/boegelbot/chopper/0.9.0/GCCcore-12.3.0/tmp_crate_q30qhqp7/custom_derive-0.1.7/enum_derive to /tmp/boegelbot/chopper/0.9.0/GCCcore-12.3.0/easybuild_vendor/enum_derive: [Errno 2] No such file or directory: '/tmp/boegelbot/chopper/0.9.0/GCCcore-12.3.0/tmp_crate_q30qhqp7/custom_derive-0.1.7/enum_derive' (at easybuild/easybuild-framework/easybuild/tools/filetools.py:2720 in copy_dir)
== 2025-10-10 18:07:44,319 build_log.py:322 INFO ... (took 9 secs)

@Flamefire Can you take a look?

Flamefire · 2025-10-14T10:36:50Z

I investigated this and the issue is caused by sources downloaded from crates.io. They keep their original Cargo.toml which lists workspace members. However it seems that upon uploading to crates.io those are split into individual crates so the subfolders don't exist anymore.
I've seen this in the example of custom_derive-0.1.7.tar.gz for modkit-0.4.1 where the GitHub tagged version has the subfolders but the crates.io download does not which causes the failure.

The solution I implemented is to only attempt the "split-up" when any member folder exists assuming when one exist then all have to exist. This involved a bit of cleanup and whitespace change which makes the diff of that commit look larger

However for me `modkit-0.4.1-GCCcore-13.3.0.eb fails in the test step due to a concurrency issue which exists already unrelated to this PR, see nanoporetech/modkit#520

Fixes for unrelated failures:

boegel · 2025-10-15T06:30:52Z

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_ARGS="--installpath /tmp/$USER/pr3665 alevin-fry-0.9.0-GCCcore-13.2.0.eb bamtofastq-1.4.1-GCCcore-12.3.0.eb bcrypt-4.0.1-GCCcore-12.3.0.eb bcrypt-4.1.3-GCCcore-13.2.0.eb bcrypt-4.3.0-GCCcore-13.3.0.eb bcrypt-4.3.0-GCCcore-14.2.0.eb bcrypt-4.3.0-GCCcore-14.3.0.eb cargo-c-0.10.15-GCCcore-14.3.0.eb cargo-c-0.9.32-GCCcore-13.3.0.eb chopper-0.9.0-GCCcore-12.3.0.eb Clarabel.rs-0.7.1-gfbf-2023a.eb clearml-1.16.5-foss-2023b.eb cramino-0.14.5-GCC-12.3.0.eb cramjam-2.9.0-GCCcore-12.3.0.eb cryptography-41.0.1-GCCcore-12.3.0.eb cryptography-41.0.5-GCCcore-13.2.0.eb cryptography-42.0.8-GCCcore-13.3.0.eb cryptography-44.0.2-GCCcore-14.2.0.eb cryptography-45.0.5-GCCcore-14.3.0.eb DeltaLake-0.15.1-gfbf-2023a.eb DeltaLake-0.25.5-gfbf-2023b.eb DeltaLake-0.25.5-gfbf-2024a.eb elfx86exts-0.6.2-GCC-12.3.0.eb Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb fastparquet-2023.4.0-gfbf-2022b.eb HERRO-0.1.0_20240808-foss-2023a.eb huggingface_hub-0.34.4-GCCcore-13.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb juliaup-1.17.9-GCCcore-12.3.0.eb jupyter-collaboration-2.1.1-GCCcore-13.2.0.eb jupyter-server-2.14.0-GCCcore-13.2.0.eb jupyter-server-2.14.2-GCCcore-13.3.0.eb jupyter-server-2.16.0-GCCcore-14.2.0.eb jupyter-server-2.17.0-GCCcore-14.3.0.eb jupyter-server-2.7.2-GCCcore-12.3.0.eb JupyterLab-4.0.3-GCCcore-12.2.0.eb kyber-0.4.0-GCC-12.3.0.eb lazrs-0.7.0-GCCcore-14.2.0.eb Longshot-0.4.5-GCCcore-11.3.0.eb Longshot-1.0.0-GCCcore-12.3.0.eb Longshot-1.0.0-GCCcore-13.2.0.eb maturin-1.1.0-GCCcore-12.2.0.eb maturin-1.1.0-GCCcore-12.3.0.eb maturin-1.3.1-GCCcore-13.2.0.eb maturin-1.3.2-GCCcore-11.3.0-Rust-1.65.0.eb maturin-1.4.0-GCCcore-12.2.0-Rust-1.75.0.eb maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb maturin-1.5.0-GCCcore-13.2.0-Rust-1.76.0.eb maturin-1.6.0-GCCcore-13.3.0.eb maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb maturin-1.8.3-GCCcore-13.3.0-Rust-1.83.0.eb maturin-1.8.3-GCCcore-14.2.0.eb maturin-1.9.1-GCCcore-14.3.0.eb mfqe-0.5.0-GCC-12.3.0.eb modkit-0.3.3-GCCcore-12.3.0.eb modkit-0.4.1-GCCcore-13.3.0.eb modkit-0.5.0-GCCcore-13.3.0.eb orjson-3.9.15-GCCcore-12.3.0.eb phasius-0.2.0-GCC-12.3.0.eb poetry-1.7.1-GCCcore-12.3.0.eb poetry-1.7.1-GCCcore-13.2.0.eb poetry-1.8.3-GCCcore-13.3.0.eb poetry-2.1.2-GCCcore-14.2.0.eb poetry-2.1.3-GCCcore-14.3.0.eb polars-0.19.19-gfbf-2023a.eb polars-0.20.2-gfbf-2023a.eb polars-1.28.1-gfbf-2023b.eb polars-1.28.1-gfbf-2024a.eb pybigtools-0.1.2-gfbf-2023a.eb pydantic-2.11.7-GCCcore-14.2.0.eb pydantic-2.5.3-GCCcore-12.2.0.eb pydantic-2.5.3-GCCcore-12.3.0.eb pydantic-2.6.4-GCCcore-13.2.0.eb pydantic-2.7.4-GCCcore-13.2.0.eb pydantic-2.9.1-GCCcore-13.3.0.eb pymatgen-2023.12.18-foss-2023a.eb Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb RDP-0.13.5-GCCcore-13.3.0.eb ripunzip-0.4.0.eb rustworkx-0.12.1-foss-2022a.eb rustworkx-0.12.1-foss-2023a.eb rustworkx-0.15.1-gfbf-2023a.eb rustworkx-0.16.0-gfbf-2024a.eb Safetensors-0.3.1-foss-2022a.eb Safetensors-0.4.3-gfbf-2022b.eb Safetensors-0.4.3-gfbf-2023a.eb Safetensors-0.4.4-gfbf-2023b.eb Safetensors-0.6.2-gfbf-2024a.eb scatac_fragment_tools-0.1.0-foss-2023a.eb SKA2-0.3.7-GCCcore-12.2.0.eb skani-0.2.2-GCCcore-12.3.0.eb smafa-0.8.0-GCC-12.3.0.eb SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb Spyder-6.0.1-GCCcore-13.2.0.eb subset-bam-1.1.0-GCCcore-10.3.0.eb tiktoken-0.6.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-13.2.0.eb tiktoken-0.9.0-GCCcore-13.3.0.eb timm-0.6.13-foss-2022a-CUDA-11.7.0.eb timm-0.9.7-foss-2022a-CUDA-11.7.0.eb tlparse-0.3.37-GCCcore-13.3.0.eb tlparse-0.3.5-GCCcore-13.2.0.eb tokenizers-0.13.3-GCCcore-11.3.0.eb tokenizers-0.13.3-GCCcore-12.2.0.eb tokenizers-0.15.2-GCCcore-12.3.0.eb tokenizers-0.19.1-GCCcore-13.2.0.eb tokenizers-0.21.4-GCCcore-13.3.0.eb torch-em-0.7.1-foss-2023a.eb tree-sitter-0.25.3-GCCcore-14.2.0.eb uv-0.2.30-GCCcore-13.3.0.eb vLLM-0.4.0-foss-2023a-CUDA-12.1.1.eb yacrd-1.0.0-foss-2023a.eb"

boegelbot · 2025-10-15T06:33:09Z

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3665 EB_ARGS="--installpath /tmp/$USER/pr3665 alevin-fry-0.9.0-GCCcore-13.2.0.eb bamtofastq-1.4.1-GCCcore-12.3.0.eb bcrypt-4.0.1-GCCcore-12.3.0.eb bcrypt-4.1.3-GCCcore-13.2.0.eb bcrypt-4.3.0-GCCcore-13.3.0.eb bcrypt-4.3.0-GCCcore-14.2.0.eb bcrypt-4.3.0-GCCcore-14.3.0.eb cargo-c-0.10.15-GCCcore-14.3.0.eb cargo-c-0.9.32-GCCcore-13.3.0.eb chopper-0.9.0-GCCcore-12.3.0.eb Clarabel.rs-0.7.1-gfbf-2023a.eb clearml-1.16.5-foss-2023b.eb cramino-0.14.5-GCC-12.3.0.eb cramjam-2.9.0-GCCcore-12.3.0.eb cryptography-41.0.1-GCCcore-12.3.0.eb cryptography-41.0.5-GCCcore-13.2.0.eb cryptography-42.0.8-GCCcore-13.3.0.eb cryptography-44.0.2-GCCcore-14.2.0.eb cryptography-45.0.5-GCCcore-14.3.0.eb DeltaLake-0.15.1-gfbf-2023a.eb DeltaLake-0.25.5-gfbf-2023b.eb DeltaLake-0.25.5-gfbf-2024a.eb elfx86exts-0.6.2-GCC-12.3.0.eb Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb fastparquet-2023.4.0-gfbf-2022b.eb HERRO-0.1.0_20240808-foss-2023a.eb huggingface_hub-0.34.4-GCCcore-13.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb juliaup-1.17.9-GCCcore-12.3.0.eb jupyter-collaboration-2.1.1-GCCcore-13.2.0.eb jupyter-server-2.14.0-GCCcore-13.2.0.eb jupyter-server-2.14.2-GCCcore-13.3.0.eb jupyter-server-2.16.0-GCCcore-14.2.0.eb jupyter-server-2.17.0-GCCcore-14.3.0.eb jupyter-server-2.7.2-GCCcore-12.3.0.eb JupyterLab-4.0.3-GCCcore-12.2.0.eb kyber-0.4.0-GCC-12.3.0.eb lazrs-0.7.0-GCCcore-14.2.0.eb Longshot-0.4.5-GCCcore-11.3.0.eb Longshot-1.0.0-GCCcore-12.3.0.eb Longshot-1.0.0-GCCcore-13.2.0.eb maturin-1.1.0-GCCcore-12.2.0.eb maturin-1.1.0-GCCcore-12.3.0.eb maturin-1.3.1-GCCcore-13.2.0.eb maturin-1.3.2-GCCcore-11.3.0-Rust-1.65.0.eb maturin-1.4.0-GCCcore-12.2.0-Rust-1.75.0.eb maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb maturin-1.5.0-GCCcore-13.2.0-Rust-1.76.0.eb maturin-1.6.0-GCCcore-13.3.0.eb maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb maturin-1.8.3-GCCcore-13.3.0-Rust-1.83.0.eb maturin-1.8.3-GCCcore-14.2.0.eb maturin-1.9.1-GCCcore-14.3.0.eb mfqe-0.5.0-GCC-12.3.0.eb modkit-0.3.3-GCCcore-12.3.0.eb modkit-0.4.1-GCCcore-13.3.0.eb modkit-0.5.0-GCCcore-13.3.0.eb orjson-3.9.15-GCCcore-12.3.0.eb phasius-0.2.0-GCC-12.3.0.eb poetry-1.7.1-GCCcore-12.3.0.eb poetry-1.7.1-GCCcore-13.2.0.eb poetry-1.8.3-GCCcore-13.3.0.eb poetry-2.1.2-GCCcore-14.2.0.eb poetry-2.1.3-GCCcore-14.3.0.eb polars-0.19.19-gfbf-2023a.eb polars-0.20.2-gfbf-2023a.eb polars-1.28.1-gfbf-2023b.eb polars-1.28.1-gfbf-2024a.eb pybigtools-0.1.2-gfbf-2023a.eb pydantic-2.11.7-GCCcore-14.2.0.eb pydantic-2.5.3-GCCcore-12.2.0.eb pydantic-2.5.3-GCCcore-12.3.0.eb pydantic-2.6.4-GCCcore-13.2.0.eb pydantic-2.7.4-GCCcore-13.2.0.eb pydantic-2.9.1-GCCcore-13.3.0.eb pymatgen-2023.12.18-foss-2023a.eb Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb RDP-0.13.5-GCCcore-13.3.0.eb ripunzip-0.4.0.eb rustworkx-0.12.1-foss-2022a.eb rustworkx-0.12.1-foss-2023a.eb rustworkx-0.15.1-gfbf-2023a.eb rustworkx-0.16.0-gfbf-2024a.eb Safetensors-0.3.1-foss-2022a.eb Safetensors-0.4.3-gfbf-2022b.eb Safetensors-0.4.3-gfbf-2023a.eb Safetensors-0.4.4-gfbf-2023b.eb Safetensors-0.6.2-gfbf-2024a.eb scatac_fragment_tools-0.1.0-foss-2023a.eb SKA2-0.3.7-GCCcore-12.2.0.eb skani-0.2.2-GCCcore-12.3.0.eb smafa-0.8.0-GCC-12.3.0.eb SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb Spyder-6.0.1-GCCcore-13.2.0.eb subset-bam-1.1.0-GCCcore-10.3.0.eb tiktoken-0.6.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-12.3.0.eb tiktoken-0.7.0-GCCcore-13.2.0.eb tiktoken-0.9.0-GCCcore-13.3.0.eb timm-0.6.13-foss-2022a-CUDA-11.7.0.eb timm-0.9.7-foss-2022a-CUDA-11.7.0.eb tlparse-0.3.37-GCCcore-13.3.0.eb tlparse-0.3.5-GCCcore-13.2.0.eb tokenizers-0.13.3-GCCcore-11.3.0.eb tokenizers-0.13.3-GCCcore-12.2.0.eb tokenizers-0.15.2-GCCcore-12.3.0.eb tokenizers-0.19.1-GCCcore-13.2.0.eb tokenizers-0.21.4-GCCcore-13.3.0.eb torch-em-0.7.1-foss-2023a.eb tree-sitter-0.25.3-GCCcore-14.2.0.eb uv-0.2.30-GCCcore-13.3.0.eb vLLM-0.4.0-foss-2023a-CUDA-12.1.1.eb yacrd-1.0.0-foss-2023a.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3665 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 8382

Test results coming soon (I hope)...

Details

- notification for comment with ID 3404771062 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2025-10-15T22:58:44Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS alevin-fry-0.9.0-GCCcore-13.2.0.eb
SUCCESS bamtofastq-1.4.1-GCCcore-12.3.0.eb
SUCCESS bcrypt-4.0.1-GCCcore-12.3.0.eb
SUCCESS bcrypt-4.1.3-GCCcore-13.2.0.eb
SUCCESS bcrypt-4.3.0-GCCcore-13.3.0.eb
SUCCESS bcrypt-4.3.0-GCCcore-14.2.0.eb
SUCCESS bcrypt-4.3.0-GCCcore-14.3.0.eb
SUCCESS cargo-c-0.10.15-GCCcore-14.3.0.eb
SUCCESS cargo-c-0.9.32-GCCcore-13.3.0.eb
SUCCESS chopper-0.9.0-GCCcore-12.3.0.eb
SUCCESS cramino-0.14.5-GCC-12.3.0.eb
SUCCESS cryptography-41.0.1-GCCcore-12.3.0.eb
SUCCESS cryptography-41.0.5-GCCcore-13.2.0.eb
SUCCESS cryptography-42.0.8-GCCcore-13.3.0.eb
SUCCESS elfx86exts-0.6.2-GCC-12.3.0.eb
SUCCESS Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb
SUCCESS HERRO-0.1.0_20240808-foss-2023a.eb
SUCCESS juliaup-1.17.9-GCCcore-12.3.0.eb
SUCCESS kyber-0.4.0-GCC-12.3.0.eb
SUCCESS Longshot-0.4.5-GCCcore-11.3.0.eb
SUCCESS Longshot-1.0.0-GCCcore-12.3.0.eb
SUCCESS Longshot-1.0.0-GCCcore-13.2.0.eb
SUCCESS maturin-1.1.0-GCCcore-12.2.0.eb
SUCCESS maturin-1.1.0-GCCcore-12.3.0.eb
SUCCESS maturin-1.3.1-GCCcore-13.2.0.eb
SUCCESS maturin-1.3.2-GCCcore-11.3.0-Rust-1.65.0.eb
SUCCESS maturin-1.4.0-GCCcore-12.2.0-Rust-1.75.0.eb
SUCCESS maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb
SUCCESS maturin-1.5.0-GCCcore-13.2.0-Rust-1.76.0.eb
SUCCESS maturin-1.6.0-GCCcore-13.3.0.eb
SUCCESS maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb
SUCCESS maturin-1.8.3-GCCcore-13.3.0-Rust-1.83.0.eb
SUCCESS maturin-1.8.3-GCCcore-14.2.0.eb
SUCCESS maturin-1.9.1-GCCcore-14.3.0.eb
SUCCESS mfqe-0.5.0-GCC-12.3.0.eb
SUCCESS modkit-0.3.3-GCCcore-12.3.0.eb
SUCCESS modkit-0.4.1-GCCcore-13.3.0.eb
SUCCESS modkit-0.5.0-GCCcore-13.3.0.eb
SUCCESS orjson-3.9.15-GCCcore-12.3.0.eb
SUCCESS phasius-0.2.0-GCC-12.3.0.eb
SUCCESS poetry-1.7.1-GCCcore-12.3.0.eb
SUCCESS poetry-1.7.1-GCCcore-13.2.0.eb
SUCCESS poetry-1.8.3-GCCcore-13.3.0.eb
SUCCESS pybigtools-0.1.2-gfbf-2023a.eb
SUCCESS pydantic-2.11.7-GCCcore-14.2.0.eb
SUCCESS pydantic-2.5.3-GCCcore-12.2.0.eb
SUCCESS pydantic-2.5.3-GCCcore-12.3.0.eb
SUCCESS pydantic-2.6.4-GCCcore-13.2.0.eb
SUCCESS pydantic-2.7.4-GCCcore-13.2.0.eb
SUCCESS pydantic-2.9.1-GCCcore-13.3.0.eb
SUCCESS pymatgen-2023.12.18-foss-2023a.eb
SUCCESS Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb
SUCCESS RDP-0.13.5-GCCcore-13.3.0.eb
SUCCESS ripunzip-0.4.0.eb
SUCCESS rustworkx-0.12.1-foss-2022a.eb
SUCCESS rustworkx-0.12.1-foss-2023a.eb
SUCCESS rustworkx-0.15.1-gfbf-2023a.eb
SUCCESS rustworkx-0.16.0-gfbf-2024a.eb
SUCCESS Safetensors-0.3.1-foss-2022a.eb
SUCCESS Safetensors-0.4.3-gfbf-2022b.eb
SUCCESS Safetensors-0.4.3-gfbf-2023a.eb
SUCCESS Safetensors-0.4.4-gfbf-2023b.eb
SUCCESS Safetensors-0.6.2-gfbf-2024a.eb
SUCCESS SKA2-0.3.7-GCCcore-12.2.0.eb
SUCCESS skani-0.2.2-GCCcore-12.3.0.eb
SUCCESS smafa-0.8.0-GCC-12.3.0.eb
SUCCESS subset-bam-1.1.0-GCCcore-10.3.0.eb
SUCCESS tiktoken-0.6.0-GCCcore-12.3.0.eb
SUCCESS tiktoken-0.7.0-GCCcore-12.3.0.eb
SUCCESS tiktoken-0.7.0-GCCcore-13.2.0.eb
SUCCESS tiktoken-0.9.0-GCCcore-13.3.0.eb
SUCCESS timm-0.6.13-foss-2022a-CUDA-11.7.0.eb
SUCCESS timm-0.9.7-foss-2022a-CUDA-11.7.0.eb
SUCCESS tlparse-0.3.37-GCCcore-13.3.0.eb
SUCCESS tlparse-0.3.5-GCCcore-13.2.0.eb
SUCCESS tokenizers-0.13.3-GCCcore-11.3.0.eb
SUCCESS tokenizers-0.13.3-GCCcore-12.2.0.eb
SUCCESS tokenizers-0.15.2-GCCcore-12.3.0.eb
SUCCESS tokenizers-0.19.1-GCCcore-13.2.0.eb
SUCCESS torch-em-0.7.1-foss-2023a.eb
SUCCESS tree-sitter-0.25.3-GCCcore-14.2.0.eb
SUCCESS uv-0.2.30-GCCcore-13.3.0.eb
SUCCESS vLLM-0.4.0-foss-2023a-CUDA-12.1.1.eb
SUCCESS yacrd-1.0.0-foss-2023a.eb
SUCCESS Clarabel.rs-0.7.1-gfbf-2023a.eb
SUCCESS clearml-1.16.5-foss-2023b.eb
SUCCESS cramjam-2.9.0-GCCcore-12.3.0.eb
SUCCESS cryptography-44.0.2-GCCcore-14.2.0.eb
SUCCESS cryptography-45.0.5-GCCcore-14.3.0.eb
SUCCESS DeltaLake-0.15.1-gfbf-2023a.eb
SUCCESS DeltaLake-0.25.5-gfbf-2023b.eb
SUCCESS DeltaLake-0.25.5-gfbf-2024a.eb
SUCCESS fastparquet-2023.4.0-gfbf-2022b.eb
SUCCESS huggingface_hub-0.34.4-GCCcore-13.3.0.eb
SUCCESS jiter-0.4.1-GCCcore-12.3.0.eb
SUCCESS jupyter-collaboration-2.1.1-GCCcore-13.2.0.eb
SUCCESS jupyter-server-2.14.0-GCCcore-13.2.0.eb
SUCCESS jupyter-server-2.14.2-GCCcore-13.3.0.eb
SUCCESS jupyter-server-2.16.0-GCCcore-14.2.0.eb
SUCCESS jupyter-server-2.17.0-GCCcore-14.3.0.eb
SUCCESS jupyter-server-2.7.2-GCCcore-12.3.0.eb
SUCCESS JupyterLab-4.0.3-GCCcore-12.2.0.eb
SUCCESS lazrs-0.7.0-GCCcore-14.2.0.eb
SUCCESS poetry-2.1.2-GCCcore-14.2.0.eb
SUCCESS poetry-2.1.3-GCCcore-14.3.0.eb
SUCCESS polars-0.19.19-gfbf-2023a.eb
SUCCESS polars-0.20.2-gfbf-2023a.eb
SUCCESS polars-1.28.1-gfbf-2023b.eb
SUCCESS polars-1.28.1-gfbf-2024a.eb
FAIL (build issue) Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb (partial log available at https://gist.github.com/boegelbot/bcc75f12401c4354b536879f1e3b04f0)
SUCCESS Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb
SUCCESS scatac_fragment_tools-0.1.0-foss-2023a.eb
FAIL (build issue) SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb (partial log available at https://gist.github.com/boegelbot/33ea3cf1d4cc57be027a516fdd149b92)
SUCCESS Spyder-6.0.1-GCCcore-13.2.0.eb
SUCCESS tokenizers-0.21.4-GCCcore-13.3.0.eb

Build succeeded for 113 out of 115 (115 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/3be5a059e4c2bcd949abc9173960f4f0 for a full test report.

Flamefire · 2025-10-16T07:53:05Z

FAIL (build issue) Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb (partial log available at https://gist.github.com/boegelbot/bcc75f12401c4354b536879f1e3b04f0)

main Python module was missing/being rebuild

FAIL (build issue) SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb (partial log available at https://gist.github.com/boegelbot/33ea3cf1d4cc57be027a516fdd149b92)

unrelated intermediate failure

Flamefire · 2025-10-16T08:02:03Z

Test report by @Flamefire

Overview of tested easyconfigs (in order)

SUCCESS SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
c144 - Linux AlmaLinux 9.4, x86_64, AMD EPYC 9334 32-Core Processor (zen4), 4 x NVIDIA NVIDIA H100, 560.35.03, Python 3.9.18
See https://gist.github.com/Flamefire/bde8b8364932b977312ca2405ed62bf7 for a full test report.

boegel · 2025-10-16T08:59:24Z

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_ARGS="--installpath /tmp/$USER/pr3665 Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb"

boegelbot · 2025-10-16T09:13:09Z

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3665 EB_ARGS="--installpath /tmp/$USER/pr3665 Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3665 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 8421

Test results coming soon (I hope)...

Details

- notification for comment with ID 3409885625 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2025-10-16T11:44:20Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS Python-bundle-PyPI-2025.04-GCCcore-14.2.0.eb
SUCCESS SnapATAC2-2.9.0-dev0-20250630-foss-2024a.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/cc24bb60bdf0a0c41e0c41ec2bcef5dc for a full test report.

boegel

lgtm

Flamefire marked this pull request as draft March 14, 2025 16:25

Flamefire marked this pull request as ready for review March 14, 2025 17:06

Flamefire changed the base branch from develop to 5.0.x March 15, 2025 11:30

boegel changed the base branch from 5.0.x to develop March 19, 2025 11:12

boegel added the code cleanup label Mar 19, 2025

boegel added this to the 5.x milestone Mar 19, 2025

boegel modified the milestones: 5.x.x, 5.x Apr 9, 2025

Flamefire force-pushed the 20250314170220_new_pr_cargo branch from ddb8ddd to 4eef8a4 Compare May 23, 2025 09:01

Unify handling of vendored crates in cargo EasyBlock

5c3dee5

Flamefire force-pushed the 20250314170220_new_pr_cargo branch 2 times, most recently from 9b60a7d to 5e67e77 Compare July 22, 2025 13:32

Support non-virtual workspaces

9188602

Flamefire force-pushed the 20250314170220_new_pr_cargo branch from 5e67e77 to 9188602 Compare July 22, 2025 13:34

Flamefire mentioned this pull request Jul 22, 2025

{tools}[GCCcore/14.2.0] uv v0.7.13 easybuilders/easybuild-easyconfigs#23143

Merged

1 task

Flamefire changed the title ~~Unify handling of vendored crates in cargo EasyBlock~~ Unify handling of vendored crates in cargo EasyBlock and add support for non-virtual workspaces Jul 22, 2025

Flamefire added 2 commits July 23, 2025 16:47

Allow short commit hashes for Cargo crates

661d543

Seen e.g. for lsp-types in ruff: [[package]] name = "lsp-types" version = "0.95.1" source = "git+https://github.com/astral-sh/lsp-types.git?rev=3512a9f#3512a9f33eadc5402cfab1b8f7340824c8ca1439"

This was referenced Jul 23, 2025

{tools}[GCCcore/13.3.0] ruff v0.7.0 easybuilders/easybuild-easyconfigs#23466

Merged

switch to CargoPythonBundle easyblock for orjson 3.9.15 easybuilders/easybuild-easyconfigs#21392

Merged

Micket previously approved these changes Aug 3, 2025

View reviewed changes

Micket added the bug fix label Aug 3, 2025

Thyre mentioned this pull request Sep 18, 2025

{tools}[GCCcore/14.3.0] fish v4.0.6 easybuilders/easybuild-easyconfigs#23907

Draft

boegel modified the milestones: 5.x, next release (5.2.0?) Oct 9, 2025

Fix handling of members in crates.io sources

73bf121

Flamefire dismissed Micket’s stale review via 73bf121 October 14, 2025 10:29

Flamefire mentioned this pull request Oct 14, 2025

Avoid test failures due to concurrency issue in modkit easybuilders/easybuild-easyconfigs#24234

Merged

boegel approved these changes Oct 22, 2025

View reviewed changes

boegel merged commit 25ac4a6 into easybuilders:develop Oct 22, 2025
22 checks passed

Flamefire deleted the 20250314170220_new_pr_cargo branch October 22, 2025 08:12

This was referenced Dec 14, 2025

enhance Cargo easyblock to merge Cargo.toml file with workspace file #4016

Merged

cargo EasyBlock may set source directory (and finalpath) incorrectly if sources include subdirs #4021

Closed

Flamefire mentioned this pull request Dec 15, 2025

fix logic in Cargo easyblock to determine value for 'finalpath' after extracting sources #4022

Merged

Unify handling of vendored crates in cargo EasyBlock and add support for non-virtual workspaces #3665

Unify handling of vendored crates in cargo EasyBlock and add support for non-virtual workspaces #3665

Uh oh!

Conversation

Flamefire commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Flamefire commented Mar 14, 2025

Overview of tested easyconfigs (in order)

Uh oh!

lexming commented Mar 15, 2025

Uh oh!

Flamefire commented Mar 15, 2025

Uh oh!

boegel commented Mar 19, 2025

Uh oh!

Flamefire commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview of tested easyconfigs (in order)

Uh oh!

Micket left a comment

Choose a reason for hiding this comment

Uh oh!

boegel commented Oct 10, 2025

Uh oh!

boegelbot commented Oct 10, 2025

Uh oh!

boegelbot commented Oct 11, 2025

Overview of tested easyconfigs (in order)

Uh oh!

boegel commented Oct 11, 2025

Uh oh!

Flamefire commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegel commented Oct 15, 2025

Uh oh!

boegelbot commented Oct 15, 2025

Uh oh!

boegelbot commented Oct 15, 2025

Overview of tested easyconfigs (in order)

Uh oh!

Flamefire commented Oct 16, 2025

Uh oh!

Flamefire commented Oct 16, 2025

Overview of tested easyconfigs (in order)

Uh oh!

boegel commented Oct 16, 2025

Uh oh!

boegelbot commented Oct 16, 2025

Uh oh!

boegelbot commented Oct 16, 2025

Overview of tested easyconfigs (in order)

Uh oh!

boegel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Flamefire commented Mar 14, 2025 •

edited

Loading

Flamefire commented Jul 22, 2025 •

edited

Loading

Flamefire commented Oct 14, 2025 •

edited

Loading