{bio}[foss/2023a] RELION v5.0.0 w/ CUDA 12.1.1

[jfgrimm]

requires:

• {vis}[GCCcore/12.3.0] napari v0.4.18, Qtconsole v5.5.1, ... #19652
• {ai}[foss/2023a] PyTorch v2.1.2 w/ CUDA 12.1.1 #19666
• {bio,chem}[foss/2023a] ESM-2 v2.0.0, topaz v0.2.5.20231120 w/ CUDA 12.1.1 #19667
• {tools}[foss/2023a] Faiss v1.7.4, t-SNE-CUDA v3.0.1 w/ CUDA 12.1.1 #19669
• demote meson-python to a build dependency in recent matplotlib easyconfigs #19670
• add patch to fix upb CopyFrom error in protobuf-python 4.24.0 #19671
• bump versions of tensorboard v2.15.1 extensions and add reference to protobuf-python patch #19673
• demote poetry to build dependency in easyconfig for expecttest v0.1.5 #19675
• {tools}[GCCcore/12.3.0] Meson v1.3.1, meson-python v0.15.0 #19676
• {tools}[GCCcore/12.3.0] PBZIP2 v1.1.13 #19677
• {vis}[foss/2023a] napari v0.4.19.post1, VisPy v0.14.1 w/ Python v3.11.3 #19664
• {bio}[gompi/2023a, GCCcore/12.3.0] PyHMMER v0.10.6, psutil v5.9.8 #20544

(created using eb --new-pr)

[ThomasHoffmann77]

@jfgrimm I committed AreTomo2 and MotionCor3 (#19681) which could be added to the dependencies

[jfgrimm]

@jfgrimm I committed AreTomo2 and MotionCor3 (#19681) which could be added to the dependencies

@ThomasHoffmann77 thanks, I've added those to the optional section (disabled by default). My current thinking is to default enable things that add functionality, and disable those that replace various relion features (motioncor, resmap, etc)

[ThomasHoffmann77]

@jfgrimm some of the tomography_python_programs require the python package yet-another-imod-wrapper and therefore IMOD.
I am currently working on IMOD/4.12.58-foss-2023a-CUDA-12.1.1

[ThomasHoffmann77]

@jfgrimm some of the tomography_python_programs require the python package yet-another-imod-wrapper and therefore IMOD. I am currently working on IMOD/4.12.58-foss-2023a-CUDA-12.1.1
#19759

[cjroberts]

@jfgrimm I am looking to test this on our systems. Apologies if I am missing something, but I can't seem to find the Seaborn EasyConfig. I think I have included all the dependent PR's and have an up to date copy of the EasyConfigs GitHub repository.

Apptainer> ebskltesting -Dr --from-pr 19652,19666,19667,19669,19678 RELION-5.0.0_beta.2023.12.21-foss-2023a-CUDA-12.1.1.eb                              == Temporary log file in case of crash /tmp/slurm-47121656/eb-hgj4a2wx/easybuild-y5c1xkhp.log
== found valid index for /apps/eb/2020b/skylake/software/EasyBuild/4.9.0/easybuild/easyconfigs, so using it...
ERROR: Missing dependencies: Seaborn/0.13.1-foss-2023a (no easyconfig file or existing module found)

[jfgrimm]

@cjroberts that was indeed missing from my PRs. I've updated the easyconfig to use 0.13.2 which was already merged in #19772

[cjroberts]

Thank you, @jfgrimm . We're having problems building jupyter-lmod as we don't currently use Lmod by default, although we will at some point. This seems to be a dependency of napari. Are the packages under "dependencies from conda environment.yaml file" required or could we comment out napari for our local build for the time being?

[jfgrimm]

@cjroberts I think the easiest would be for you to build a Jupyter-bundle-20230823-GCCcore-12.3.0.eb without the ('jupyterlmod', '4.0.3') dependency. You could do this either by modifying that easyconfig, or creating a hook that automatically removes jupyterlmod from Jupyter-bundle

[jfgrimm]

we have this installed in prod, and our researchers are happy with it; performance is also better than with conda

[ThomasHoffmann77]

will you keep the Napari dependency with version 0.4.18? I required to use 0.4.19 for pyTME (see. #19684, #19664)

[baverhey]

this pr worked for us, RHEL 8- lmod. what needs to be done to get this included? and rerun the tests? we want to port to the 2025 toolset

[akesandgren]

@jfgrimm can you take a look at the complaints from the checks

[jfgrimm]

@akesandgren tests are green now

[akesandgren]

LGTM

[baverhey]

thx guys!

[akesandgren]

Test report by @akesandgren
FAILED
Build succeeded for 21 out of 22 (total: 1 hour 52 mins 20 secs) (1 easyconfigs in total)
b-cn1611.hpc2n.umu.se - Linux Ubuntu 22.04, x86_64, AMD EPYC 7313 16-Core Processor, 1 x NVIDIA NVIDIA A100 80GB PCIe, 580.95.05, Python 3.10.12
See https://gist.github.com/akesandgren/ae7b04d200b51fcf1d973098997858d2 for a full test report.

@jfgrimm
Cause:
poetry 1.5.1 has requirement build<0.11.0,>=0.10.0, but you have build 1.1.1.

[jfgrimm]

@akesandgren you might need to rebuild #19675

[akesandgren]

Test report by @akesandgren
SUCCESS
Build succeeded for 1 out of 1 (total: 31 mins 54 secs) (1 easyconfigs in total)
b-cn1611.hpc2n.umu.se - Linux Ubuntu 22.04, x86_64, AMD EPYC 7313 16-Core Processor, 1 x NVIDIA NVIDIA A100 80GB PCIe, 580.95.05, Python 3.10.12
See https://gist.github.com/akesandgren/658759dc4ea096d276061fec655a2ed7 for a full test report.

After rebuilding expecttest

[akesandgren]

@boegelbot Please test @ jsc-zen3

[boegelbot]

@akesandgren: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=19678 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_19678 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 9258

Test results coming soon (I hope)...

Details

- notification for comment with ID 3674987614 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[jfgrimm]

@akesandgren jsc bot nodes are currently down, see slack thread in test-bots

[boegelbot]

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (total: 28 mins 32 secs) (1 easyconfigs in total)
jsczen3c4.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/d5e4d601012ee4d2f9605c4c4ab9bc0a for a full test report.

[akesandgren]

Going in, thanks @jfgrimm!