{numlib}[foss/2022.05] HPL v2.3 (incl. FFTW.MPI v3.3.10 + ScaLAPACK v2.2.0) by bartoldeman · Pull Request #15561 · easybuilders/easybuild-easyconfigs

bartoldeman · 2022-05-29T18:13:45Z

(created using eb --new-pr)

requires:

add custom easyblock for FFTW.MPI + modify FFTW's sanity check step to allow checking only for MPI parts of FFTW installation easybuild-easyblocks#2724

…I-3.3.10-gompi-2022.05.eb, ScaLAPACK-2.2.0-gompi-2022.05-fb.eb and patches: ScaLAPACK-2.2.0_fix-GCC-10.patch

bartoldeman · 2022-05-29T18:36:21Z

@boegel some comments on FFTW.MPI.

This is doing everything in the easyconfig, but some of it could be moved to the easyblock.
Do you prefer in that case a seperate EB_FFTW_period_MPI easyblock, or use if statements in EB_FFTW?

The way it works is that it builds serial and MPI components (since in the build MPI depends on serial) but not threads/openmpi, and no quadruple since quadruple precision isn't supported by MPI.

Then it removes almost everything except for the MPI libraries (libfftw3*_mpi*) and all headers, i.e.:

total 205
-rw-r--r-- 1 oldeman oldeman  2447 May 25 14:44 fftw3.f
-rw-r--r-- 1 oldeman oldeman 55032 May 25 14:44 fftw3.f03
-rw-r--r-- 1 oldeman oldeman 31986 May 25 14:44 fftw3.h
-rw-r--r-- 1 oldeman oldeman 27203 May 25 14:44 fftw3l.f03
-rw-r--r-- 1 oldeman oldeman 18678 May 25 14:44 fftw3l-mpi.f03
-rw-r--r-- 1 oldeman oldeman 36969 May 25 14:44 fftw3-mpi.f03
-rw-r--r-- 1 oldeman oldeman  9624 May 25 14:44 fftw3-mpi.h
-rw-r--r-- 1 oldeman oldeman 25902 May 25 14:44 fftw3q.f03

vs

$ ls -l $EBROOTFFTW/include
total 141
-rw-r--r-- 1 oldeman oldeman  2447 May 11 14:27 fftw3.f
-rw-r--r-- 1 oldeman oldeman 55032 May 11 14:27 fftw3.f03
-rw-r--r-- 1 oldeman oldeman 31986 May 11 14:27 fftw3.h
-rw-r--r-- 1 oldeman oldeman 27203 May 11 14:27 fftw3l.f03
-rw-r--r-- 1 oldeman oldeman 25902 May 11 14:27 fftw3q.f03

Why keep all headers thereby duplicating some headers in the serial MPI, but don't do the same for libraries you ask?

Well first of all libraries are needed both at runtime and compile time, even if the libraries are 100% identical it's confusing to have them in two different paths.

For headers this is not as important since it's compile time only, and secondly, having all the headers under a single include path simplifies some cases, e.g. FFT_INC_DIR can only be a single path.

Framework PR easybuilders/easybuild-framework#4012 sets FFT_INC_DIR to the MPI path only if MPI is used. Now the FFTW (non-MPI) module will also set the non-MPI path in CPATH, but this only helps C, not Fortran.

During link time the non-MPI path is in LIBRARY_PATH so the library -lfftw will be found anyway, unless the build procedure uses special tricks (which should be rare and easily worked around).

bartoldeman · 2022-05-30T01:53:03Z

Now depends on easybuilders/easybuild-easyblocks#2724 for custom easyblock for FFTW.MPI

bartoldeman · 2022-05-30T01:54:34Z

@boegelbot Please test @ generoso
EB_ARGS='--include-easyblocks-from-pr 2724'

boegelbot · 2022-05-30T01:55:08Z

@bartoldeman: Request for testing this PR well received on login1

PR test command 'EB_PR=15561 EB_ARGS="--include-easyblocks-from-pr 2724" /opt/software/slurm/bin/sbatch --job-name test_PR_15561 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 8585

Test results coming soon (I hope)...

Details

- notification for comment with ID 1140600277 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2022-05-30T02:17:42Z

Test report by @boegelbot
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#2724
SUCCESS
Build succeeded for 5 out of 5 (4 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/aaeb146ad828843572f9e0604691ecbc for a full test report.

SebastianAchilles · 2022-05-30T08:15:21Z

@boegelbot please test @ jsc-zen2
EB_ARGS='--include-easyblocks-from-pr 2724'

boegelbot · 2022-05-30T08:17:09Z

@SebastianAchilles: Request for testing this PR well received on jsczen2l1.int.jsc-zen2.easybuild-test.cluster

PR test command 'EB_PR=15561 EB_ARGS="--include-easyblocks-from-pr 2724" /opt/software/slurm/bin/sbatch --job-name test_PR_15561 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen2.sh' executed!

exit code: 0
output:

Submitted batch job 1242

Test results coming soon (I hope)...

Details

- notification for comment with ID 1140845566 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2022-05-30T08:31:46Z

Test report by @boegelbot
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#2724
SUCCESS
Build succeeded for 5 out of 5 (4 easyconfigs in total)
jsczen2c1.int.jsc-zen2.easybuild-test.cluster - Linux Rocky Linux 8.5, x86_64, AMD EPYC 7742 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/17f7471cb06034a42fd77df624f5687b for a full test report.

SebastianAchilles · 2022-05-30T09:13:18Z

Test report by @SebastianAchilles
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#2724
SUCCESS
Build succeeded for 4 out of 4 (4 easyconfigs in total)
zen2-ubuntu-eb - Linux Ubuntu 22.04, x86_64, AMD EPYC 7452 32-Core Processor (zen2), Python 3.10.4
See https://gist.github.com/04968d8e4643818460b104e6c3dc9289 for a full test report.

jfgrimm · 2022-05-30T09:40:10Z

Test report by @jfgrimm
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#2724
SUCCESS
Build succeeded for 4 out of 4 (4 easyconfigs in total)
node011.pri.viking.alces.network - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/da0bfc4289270357161658c49eeb2819 for a full test report.

…ompi/2022.05

boegel · 2022-06-04T13:40:03Z

Test report by @boegel
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#2724
SUCCESS
Build succeeded for 4 out of 4 (4 easyconfigs in total)
node3502.doduo.os - Linux RHEL 8.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/1b957895d99047b7371fa29d58868dc2 for a full test report.

boegel · 2022-06-04T13:51:43Z

Test report by @boegel
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#2724
SUCCESS
Build succeeded for 4 out of 4 (4 easyconfigs in total)
node3108.skitty.os - Linux RHEL 8.4, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/74aa527972a80682c88c43af01bef324 for a full test report.

boegel · 2022-06-04T14:16:35Z

@boegelbot Please test @ generoso
EB_ARGS='--include-easyblocks-from-pr 2724'

boegelbot · 2022-06-04T14:35:08Z

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=15561 EB_ARGS="--include-easyblocks-from-pr 2724" /opt/software/slurm/bin/sbatch --job-name test_PR_15561 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 8651

Test results coming soon (I hope)...

Details

- notification for comment with ID 1146621145 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2022-06-04T14:51:06Z

Test report by @boegelbot
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#2724
SUCCESS
Build succeeded for 4 out of 4 (4 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/fc4b3f859a32ce4196f412d0bcb9cdb9 for a full test report.

boegel

lgtm

boegel · 2022-06-04T15:31:12Z

Going in, thanks @bartoldeman!

adding easyconfigs: HPL-2.3-foss-2022.05.eb, foss-2022.05.eb, FFTW.MP…

e4ebe15

…I-3.3.10-gompi-2022.05.eb, ScaLAPACK-2.2.0-gompi-2022.05-fb.eb and patches: ScaLAPACK-2.2.0_fix-GCC-10.patch

bartoldeman added new update labels May 29, 2022

bartoldeman requested a review from boegel May 29, 2022 18:36

Use special easyblock for FFTW.MPI

d01fc49

jfgrimm mentioned this pull request May 30, 2022

{lang}[foss/2022.05] SciPy-bundle v2022.05 w/ Python 3.10.4 #15563

Merged

1 task

boegel added this to the next release (4.5.5?) milestone May 30, 2022

jfgrimm mentioned this pull request May 31, 2022

{math}[foss/2022.05] Hypre v2.24.0, METIS v5.1.0, MPFR v4.1.0, ... w/ Python 3.10.4 #15574

Closed

3 tasks

bartoldeman and others added 2 commits June 3, 2022 11:55

Bump FlexiBLAS to 3.2.0

6542dc9

also bump to FlexiBLAS 3.2.0 in easyconfig for ScaLAPACK 2.2.0 with g…

68b3864

…ompi/2022.05

easybuilders deleted a comment from boegelbot Jun 4, 2022

boegel changed the title ~~{numlib}[gompi/2022.05] HPL v2.3, foss v2022.05, FFTW.MPI v3.3.10, ...~~ {numlib}[foss/2022.05] HPL v2.3 (incl. FFTW.MPI v3.3.10 + ScaLAPACK v2.2.0) Jun 4, 2022

easybuilders deleted a comment from boegelbot Jun 4, 2022

boegel approved these changes Jun 4, 2022

View reviewed changes

boegel merged commit 3ce2212 into easybuilders:develop Jun 4, 2022

Conversation

bartoldeman commented May 29, 2022 • edited by boegel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bartoldeman commented May 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bartoldeman commented May 30, 2022

Uh oh!

bartoldeman commented May 30, 2022

Uh oh!

boegelbot commented May 30, 2022

Uh oh!

boegelbot commented May 30, 2022

Uh oh!

SebastianAchilles commented May 30, 2022

Uh oh!

boegelbot commented May 30, 2022

Uh oh!

boegelbot commented May 30, 2022

Uh oh!

SebastianAchilles commented May 30, 2022

Uh oh!

jfgrimm commented May 30, 2022

Uh oh!

boegel commented Jun 4, 2022

Uh oh!

boegel commented Jun 4, 2022

Uh oh!

boegel commented Jun 4, 2022

Uh oh!

boegelbot commented Jun 4, 2022

Uh oh!

boegelbot commented Jun 4, 2022

Uh oh!

boegel left a comment

Choose a reason for hiding this comment

Uh oh!

boegel commented Jun 4, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bartoldeman commented May 29, 2022 •

edited by boegel

Loading

bartoldeman commented May 29, 2022 •

edited

Loading