enhance LAMMPS easyblock to use FFT_KOKKOS configure option#3784
enhance LAMMPS easyblock to use FFT_KOKKOS configure option#3784laraPPr merged 11 commits intoeasybuilders:developfrom
FFT_KOKKOS configure option#3784Conversation
From v `29Aug2024` it is possible to specify the FFT library used in Kokkos by the Cmake setting `FFT_KOKKOS`. LAMMPS is shipped with the internal KISS FFT library, which is selected by default. However, for CUDA builds, messages appear at runtime advising the user to use `cuFFT` for better performance. For CPU-only builds, it is also advised to use FFTW or MKL, if available.
|
This should be thoroughly tested to make sure we don't run into surprises... @laraPPr Are you up for taking a look at this? |
|
Yes I'll tackle this |
|
@lorisercole would you mind if I snuck this one as well in your pr? https://github.com/laraPPr/easybuild-easyblocks/blob/a23b7f654efc76f6afa72ff215fc0984ea165128/easybuild/easyblocks/l/lammps.py#L363-L366 |
Sure, feel free to push it! |
Signed-off-by: laraPPr <lara.peeters@ugent.be>
|
Test report by @laraPPr Overview of tested easyconfigs (in order)
Build succeeded for 21 out of 24 (4 easyconfigs in total) |
|
Hi @laraPPr, I actually have a greater update about using Kokkos that I'd like to include in this PR. I discovered that with the Since version |
This reverts commit 00f237b. It is not needed as Easybuild's wrappers already take care or using the correct rpath-ing flags. LAMMPS Cmake option is not useful and does not work properly.
|
I finished testing this PR, now I'm done. |
|
@boegelbot please test @ jsc-zen3 EB_ARGS="LAMMPS-2Aug2023_update2-foss-2023a-kokkos.eb LAMMPS-23Jun2022-foss-2021a-kokkos.eb LAMMPS-23Jun2022-foss-2021b-kokkos.eb LAMMPS-23Jun2022-foss-2022a-kokkos.eb LAMMPS-29Aug2024-foss-2023b-kokkos.eb LAMMPS-29Aug2024_update2-foss-2023a-kokkos.eb LAMMPS-29Aug2024_update2-foss-2024a-kokkos.eb" |
|
@laraPPr: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de PR test command '
Test results coming soon (I hope)... Details- notification for comment with ID 3187765122 processed Message to humans: this is just bookkeeping information for me, |
|
Test report by @laraPPr Overview of tested easyconfigs (in order)
Build succeeded for 3 out of 4 (4 easyconfigs in total) |
|
LAMMPS-29Aug2024_update2-foss-2024a-kokkos.eb failed because of missing dependencies. restarted it now but should be ok |
|
Test report by @boegelbot Overview of tested easyconfigs (in order)
Build succeeded for 15 out of 15 (7 easyconfigs in total) |
|
@boegelbot please test @ jsc-zen3-a100 EB_ARGS="LAMMPS-2Aug2023_update2-foss-2023a-kokkos-CUDA-12.1.1.eb LAMMPS-2Aug2023_update2-foss-2024a-kokkos-CUDA-12.6.0.eb LAMMPS-23Jun2022-foss-2021a-kokkos-CUDA-11.3.1.eb LAMMPS-23Jun2022-foss-2021b-kokkos-CUDA-11.4.1.eb LAMMPS-28Oct2024-foss-2023a-kokkos-mace-CUDA-12.1.1.eb LAMMPS-29Aug2024_update2-foss-2023b-kokkos-CUDA-12.4.0.eb LAMMPS-29Aug2024_update2-foss-2024a-kokkos-CUDA-12.6.0.eb" |
|
@laraPPr: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de PR test command '
Test results coming soon (I hope)... Details- notification for comment with ID 3188426479 processed Message to humans: this is just bookkeeping information for me, |
|
Test report by @boegelbot Overview of tested easyconfigs (in order)
Build succeeded for 3 out of 8 (7 easyconfigs in total) |
|
Test report by @laraPPr Overview of tested easyconfigs (in order)
Build succeeded for 2 out of 2 (1 easyconfigs in total) |
|
CPU builds seem ok but the CUDA builds ran in all kinds of trouble. Am now looking at LAMMPS-2Aug2023_update2-foss-2024a-kokkos-CUDA-12.6.0.eb |
|
I wonder, do these testing nodes have CUDA drivers installed? Do you normally use them to build CUDA software? |
|
Yes it was build on the test system at JSC (jsc-zen3-a100) where we test all GPU builds. So it should be. |
|
Test report by @laraPPr Overview of tested easyconfigs (in order)
Build succeeded for 1 out of 1 (1 easyconfigs in total) |
|
The boegelbot results look weird. See e.g.: easybuilders/easybuild-easyconfigs#23653 (comment) I think you managed to find a bug here. |
|
I'm gonna ignore the failure of LAMMPS-28Oct2024-foss-2023a-kokkos-mace-CUDA-12.1.1.eb this uses the nvcc-wrapper so is failing with missing RPATH as well. Perhaps this will also be solved by easybuilders/easybuild-framework#4981 but I don't think we should delay this pr on that one. |
|
Ok let me retrigger this one to see if it submits to the right system |
I've pinged Kenneth and Sebastian on this yesterday evening. This requires a small change in the shell script used for it. |
|
@laraPPr: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de PR test command '
Test results coming soon (I hope)... Details- notification for comment with ID 3199602034 processed Message to humans: this is just bookkeeping information for me, |
|
Ok will test on our clusters in Gent in the mean time |
|
Test report by @boegelbot Overview of tested easyconfigs (in order)
Build succeeded for 0 out of 1 (1 easyconfigs in total) |
|
Test report by @laraPPr Overview of tested easyconfigs (in order)
Build succeeded for 4 out of 4 (4 easyconfigs in total) |
|
Test report by @laraPPr Overview of tested easyconfigs (in order)
Build succeeded for 4 out of 4 (4 easyconfigs in total) |
|
Test report by @laraPPr Overview of tested easyconfigs (in order)
Build succeeded for 1 out of 1 (1 easyconfigs in total) |
|
Going in, thanks @lorisercole! |
|
@boegelbot please test @ jsc-zen3-a100 EB_ARGS="LAMMPS-2Aug2023_update2-foss-2024a-kokkos-CUDA-12.6.0.eb" |
|
@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de PR test command '
Test results coming soon (I hope)... Details- notification for comment with ID 3207096727 processed Message to humans: this is just bookkeeping information for me, |
|
Test report by @boegelbot Overview of tested easyconfigs (in order)
Build succeeded for 0 out of 1 (1 easyconfigs in total) |
FFT_KOKKOS configure option
From version
29Aug2024it is possible to specify the FFT library used in Kokkos by the Cmake settingFFT_KOKKOS.LAMMPS is shipped with the internal KISS FFT library, which is selected by default.
However, for CUDA builds, messages appear at runtime advising the user to use
cuFFTfor better performance.For CPU-only builds, it is also advised to use FFTW or MKL, if available.