{numlib,compiler,toolchain,mpi}[lfoss/2023b] Added all ECs required to compile up to the lfoss and lmpflf toolchains#23051
{numlib,compiler,toolchain,mpi}[lfoss/2023b] Added all ECs required to compile up to the lfoss and lmpflf toolchains#23051Crivella wants to merge 16 commits intoeasybuilders:developfrom
lfoss and lmpflf toolchains#23051Conversation
Updated software
|
|
Test report by @Crivella NotesTest performed with the branch from easybuilders/easybuild-framework#4914 checked out |
Manual reportBuild using EESSI + EESSI-Extend (with LLVM-20.1.5 coming from one of the builds performed in easybuilders/easybuild-easyblocks#3741) Uploading only one .md (to show the environment) as i forgot The file https://gist.github.com/Crivella/4226f6bbc4199f6b96ced3d3ff840be3 Final results |
|
New unified report for the same run as in #23051 (comment) https://gist.github.com/Crivella/68eb305fa15e9f471146023b43001106 |
Crivella
changed the title
lfoss toolchain WIPlfoss toolchain
|
Test report by @Crivella |
easybuild/easyconfigs/o/OpenBLAS/OpenBLAS-0.3.29-LLVMtc-20.1.5.eb
Outdated
Show resolved
Hide resolved
|
fyi, currently testing UPDATE: actually, that fails installing psutils with |
Yeah i've to sync that branch with develop will do it ASAP |
|
I've rebased all the LLVM-toolchain PRs on develop |
|
has anyone tested this on Arm? On a64fx, the OpenBLAS tests are failing with If it's specific to a64fx we'll deal with it locally ad Deucalion, but I'd say it's important to test on more generic Arm |
|
There seems to have been a good bit of work recently related to A64FX, can we bump the OpenBLAS version to 0.3.30? We probably also want this patch: OpenMathLib/OpenBLAS#5240 |
|
also, and this seems to apply to any Arm with SVE, when building BLIS with LLVM I got had to add (and then also the a64fx specific |
|
I think I've had a similar problem ( I was considering to put it as a default option in the TC, but might be too agressive... |
Could we make this a toolchain option, similar to |
Yeah most likely, especially as I am doing the same for |
indeed, with 0.3.30 there's no segmentation fault later there's one (out of 126) failed test but it doesn't seem to be as serious |
|
update: at Deucalion we built GROMACS for a64fx using this toolchain, and it's ~20x (yes, 20 times!) slower than any other build. But then I changed only the LLVM version to 19, and it's 1.2x faster than with GCC (which we had already observed and motived our interest in this toolchain) Can anyone think of what could bring such a severe (absurd, actually) performance regression on LLVM 20? I don't see any differences when profiling with I'll be running more tests to see if it also happens on non-a64fx arm chips and on x86 |
So basically rewriting everything to work on top of |
Quickly glancing through the LLVM issues, there were a few reports about performance regressions, which seem to be fixed in 20.1.6 or later. |
I created an LLVM-19.1.7-GCCcore-13.2.0.eb easyconfig to keep the GCC toolchain version, and added a |
Thanks, will also try that
Indeed. At this point I'm assuming it's a64fx specific, but even then it's weird. Will try to gather more information |
Crivella
changed the title
lfoss toolchainlfoss and lmpflf toolchains
|
Test report by @Crivella |
|
Test report by @Crivella |
|
Test report by @Crivella NOTESthis was tested with easybuilders/easybuild-framework@31a27c6 to ensure we are also using rpath-wrappers for the LLVM linkers. As far as i can tell, |
|
Test report by @Crivella |
|
NOTE: forgot the EB Test report by @Crivella |
|
Tested with FW PR at easybuilders/easybuild-framework@232ef0b Test report by @Crivella |
|
Test report by @boegel |
|
@boegelbot please test @ jsc-zen3 |
|
@Crivella: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de PR test command '
Test results coming soon (I hope)... Details- notification for comment with ID 3621284310 processed Message to humans: this is just bookkeeping information for me, |
Signed-off-by: Jan Andre Reuter <jan@zyten.de>
|
Test report by @boegelbot |
| gcc_ver = tc_gen.split('-', 1)[1].rstrip('.') | ||
| # Either map subtoolchain to known toolchain name, or label as unknown_map | ||
| # This will preserve the old behavior by grouping all unknown toolchains together | ||
| tc_family = tc_maps.get(tc_name, 'unknown_map') |
There was a problem hiding this comment.
What about GCCcore? Or just tc_maps.get(tc_name, tc_name) to use that directly as the family?
There was a problem hiding this comment.
I want to use a common string for everything not in the tc_map dict so we revert to the old behavior when the toolchain type is not identified
There was a problem hiding this comment.
Then I'd rather use either None: tc_maps.get(tc_name) and filter that out in the message, or give it a better generic name. Otherwise the error message would be confusing:
Found 3 variants of Foo in easyconfigs using 'unknown_map' toolchain '2024a' generation
User might read the "unknown" as "error happened leading to 'unknown'" while this is not unknown but expected, e.g. for GCCcore.
What would we even have in here? Maybe require all to be present and use tc_maps[tc_name] and we decide where we put it if and when it fails?
| for tc_name in sorted(all_deps.keys()): | ||
| ptr1 = all_deps[tc_name] |
There was a problem hiding this comment.
ptr isn't a great name. And you could do for tc_name, tc_gen_map in sorted(all_deps.items()):
|
Test report by @boegel |
|
Closing this in favor of moving all efforts for the LLVM toolchain in the 2025b and newer |
8 participants
New Easyconfigs to test the proposed LLVM based toolchain in
Requires also Easyblock PR: