Modify Met Office regtest matrix to run regtests on the new EX machine

[ukmo-juan-castillo]

Pull Request Summary

New regtest matrix for the new Met Office EX machine

Description

The Met Office has a new supercomputer called EX, while the old CrayXC40 is going to be retired in less than two weeks. To allow running regtests on the new machines, the regtest matrix files for the Met Office need updating.

Suggested reviewer: ukmo-kitstokes, mickaelaccensi
Suggested label to be added: new_feature
No changed of answers is expected, as this is a technical and not a scientific change.

Issue(s) addressed

• fixes Regtest matrix for new Met Office EX machine #1447

Commit Message

New regtest matrix for the Met Office EX supercomputer

Check list

• [ x] Branch is up to date with the authoritative repository (NOAA-EMC) develop branch.
• [ x] Checked the checklist for a developer submitting to develop.
• [ x] If a version number update is required, checked the updating version number checklist.
• [ x] If a new feature was added, a regression test for testing the new feature is added.

Testing

• How were these changes tested?
  A full regtest was run on the Met Office EX supercomputer. Please notice that the intel compiler is not available on EX.
• Are the changes covered by regression tests? (If not, why? Do new tests need to be added?)
  Yes
• Have the matrix regression tests been run (if yes, please note HPC and compiler)?
  Yes
• Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.)
  No changes expected, as this is a technical change to run on a new machine
• Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt):

[ukmo-juan-castillo]

How can we add ukmo-kitstokes to the list of available reviewers? It might be a good idea if he checked he can also run regtests with these changes.

[mickaelaccensi]

How can we add ukmo-kitstokes to the list of available reviewers? It might be a good idea if he checked he can also run regtests with these changes.

maybe @JessicaMeixner-NOAA knows how to add new reviewers ?

[JessicaMeixner-NOAA]

@ukmo-kitstokes I have sent you an invitation for this repo so that I can then add you as a reviewer.

@ukmo-juan-castillo - I will wait to process this PR until I either see a comment approving or an official review from Kit.

[ukmo-ccbunney]

Just a few comments from me that I will leave to your discretion as to whether you want to action them.

[ukmo-ccbunney]

In my original tests, I didn't use the parallel netcdf/hdf5 versions (they're not required for the netcdf pre/post-processors). I was using these non-parallel versions (which are slightly newer versions):

module load cray-hdf5/1.12.2.3
module load cray-netcdf/4.9.0.3

Perhaps it's fine to use the parallel versions? But it seems unnecessary. I'll leave that decision up to you.

[ukmo-juan-castillo]

I think it does not matter much, I just added the default libraries for running global and regional coupled configurations at the Met Office.

[ukmo-ccbunney]

One other thing that I had to add to the cmake build were the following lines:

# For CMAKE
echo "export CC=cc"             >> matrix.head
echo "export FTN=ftn"           >> matrix.head

This was because cmake would quietly go and find the crayftn compiler rather than using the proper wrapper scripts. This caused me linker problems originally as the ftn wrappers script sets the library paths.

[ukmo-juan-castillo]

Thank you, I will do so. Is there any documentation on how to compile ParMETIS? I have some test failing and I think it is because I just copied the files from the XC40s, and there are library versions incompatibility...

[ukmo-ccbunney]

You might want to focus on getting SCOTCH compiled instead? IIRC ParMETIS was going to be replaced by SCOTCH.
WE could not compile scotch on our old HPC due to the libraries being too old. I think it should be fine on the new HPC though. I'll ping you some instructions regarding ParMETIS.

[ukmo-ccbunney]

LGTM! 👍

[JessicaMeixner-NOAA]

@ukmo-ccbunney instructions are probably better for your machines but here is a link for help with building parmetis: https://github.com/NOAA-EMC/WW3/wiki/FAQs-page#how-to-install-Metis-and-Parmetis
and a link for building SCOTCH: https://github.com/NOAA-EMC/WW3/wiki/FAQs-page#how-to-install-SCOTCH

For now we have both parmetis and scotch in WW3, although at EMC we use SCOTCH in operations, not parMETIS.

[JessicaMeixner-NOAA]

@ukmo-juan-castillo - I just added @ukmo-kitstokes as a reviewer. Let me know if you want me to wait or if you're ready for this to be merged.

[ukmo-juan-castillo]

@ukmo-kitstokes I have sent you an invitation for this repo so that I can then add you as a reviewer.

@ukmo-juan-castillo - I will wait to process this PR until I either see a comment approving or an official review from Kit.

Thank you, I will leave it in draft state until I compile parmetis and scotch, and I have a full list of successful tests.

[ukmo-kitstokes]

I don't yet have sufficient experience with compiling on our HPC to provide useful comments, but I have reviewed the changeset for my learning experience. All looks sensible to my untrained eye. Thanks again to Chris for supporting in this PR.

[ukmo-juan-castillo]

I maganed to compile scotch and parmetis with both the Cray and the GNU compilers. I run all the GNU tests successfully on the EX machines at the Met Office. I attach the matrix.out file. The pull request is ready for review.
matrix.out.tar.gz