Intel oneAPI LLVM Compilers Configuration

[islas]

Addition of compile configuration using the new Intel LLVM compilers ifx and icx for Fortran and C code, respectively

TYPE: enhancement

KEYWORDS: Intel, LLVM, oneAPI, compilation

SOURCE: internal

DESCRIPTION OF CHANGES:
Problem:
Addressing issue #1884 for Intel oneAPI ifx/icx builds

Solution:
Add configuration to configure.defaults mirroring original Intel ifort/icc build with minor tweaking

ISSUE: For use when this PR closes an issue.
Fixes #1884

LIST OF MODIFIED FILES:
M arch/configure.defaults

TESTS CONDUCTED:

1. WRF Core em_real compiles with ifx/icx (see attached log)

RELEASE NOTE:
Added build configuration for new Intel oneAPI LLVM ifx/icx compilers, which will be available on NCAR's new computer, Derecho.

build_llvm.log

[islas]

@HathewayWill I believe the guidance provided in the Intel forum by the moderator was to address the specific issues of the bash export command not taking spaced arguments without quotes. Using the same methodology other WRF configurations use does yield MPI using the new Intel compilers. A quick peek into the executable produced outputs:

strings main/wrf.exe | grep -i oneapi
Intel(R) oneAPI DPC++/C++ Compiler 2022.0.0 (2022.0.0.20211123)

[weiwangncar]

Regression test results:

Test Type              | Expected  | Received |  Failed
= = = = = = = = = = = = = = = = = = = = = = = =  = = = =
Number of Tests        : 23           24
Number of Builds       : 60           57
Number of Simulations  : 158           150        0
Number of Comparisons  : 95           86        0

Failed Simulations are: 
None
Which comparisons are not bit-for-bit: 
None

[islas]

@mgduda The primary differences in flags between spack/spack#38607 and this aside from architecture specific flags is the use of -flto [-fuse-ld=lld]. -flto is in lieu of -ip/-ipo but requires usage of lld, which is not necessary or guaranteed to be alongside ifx/icx (e.g. does not exist on cheyenne)

Though to get the full benefits of the LLVM compilation, Intel recommends using -flto. I'm inclined to go with not using -flto if it guarantees having less linking issues for users.

[weiwangncar]

@islas @mgduda Should we consider this for 4.5.1 since we will have access to Derecho soon?

[mgduda]

@islas @mgduda Should we consider this for 4.5.1 since we will have access to Derecho soon?

I think this would be good to include.

[HathewayWill]

@HathewayWill I believe the guidance provided in the Intel forum by the moderator was to address the specific issues of the bash export command not taking spaced arguments without quotes. Using the same methodology other WRF configurations use does yield MPI using the new Intel compilers. A quick peek into the executable produced outputs:

strings main/wrf.exe | grep -i oneapi
Intel(R) oneAPI DPC++/C++ Compiler 2022.0.0 (2022.0.0.20211123)

@islas @weiwangncar

How would you use Intel compilers mentioned in the configure.wrf guide with the new llvm compilers

#        If you have Intel MPI installed and wish to use instead, make the
#        following changes to settings below:
#        DM_FC  = mpiifort
#        DM_CC  = mpiicc
#        and source bin64/mpivars.sh file from your Intel MPI installation
#        before the build.

if DM_FC nedds to be mpiifort -fc=ifx and DM_CC needs to be mpiicc -cc=icx?

I haven't gotten that to work. Have you? If so how did you do it?

[weiwangncar]

@islas What modules does one need to use this option on Derecho?

[islas]

@islas What modules does one need to use this option on Derecho?

This will not build on Derecho as it currently is. It will build on Cheyenne with the intel/2022.1 and netcdf modules. Derecho only has later versions that then enforce ISO C99 requirements.
Assuming merged #1823, intel or intel-oneapi (synonymous) and netcdf on Derecho should work. I've tested the latest versions on a separate system and the module on Derecho in an interactive job*

* I'm having difficulty compiling [with a limited number of make -j jobs down to just -j 1] on the login node, I suspect this is due to memory issues when compiling very large files. This problem is not evident in my other tests