CCPP CMake cleanup, update submodule pointer for stochastic_physics to include emergency bugfix for restart runs

[climbfuji]

PR Checklist

• Ths PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.

• This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR

• An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
  are specified below.

• Results for one or more of the regression tests change and the reasons for the changes are understood and explained below.

• New or updated input data is required by this PR. If checked, please work with the code managers to update input data sets on all platforms.

Description

This PR makes changes in the CMake config files for Intel and GNU that are required by the cleanup of the CCPP cmake config in fv3atm, ccpp-framework, ccpp-physics.

This also fixes a bug in the current build config that adds -no-prec-div -no-pre-sqrt to the CCPP 32bit compile options, although it should only be added to the 64bit compile options. This used to be the case in the past when IPD and CCPP coexisted, but for some unknown reason the bug was introduced since then.

This bug fix changes the results of all 32bit regression tests that use GFDL MP with the saturation adjustment in the dycore. In addition, because the flag -xCORE-AVX2 is removed from the ccpp-physics compiler flags, the preceding -march=core-avx2 applies and the results of most other regression tests change as well. Removing the Intel-chip specific flags is something that was recommended to us because the next operational system wcoss2 does not have Intel chips.

Last minute change 2021/12/13: the submodule pointer for ccpp-physics is updated to include an emergency bugfix for the cellular automata code when the model is warm-started and the CA restart files are not in the input directory (NOAA-PSL/stochastic_physics#50). This does not affect the UFS WM regression tests (verified by running the regression tests with the emergency bugfix included against the newly created baseline that did not have the emergency bugfix).

Issue(s) addressed

Fixes #945

Testing

Regression tests were run on Hera with Intel and GNU during the development. Full regression tests will be run on all tier-1 platforms at the time of commit.

• hera.intel
• hera.gnu
• orion.intel
• cheyenne.intel
• cheyenne.gnu
• gaea.intel
• jet.intel
• wcoss_cray
• wcoss_dell_p3
• opnReqTest for newly added/changed feature
• CI - f826dba

Rerun of regression tests/CI tests against newly created baselines after including the stochastic_physics emergency bugfix:

• hera.intel
• hera.gnu
• orion.intel
• cheyenne.intel
• cheyenne.gnu
• gaea.intel
• jet.intel
• wcoss_cray
• wcoss_dell_p3
• opnReqTest for newly added/changed feature
• CI - 8395e78

Dependencies

• waiting on Major cleanup of cmake build config, contains "Remove Laurie from CODEOWNERS" (#421) NCAR/ccpp-framework#420
• waiting on Major cleanup of cmake build config NCAR/ccpp-physics#792
• waiting on Major cleanup of CCPP cmake build config NOAA-EMC/ufsatm#436

[SamuelTrahanNOAA]

Remember to change the .gitmodules back!

[junwang-noaa]

@climbfuji Will this PR also fix the ufs-weather-model REPRO mode?

[climbfuji]

@climbfuji Will this PR also fix the ufs-weather-model REPRO mode?

Yes, I am confident it does. Here are the REPRO compiler flags for one file in ccpp-physics. Notice that there is no -O3 flag anymore:

cd /scratch1/BMC/gmtb/Dom.Heinzeller/ufs-weather-model/ufs-weather-model-develop-cleanup-cmake-flags-part2/intel_repro/tests/build_fv3/FV3/ccpp/physics && /apps/intel/compilers_and_libraries_2018/linux/mpi/intel64/bin/mpiifort -DFV3 -DINTERNAL_FILE_NML -DMOIST_CAPPA -DMPI -DNEMS_GSM -DREPRO -DUSE_COND -I/scratch1/BMC/gmtb/Dom.Heinzeller/ufs-weather-model/ufs-weather-model-develop-cleanup-cmake-flags-part2/intel_repro/tests/build_fv3/FV3/ccpp/physics -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/w3nco/2.4.1/include_d -I/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/netcdf/4.7.4/include -g -traceback -fpp -fno-alias -auto -safe-cray-ptr -ftz -assume byterecl -nowarn -sox -align array64byte -qno-opt-dynamic-align -real-size 64 -O2 -debug minimal -fp-model consistent -qoverride-limits -O2 -fPIC   -qopenmp -c /scratch1/BMC/gmtb/Dom.Heinzeller/ufs-weather-model/ufs-weather-model-develop-cleanup-cmake-flags-part2/intel_repro/FV3/ccpp/physics/physics/mp_thompson_pre.F90 -o CMakeFiles/ccpp_physics.dir/physics/mp_thompson_pre.F90.o

[climbfuji]

All regression tests passed, also for the rerun after updating the submodule pointer for stochastic_physics to include an emergency bugfix. Will start with merging the CCPP PRs.

[climbfuji]

Hash for fv3atm is correct (a1b0f33), ready to merge.