MPAS dycore + Atmosphere Generalization

[dustinswales]

Description

This pull request introduces the dynamical core from the Model for Prediction Across Scales (MPAS) and contains changes to generalize atmospheric component of the UFS Weather Model to accomodate multiple dynamical cores.

This PR supersedes #873

Additionally, a common thread woven through these changes is the migration from the fv3atm naming convention to using ufsatm-fv3 or ufsatm-mpas

1. Build system changes:

• Refactor the CMakeLists to build an atmospheric component with either an FV3 or MPAS dynamical core, coupled to the CCPP Physics. Also, add ufsatm-mpas build to CmakeLists.txt
• Introduce a new submodule for the MPAS dynamical core, mpas/MPAS-Model.
• Add CMakeLists.txt to build MPAS core_atmosphere component. The CMake recipes from the "MPAS atmospheric core" were adapted to work in the UFS build system.
• Modify CCPP physics CmakeLists.txt to build independent MPAS/FV3 couplings to the physics.

2. Generalization (physics):

• Partition ccpp/CCPP_Driver.F90 to work with both FV3/MPAS physics-to-dynamics coupling strategies.
• Allow ccpp/CMakeLists.txt to build with either MPAS or FV3 physics. Note This may not be necessary once all the physics are hooked up with MPAS.
• Create FV3 and MPAS specific ccpp/config/ccpp_prebuild_config.py files. Note This may not be necessary once all the physics are hooked up with MPAS.
• Began partitioning ccpp/data/GFS_typedefs.F90 for FV3 and MPAS. Working through initialization (e.g physics namelist ingestion)
• Create ccpp/driver/MPAS_init.F90 from ccpp/driver/GFS_init.F90

3. Generalization (atmosphere)

• Create common module for code to be shared across dycores in the UWM, ufsatm_util.F90. Initially, a single routine is shared by the MPAS and FV3 atmospheric drivers, needed to set up the tracers/constituents/scalars/WhateverYourHostCallsThese.
• Move io/ out from subdirectory of fv3/. Any dycore specific pieces with be distinguished when we generalize the write-grid component for MPAS and FV3.

4. Added initial CCPP suite definition file for MPAS enabled RRFS. Only RRTMG initialization working.

5. New typedefs modules for MPAS. Note This may be merged with GFS_typedefs.F90 in the future.

6. Create mpas coupling infrastructure for the UWM. This includes ESMF hooks for the MPAS atmospheric driver and unifying the NUOPC cap. ATTN: @danrosen25

• Create MPAS based atmospheric driver for coupling the dynamical core with the CCPP Physics, mpas/atmos_model.F90
• Create ESMF forecast grid component with appropriate hooks to the atmospheric driver, mpas/module_fcst_grid_comp.F90
• Rename and generalization NUOPC Cap from fv3atm_cap.F90 to ufsatm_cap.F90.
• Add Host controlled Macro to NUOPC Cap to use forecast grid component for selected dynamical core, ATMFV3 and ATMMPAS.

7. Create initial physics-to-dynamics coupling infrastructure, mpas/atmos_coupling.F90

• Define data containers for input/export state between the MPAS dycore and CCPP Physics.
• Create initial routines to translate to-and-from the MPAS and CCPP state variables.

With these changes, the MPAS dynamical core builds and runs within the UWM, albeit it's not producing valid forecasts and much work remains to achieve this

There are no changes to any existing tests.

Issue(s) addressed

N/A

Testing

Tested on Hera.
All existing UWM RTs pass.
No changes to baselines.
New UWM RTs for MPAS.

Dependencies

N/A

[dustinswales]

How is ESMF handled in MPAS in this version?

By default, MPAS uses its own, stripped-down version of ESMF that provides just the clock routines. But you can tell MPAS to use an external ESMF instead, which makes sense in the context of the UFS that requires and configures its use in the CMake build system.

We are using the UFS (external) ESMF in MPAS. In mpas/CmakLists.txt it reverts back to MPAS's internal ESMF if one is not provided by the host.

[dustinswales]

@DusanJovic-NOAA @grantfirl
Thanks for the feedback.
I did a "CMake macro cleanup" and switched to using an integer identifier for the selected dycore.

[DusanJovic-NOAA]

@dustinswales I started seeing these warnings about deprecated cmake version from one of the CMakeLists.txt recently:

CMake Deprecation Warning at FV3/ccpp/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

Since you are already updating various CMakeLists.txt files in this PR, can you please update the version to, for example, 3.19 which is the version currently required in the top-level CMakeLists.txt.

[climbfuji]

This looks good to me. I'll wait for the conflict resolution and regression testing before I approve. Thanks for addressing all my comments and that of the other reviewers.

[dustinswales]

@climbfuji @grantfirl @DusanJovic-NOAA
I had to revert one change to the ccpp/CMakeLists, which was causing failures in the configurations using the GFDLMP w/ fast-physics.
Plus one other small change within MPAS to remove the timestamp from the log file. This makes the log reproducible.
All the UWM tests are passing, re-running now and will update logs in UWM PR when complete.