In fv3atm: convert GFS DDTs from blocked data structures to contiguous arrays

[climbfuji]

Commit Queue Requirements:

• Fill out all sections of this template.
• All sub component pull requests have been reviewed by their code managers.
• Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
• Commit 'test_changes.list' from previous step

---

Description:

This PR updates the submodule pointers for fv3atm, gfdl_atmos_cubed_sphere, ccpp-physics for the changes described in the associates PRs below: convert internal GFS DDTs from blocked data structures to contiguous arrays. This excludes the (external) GFS_extdiag and GFS_restart DDTs.

Commit Message:

* UFSWM - In fv3atm and submodules, convert internal GFS DDTs from blocked data structures to contiguous arrays. This excludes the (external) `GFS_extdiag` and `GFS_restart` DDTs.
  * AQM - 
  * CDEPS - 
  * CICE - 
  * CMEPS - 
  * CMakeModules - 
  * FV3 - Convert GFS DDTs from blocked data structures to contiguous arrays (not including GFS_restart and GFS_extdiag DDTs)
    * ccpp-physics - Convert GFS DDTs from blocked data structures to contiguous arrays (affects `GFS_debug.{F90,meta} only`)
    * atmos_cubed_sphere - Convert GFS DDTs from blocked data structures to contiguous arrays and remove IPD_Data super DDT
  * GOCART - 
  * HYCOM - 
  * MOM6 - 
  * NOAHMP - 
  * WW3 - 
  * stochastic_physics - 

Priority:

• Normal

Git Tracking

UFSWM:

• Closes Convert blocked data structures in FV3atm to contiguous arrays #2294

Sub component Pull Requests:

• AQM:
• CDEPS:
• CICE:
• CMEPS:
• CMakeModules:
• FV3 - Convert GFS DDTs from blocked data structures to contiguous arrays (now also includes https://github.com/NOAA-EMC/fv3atm/pull/752) NOAA-EMC/ufsatm#798
  • ccpp-physics - Convert GFS DDTs from blocked data structures to contiguous arrays (aka update GFS_debug.F90) ccpp-physics#183
  • atmos_cubed_sphere - Branch dev/emc: convert GFS DDTs from blocked data structures to contiguous arrays NOAA-GFDL/GFDL_atmos_cubed_sphere#330
• GOCART:
• HYCOM:
• MOM6:
• NOAHMP:
• WW3:
• stochastic_physics:
• None

UFSWM Blocking Dependencies:

n/a

---

Changes

Regression Test Changes (Please commit test_changes.list):

• No Baseline Changes

Input data Changes:

• None.

Library Changes/Upgrades:

• No Updates

---

Testing Log:

• RDHPCS
  • Hera
  • Orion
  • Hercules
  • Jet
  • Gaea
  • Derecho
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
• opnReqTest (complete task if unnecessary)

[FernandoAndrade-NOAA]

Gaea failed compile_atm_debug_dyn32, rerunning.

[FernandoAndrade-NOAA]

Gaea's compile_atm_debug_dyn32 is persistently failing, it looks like an out of memory issue @jkbk2004 @zach1221 FYI
/gpfs/f5/epic/scratch/Fernando.Andrade-maldonado/RT_RUNDIRS/Fernando.Andrade-maldonado/FV3_RT/rt_88232/compile_atm_debug_dyn32_intel/err

 2616 ---------^
 2617 slurmstepd: error: Detected 1 oom_kill event in StepId=88636253.batch. Some of the step tasks have been OOM Killed.

[DusanJovic-NOAA]

Gaea's compile_atm_debug_dyn32 is persistently failing, it looks like an out of memory issue @jkbk2004 @zach1221 FYI /gpfs/f5/epic/scratch/Fernando.Andrade-maldonado/RT_RUNDIRS/Fernando.Andrade-maldonado/FV3_RT/rt_88232/compile_atm_debug_dyn32_intel/err

 2616 ---------^
 2617 slurmstepd: error: Detected 1 oom_kill event in StepId=88636253.batch. Some of the step tasks have been OOM Killed.

While porting to Intel LLVM I also found that some of the compile jobs (do not remember exactly which one) fail often with OOM error on Gaea. I ended up setting memory size explicitly to 4Gb per cpu, see:

https://github.com/ufs-community/ufs-weather-model/pull/2224/files#diff-3f224109d65db4d383892d9070d370bc8ec1361abac50bb22c7c1c4d996c2105

[BrianCurtis-NOAA]

I have not been granted permission to access Acorn yet. We can skip it for now.

[FernandoAndrade-NOAA]

I'll leave another update for Gaea with the suggested 4Gb fix once the rerun concludes. Jet is rerunning control_wam intel due to a timeout.

[FernandoAndrade-NOAA]

Tests passed with the proposed 4gb per cpu fix, thanks @DusanJovic-NOAA! Please ignore discrepancies in the baseline dates in the hera, gaea, and jet logs, they are the same.

[FernandoAndrade-NOAA]

We should be good to continue with the merge process, I'll let ccpp and cubed sphere to proceed with merging.

[DusanJovic-NOAA]

Merged fv3atm.