Accumulation of changes to RRTMGP

[dustinswales]

Description

This PR contains improvements and bug fixes to the RRTMGP enabled ccpp-physics.

• Bug fix (indexing error) for temperature being passed to GP. (Only exercised in rare cases in EMC's C768L127 configuration).
• New coupling of RRTMGP to the RRFS physics. Based on fv3_regional_quilt using FV3_GFS_v15_thompson_mynn_RRTMGP.
• Revised approach to the LW flux adjustment using the full vertical profile of the Jacobian of the upwelling LW. Previously only the surface flux was adjusted.
• Move allocation of GP interstitial DDTs from the LW scheme to GFS_typedefs.F90 interstitial_create/reset.

Issue(s) addressed

N/A

Testing

This was tested on Hera using the Intel compilers. There are no changes to existing baselines. A new baseline needs to be added for the new RT, fv3_regional_quilt_RRTMGP.

Edit from @climbfuji:

• ALL RRTMGP tests change results
• Because of the bugfix for jet in ccpp-physics (previously, the compiler flag adjustments in FV3/ccpp/physics/CMakeLists.txt were ignored), most if not all results on jet change
• A number of bugfixes were required, some of which only showed up with the GNU compiler

Regression testing on tier-1 platforms:

• hera.intel
• hera.gnu
• orion.intel - skip because of ongoing filesystem (/work) problems
• cheyenne.intel
• cheyenne.gnu
• gaea.intel - log was lost due to bug in auto-rt, attaching the rt_auto log below instead.
• jet.intel
• wcoss_cray
• wcoss_dell_p3

gaea_intel_rt_auto_20210429173006.log

Dependencies

NOAA-EMC/ufsatm#275
NCAR/ccpp-physics#614

[RobertPincus]

Maybe just to emphasize that the title of the PR describes one change but the PR includes at least four changes.

[climbfuji]

Update on these PRs:

• several errors in the coderegression tests were addressed during the code review and while generating baselines yesterday
• two RRTMGP regression tests were commented out due to failures, among them the RRTMGP threading test
• the code does not compile on jet, and while we have created baselines on all other platforms we cannot proceed without knowing how the fix for jet impacts the code and the (RRTMGP) regression test results
• Note: because changes were made to detecting OpenMP for CCPP, @climbfuji compared the runtime of the existing control threading test (fv3_2threads) with the current trunk, and the runtime is identical (i.e. good)

The UFS code managers discussed, if the PR is not finalized and merged this afternoon MT, it will be postponed and we move on to the next PR.

@junwang-noaa @DusanJovic-NOAA @dustinswales @RobertPincus

[dustinswales]

@climbfuji
The GP RT w/ threading worked for me. I had to make a change to rt.conf for the test to work. Working on the c192l127test now

[climbfuji]

@climbfuji
The GP RT w/ threading worked for me. I had to make a change to rt.conf for the test to work. Working on the c192l127test now

You need to revert this change. Adding fv3 means that you are overwriting the single-thread results with the two-thread results. The 2thread test must pass against the single thread results.

More info: the fv3 column is used to determine whether a test is run when baselines are created. Tests without fv3 are only run when results are verified against existing baselines.

[dustinswales]

@

@climbfuji
The GP RT w/ threading worked for me. I had to make a change to rt.conf for the test to work. Working on the c192l127test now

You need to revert this change. Adding fv3 means that you are overwriting the single-thread results with the two-thread results. The 2thread test must pass against the single thread results.

More info: the fv3 column is used to determine whether a test is run when baselines are created. Tests without fv3 are only run when results are verified against existing baselines.

Gotcha. Will revert.

[climbfuji]

Note: CI tests passed for commit a9a3887.

[climbfuji]

Regression tests passed on all tier-1 platforms, except Orion: filesystem issues /work/ not responding, cannot run tests. I will start the merge with ccpp-physics.

[github-actions]

@dustinswales please bring these up to date with respective authoritative repositories

• fv3 NOT up to date

[climbfuji]

I fixed the submodule pointer for fv3atm, ready to merge.

[dustinswales]

I fixed the submodule pointer for fv3atm, ready to merge.

Thanks!