Update for "Changes in cloud/radiation interaction in GSD physics suite", non-aerosol-aware Thompson MP tests

[climbfuji]

This PR:

• adds regression tests for the non-aerosol-aware Thompson MP option
• contains changes in cloud/radiation interaction in GSD physics suite in FV3 and ccpp-physics from @tanyasmirnova

Associated PRs:
NOAA-GSL/ccpp-physics#16
NOAA-GSL/fv3atm#13
#11

See below for regression testing information.

[climbfuji]

@tanyasmirnova @haiqinli @hannahcbarnes @joeolson42

[climbfuji]

Regression testing on hera.intel against official DTC baseline using rt.conf: all tests pass except fv3_ccpp_gsd - this is expected because of the changes made here.

rt_full.log

[climbfuji]

Note: the code as-is fails in restart mode, apparently we need fix something there. It also fails for the CCPP dynamic build for several tests, which usually indicates that there are uninitialized variables or something else is wrong. I will investigate, but this shouldn't stop you from running the hindcasts.

[hannahcbarnes]

@tanyasmirnova @haiqinli @DomHeinzeller I put the retro results for the radiation retro in today's Global meeting folder (https://drive.google.com/drive/folders/18gIyPm90ioM-rAmI0-GC4FUo1HiftRt0) it is called Retro_#2b5768b_CldRad.pptx

[climbfuji]

Update 2020/03/06: I fixed the restart problem for the static CCPP build. Still, some tests are failing for the dynamic CCPP build:

[Dom.Heinzeller@hfe09 tests]$ grep -e FAIL log_hera.intel/*
log_hera.intel/rt_016_fv3_ccpp_gf_thompson_repro.log:Test 016 fv3_ccpp_gf_thompson FAIL
log_hera.intel/rt_017_fv3_ccpp_gsd_repro.log:Test 017 fv3_ccpp_gsd FAIL
log_hera.intel/rt_019_fv3_ccpp_gsd_warmstart_repro.log:Test 019 fv3_ccpp_gsd_warmstart FAIL
log_hera.intel/rt_020_fv3_ccpp_gsd_noah_repro.log:Test 020 fv3_ccpp_gsd_noah FAIL
log_hera.intel/rt_024_fv3_ccpp_gsd_drag_suite_repro.log:Test 024 fv3_ccpp_gsd_drag_suite FAIL

This is often an indicator of uninitialized arrays or wrong array dimensions. Will keep investigating.

[climbfuji]

I cannot reproduce the b4b issue on my macOS using the GNU compiler for a 1-hour forecast. Will try again next week on hera/intel. It only affects the phyf*.nc files - must be some diagnostic variable.

[DomHeinzeller]

I cannot reproduce the b4b issue on my macOS using the GNU compiler for a 1-hour forecast. Will try again next week on hera/intel. It only affects the phyf*.nc files - must be some diagnostic variable.

@tanyasmirnova @haiqinli @joeolson42 @hannahcbarnes Update. The difference is in the phy_f3d_02 variable (ice effective radii). These seem to be purely diagnostic, because all other results are identical. I thus believe that this comes from the cloud effective radii calculation in the Thompson MP call (because these values are not used, they are effectively overwritten by the new logic in the radiation routines). This hypothesis still needs to be confirmed.

[climbfuji]

I finally figured out the issue - uninitialized data in GFS_rrtmg_pre.F90 before going into the effective radii calculation from Thompson. Will update the code/PR and run the tests again.

[climbfuji]

After fixing the bugs in GFS_rrtmg_pre.F90 introduced by this PR, I reran the full regression tests with the same results as mentioned above in #11 (comment) (all tests pass except fv3_ccpp_gsd, which is expected).

Now the GSD regression tests (first create in static mode, then verify in static and dynamic mode) all pass, including all existing and new debug tests.

rt_ccpp_gsd_create.log
rt_ccpp_gsd_verify.log

[haiqinli]

@DomHeinzeller Yes, I think so.