Stochastic physics clean and updates to CA by pjpegion · Pull Request #832 · ufs-community/ufs-weather-model

pjpegion · 2021-09-24T18:56:07Z

PR Checklist

Ths PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.
This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR
An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
are specified below.
If new or updated input data is required by this PR, it is clearly stated in the text of the PR.

Instructions: All subsequent sections of text should be filled in as appropriate.

The information provided below allows the code managers to understand the changes relevant to this PR, whether those changes are in the ufs-weather-model repository or in a subcomponent repository. Ufs-weather-model code managers will use the information provided to add any applicable labels, assign reviewers and place it in the Commit Queue. Once the PR is in the Commit Queue, it is the PR owner's responsiblity to keep the PR up-to-date with the develop branch of ufs-weather-model.

Description

Provide a detailed description of what this PR does. What bug does it fix, or what feature does it add? Is a change of answers expected from this PR? Are any library updates included in this PR (modulefiles etc.)?

The pull request includes a clean-up of the stochastic physics code and updates to the stand alone unit tests, which are now in the unit_tests sub-directory.
This PR also includes changes to the CA code for reproducibility for different process layouts and ca_global restart capability.

Issue(s) addressed

Link the issues to be closed with this PR, whether in this repository, or in another repository.
(Remember, issues must always be created before starting work on a PR branch!)

fixes CA fails debug mode with GNU compiler #770 Clean up of stochastic physics #823 control_ca test is not reproducible with 2-threads #743
fixes stochastic_physics/issues/ufs_public_release: update version number for CCPP, disable dynamic CCPP, cleanup work and bug fixes for regression tests #25 and stochastic_physics/issues/Fix restarts and add restart tests for ufs_public_release #33
-fixes Add ocean stochastic physics NOAA-EMC/MOM6#52

Testing

How were these changes tested? What compilers / HPCs was it tested with? Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) Have regression tests and unit tests (utests) been run? On which platforms and with which compilers? (Note that unit tests can only be run on tier-1 platforms)

Changes were tests on hera using intel and gnu compiler. There are 2 new tests, control_ca_restart and datm_stochy_gefs. In addition, the control_ca test results have changed.

Hera.gnu, baselines change with addition of two tests:

control_ca_debug
control_stochy_debug

Hera.intel, baselines change with one new test and 3 tests with answer changes:

datm_cdeps_stochy_gefs (new test)
cpld_ca
control_ca
control_ca_debug

These tests have do_ca=.true. All other tests do not currently use CA.

Dependencies

If testing this branch requires non-default branches in other repositories, list them. Those branches should have matching names (ideally).

Do PRs in upstream repositories need to be merged first?
If so add the "waiting for other repos" label and list the upstream PRs

MOM6 PR 72
waiting on code updates for CA_global NCAR/ccpp-physics#737
waiting on updates associated with CA restart and PE decomposition bugs NOAA-EMC/ufsatm#396
waiting on Stochastic physics cleanup NOAA-PSL/stochastic_physics#47

pjpegion · 2021-09-24T22:27:35Z

I ran the full GNU regression tests on hera, and the model is crashing in the control_flake and control_stochy tests, and I will update and I figure out what is going on.

pjpegion · 2021-09-25T01:00:46Z

Found the bug, there was an uninitialized array in spectral_transforms.F90. Gnu regression tests now pass on hera.

jiandewang · 2021-09-27T18:39:18Z

@pjpegion
https://github.com/pjpegion/ufs-weather-model/blob/new_stoch/.gitmodules#L30-L32
you are still pointing MOM6 to EMC repo, needs to use your MOM PR branch for ufs testing

pjpegion · 2021-09-27T19:12:05Z

@pjpegion
https://github.com/pjpegion/ufs-weather-model/blob/new_stoch/.gitmodules#L30-L32
you are still pointing MOM6 to EMC repo, needs to use your MOM PR branch for ufs testing

Sorry, I always forget about the gitmodules. fixed now

jiandewang · 2021-09-27T19:17:11Z

@pjpegion
https://github.com/pjpegion/ufs-weather-model/blob/new_stoch/.gitmodules#L32
you forgot to modify the branch name here

pjpegion · 2021-09-27T19:29:52Z

@pjpegion
https://github.com/pjpegion/ufs-weather-model/blob/new_stoch/.gitmodules#L32
you forgot to modify the branch name here

I don't think the brach name matters since the checkout should point to a specific has, but I updated it, along with the stochastic_physics branch name

…odel into new_stoch

junwang-noaa · 2021-09-30T12:16:11Z

MOM6 PR points to an issue, not a PR

DeniseWorthen · 2021-09-30T12:16:20Z

@MinsukJi-NOAA did CI run?

junwang-noaa · 2021-09-30T12:21:32Z

Yes, CI ran successfully last night.

…

On Thu, Sep 30, 2021 at 8:16 AM Denise Worthen ***@***.***> wrote: @MinsukJi-NOAA <https://github.com/MinsukJi-NOAA> did CI run? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#832 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI7D6TJIGUFD4XCNSBOJYPTUERIJ7ANCNFSM5EWRLEQA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

jiandewang · 2021-09-30T13:23:14Z

MOM6 PR has been merged

MinsukJi-NOAA · 2021-09-30T14:24:17Z

Failed control_restart CI can be fixed with this change related to the utest script. I think we can implement it either here or in the next PR.

diff --git a/tests/utests/rst.sh b/tests/utests/rst.sh
index edd1dc9..5d6f2de 100644
--- a/tests/utests/rst.sh
+++ b/tests/utests/rst.sh
@@ -4,6 +4,7 @@ source $PATHRT/utests/std.sh
 # Set up date and time of restart files for restart run
 FHROT=12
 RESTART_FILE_PREFIX="${SYEAR}${SMONTH}${SDAY}.$(printf "%02d" $(( SHOUR + FHROT  )))0000"
+DEP_RUN=${TEST_NAME}
 
 if [[ $application == 'global' ]]; then
   OUTPUT_FH="3 -1"
@@ -18,7 +19,6 @@ elif [[ $application == 'cpld' ]]; then
   RESTART_FILE_SUFFIX_HRS="${SYEAR}-${SMONTH}-${SDAY}-$(printf "%02d" $(( ${FHROT} )))"
   RESTART_FILE_SUFFIX_SECS="${SYEAR}-${SMONTH}-${SDAY}-$(printf "%02d" $(( ${FHROT}*3600 )))"
   RESTART_N=$((FHMAX-$FHROT))
-  DEP_RUN=${TEST_NAME}
 fi
 
 WARM_START=.T.

DeniseWorthen · 2021-09-30T14:25:52Z

Do SHOUR and FHROT and FHMAX need to be variables ($SHOUR,$FHROT,$FHMAX)?

MinsukJi-NOAA · 2021-09-30T14:27:28Z

Do SHOUR and FHROT need to be variables ($SHOUR,$FHROT)?

Not when used like this $(( SHOUR + FHROT ))

pjpegion · 2021-09-30T14:34:19Z

Is the change only moving DEP_RUN outside the if statement?

MinsukJi-NOAA · 2021-09-30T14:35:31Z

Is the change only moving DEP_RUN outside the if statement?

Yes

pjpegion · 2021-09-30T14:40:06Z

I'm just waiting in the merge of fv3atm.

junwang-noaa · 2021-09-30T14:45:52Z

@pjpegion fv3 PR was merged

DeniseWorthen

All submodules look correct to me.

climbfuji · 2021-09-30T15:05:46Z

@lisa-bengtsson any last words of wisdom, or are you ok with this PR? Please approve if yes.

lisa-bengtsson · 2021-09-30T15:10:48Z

@lisa-bengtsson any last words of wisdom, or are you ok with this PR? Please approve if yes.

Thank you for all your hard work everyone. I will approve this PR.

DeniseWorthen · 2021-09-30T15:15:11Z

I second the hard work comment. We'll be able to close multiple issues w/ this PR.

MinsukJi-NOAA · 2021-09-30T15:23:40Z

@DeniseWorthen CI all passed

pjpegion · 2021-12-15T19:02:29Z

Can we close this issue?

DeniseWorthen · 2021-12-15T19:17:06Z

Yes---this is the PR. Is there an associated issue that needs closing?

* Perturbation/increment file for cold starts: move the call to read_da_inc outside the if external_ic/restart logic.

pjpegion added 10 commits September 8, 2021 21:30

updates for cleaned up ca code

f8ff87a

add ca restart test

a9bafe3

add ca_restart

a16f951

sync with develop

fc52bc3

add ca_restart to rt.conf

e1aadfb

update stochastic_physics submodule

5e5b148

address code review changes

fd4f196

updated submodules

a12dac7

change rcell to 0.1 for RT

c79869f

update FV3 submodule

6bd07d0

updated stochastic_phyiscs

9d02429

DeniseWorthen added the Baseline Updates Current baselines will be updated. label Sep 27, 2021