Feature/rrfs dev1: add retrospective function and changes for running on WCOSS by hu5970 · Pull Request #104 · NOAA-GSL/regional_workflow

hu5970 · 2021-05-12T22:27:51Z

This PR to add the following functions to RRFS:

Code update for running on WCOSS with FV3GFS background.
Run retrospective experiment on WCOSS when the data are still on disk.
This also builds the same dependences for all retrospective experiments.

Update RRFS_dev1 configure file for running 48/18h forecast.

…e/RRFS_dev1

…g experiment. RRFS dev1 CONUS: Hourly cycled with forecasts to 18 hrs except 48 hrs at 00/06/12/18z, cold-started at 09z/21z from HRRR, boundary from RAP. RRFS dev1 AK: Hourly cycled with forecasts to 21 hrs, 63 hours at 03/09/15/21z, cold-started at 03z/15z from GFS RRFS dev1 NA 13km: 3-hourly cycled with forecasts to 18 hrs, cold-started at 09z/21z from RAP, boundary from RAP

Use rap_e for 00/12Z cycle.

… one task for both long and normal forecast.

Add function to run retrospective experiment on WCOSS when the data are still on disk.

christinaholtNOAA

Ming, this looks pretty good, although I don't know what is needed on WCOSS.

I've made several comments that are very similar to comments I've made on previous PRs. Please keep them in mind when adding new features.

christinaholtNOAA · 2021-05-12T22:56:14Z


 DO_DACYCLE="true"
+#DO_RETRO="true"
+#LBCS_ICS_ONLY="true"


Please remove commented configurations.

We will use the retro functions a a lot soon. This is not just a comment. This is a place hold for retro because ti is much easier and safe to uncomment those options for retrospective runs.

christinaholtNOAA · 2021-05-12T22:56:57Z

+  ;;
+"JET")
+
+  for loop in $loops; do


Please use the $() syntax instead of backticks for clarity, ease of debugging, and supportability here and in the previous case statement.

Yes, change all in this scripts.

christinaholtNOAA · 2021-05-12T22:58:13Z

-DATE_FIRST_CYCL="20210401"
-DATE_LAST_CYCL="20211231"
-CYCL_HRS=( "00" "12" )
+DT_ATMOS=30


Was this an intended change for RRFS_dev1 in real time?

Yes, that is the number in real-time runs now.

christinaholtNOAA · 2021-05-12T23:00:58Z


 DO_DACYCLE="true"
+DO_RETRO="false"
+LBCS_ICS_ONLY="false"


These should be the default, and shouldn't need to be set here, right?

Again, I would like to leave them as a comment to make retro run easier. many colleagues may not know those options.

christinaholtNOAA · 2021-05-12T23:02:29Z

-POSTPROC_CYCLEDEF="00 00-23/01 ${CYCLEDAY} ${CYCLEMONTH} 2021 *"
-POSTPROC_LONG_CYCLEDEF="00 00,06,12,18 ${CYCLEDAY} ${CYCLEMONTH} 2021 *"
-ARCHIVE_CYCLEDEF="00 07 ${CYCLEDAY} ${CYCLEMONTH} 2021 *"
+#CYCLEDAY="1-4"


How and where is CYCLEDAY set? If 1-4 is a default value, please remove from the config file for clarity.

The default value is "*" but I use this one always in my test. It is much easier to leave it as place hold for testing.
Our code is under active development, there are some commend lines are there for easy pick up during develop and test.

christinaholtNOAA · 2021-05-12T23:04:31Z

+COMINgfs=""
+
+STMP="/gpfs/hps3/ptmp/Ming.Hu/stmp"  # Path to directory STMP that mostly contains input files.
+PTMP="/gpfs/hps3/ptmp/Ming.Hu/ptmp"  # Path to directory STMP that mostly contains input files.


Please leave out personal output locations for retro runs.

There are no official space on WCOSS. All tests (even parallel for operation updates) are done in person space.

christinaholtNOAA · 2021-05-12T23:05:59Z

        else
-          fns_on_disk=( "gfs.t${hh}z.pgrb2.0p25.f${fcst_hh}" )  # Get only 0.25 degree files for now.
+         # fns_on_disk=( "gfs.t${hh}z.pgrb2.0p25.f000")  # Get only 0.25 degree files for now.
+          fns_on_disk=( "gfs.t${hh}z.pgrb2.0p25.f0${fcst_hh}" "gfs.t${hh}z.sfcf0${fcst_hh}.nc")  # use netcdf


Please remove commented code.

* Feature/rrfs dev1: add retrospective function and changes for running on WCOSS (#104) * Cold start RRFS_dev1 3km CONUS hourly cycles from 09/21Z HRRR analysis. * Changes for running on RRFS cycles on WCOSS. * Copy addtional grid from UPP run directory to comout space. (#105) * Copy addtional grid from UPP run directory to comout space. * Improvements for 3-km NA (#103) * Extending walltime for "run_post" in 3-km NA * Updated settings for layout_x, layout_y to improve run time. * Increase RRFS_AK time step to match CONUS (#110) * Feature/rrfs dev1: add retro and radar tten functions (#108) * Add functions to run retro with staged data on disk. * Add namelist option for radar tten. * Bug fix for using GFS on JET for 3km NA. * Feature/rrfs dev1: delete radar tten option (#112) * delete tten option in namelist to match current USD model. * Turn off cloud analysis for future debug. (#113) * Change the "overwrite" option to "upgrade" (#111) * Change $EXPDIR backup from inside to outside (#117) * RRFS_dev1: Add python maps (#82) * Updating job script to work with maps. * Workflow for Python graphics. * Use a metatask to manage map subsets. Move skewt out of metatask. * Sorting out plotting the wgrib2-produced domains. * Updating configure scripts for appropriate domains. * Adding domains for 3DRTMA * Fixes for undefined variables and logic changes. * RRFS_dev1: Turn on radar tten option and use "upgrade" to deploy. (#121) * Add and turn on radar tten option * Change PREEXISTING_DIR_METHOD from "rename" to "upgrade" for all RRFS_dev runs. * Add configure options to decide how to clean the old cycles. (#123) Change the reference date for clean from current run time to current cycle time Co-authored-by: Jaymes Kenyon <jaymes.kenyon@noaa.gov> Co-authored-by: Trevor Alcott <69324222+TrevorAlcott-NOAA@users.noreply.github.com> Co-authored-by: Guoqing Ge <guoqing.ge@noaa.gov> Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>

hu5970 added 14 commits April 28, 2021 22:58

Cold start RRFS_dev1 3km CONUS hourly cycles from 09/21Z HRRR analysis.

2723906

Changes for running on RRFS cycles on WCOSS.

1a36dbd

scripts handle multiple time levels of radar data process and ref2tten

7200c9a

Add groups and tasks for post process.

6aa9445

Update RRFS_dev1 configure file for running 48/18h forecast.

Merge remote-tracking branch 'upstream/feature/RRFS_dev1' into featur…

21b86c8

…e/RRFS_dev1

Use early rap observation BUFR files for 00/12Z cycle.

6d540a0

Bug fix to exregional_process_bufr.sh.

3a1140f

Use rap_e for 00/12Z cycle.

Finalize the configure files for RRFS dev1 CONUS, AK, and NA 13km.

fba687b

Use Guoqing's method to generate the post process tasks so we can use…

bae8e5d

… one task for both long and normal forecast.

Merge remote-tracking branch 'upstream/feature/RRFS_dev1' into HEAD

6edf08e

Add function to do retrospective test on WCOSS.

4f1ecbd

Merge remote-tracking branch 'upstream/feature/RRFS_dev1' into HEAD

cccc4dd

Code update for running on WCOSS with FV3GFS background.

c33b68d

Add function to run retrospective experiment on WCOSS when the data are still on disk.

hu5970 requested review from CurtisAlexander-NOAA, JeffBeck-NOAA, TrevorAlcott-NOAA, christinaholtNOAA, chunhuazhou, jaymes-kenyon and terraladwig as code owners May 12, 2021 22:27

hu5970 requested a review from guoqing-noaa May 12, 2021 22:28

christinaholtNOAA approved these changes May 12, 2021

View reviewed changes

hu5970 added 2 commits May 13, 2021 15:58

Bug fix to link the observation files on WCOSS.

7ee8510

Clean the code for PR104.

b2e252b

hu5970 merged commit fe83181 into NOAA-GSL:feature/RRFS_dev1 May 13, 2021

EdwardColon-NOAA pushed a commit to EdwardColon-NOAA/regional_workflow that referenced this pull request Sep 24, 2023

Update Hash fro regional workflow and rrfs_util (NOAA-GSL#104)

5510dbb

Conversation

hu5970 commented May 12, 2021

Uh oh!

christinaholtNOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants