Feature/rrfs dev1: add retro and radar tten functions by hu5970 · Pull Request #108 · NOAA-GSL/regional_workflow

hu5970 · 2021-05-17T20:02:37Z

Add functions to run retro with staged data on disk.
Add namelist option for radar tten.

* Add 3-km CONUS and AK configure files for dev2

Add namelist option for radar tten.

guoqing-noaa · 2021-05-17T23:43:00Z

  </task>

-{%- endif %}
 {%- if machine in ["JET", "HERA"]  %}


A general question, do retros need the archive task?

I am not sure. But I never archived my retros runs and it is OK so far. If we need to archive retro for paper, we should only archive grib2 files under com.

guoqing-noaa · 2021-05-17T23:44:57Z

 if [[ $DO_RETRO == "true" ]] ; then
+
+  if [[ $MACHINE == "jet" ]] ; then
+    EXTRN_MDL_SOURCE_BASEDIR_ICS=/mnt/lfs4/BMC/wrfruc/Ruifang.Li/data/hrrr/conus/wrfnat/grib2


Do we have data staging scripts available? Or most users will get retro data from /mnt/lfs4/BMC/wrfruc/Ruifang.Li/data?

Yes, we only needs one copy of the data in the disk and all retros should use this one copy of the data to save the disk space.

guoqing-noaa · 2021-05-17T23:51:40Z

 FIX_CRTM=/home/rtrr/FIX_EXEC_MODULE/crtm/CRTM_v2.3.0
 OBSPATH_NSSLMOSIAC=/public/data/radar/nssl/mrms/conus
 RADARREFL_TIMELEVEL=(0 15 30 45)
+#FH_DFI_RADAR=(0.0,0.25,0.5)


Do you mean FH_DFI_RADAR=(0.0 0.25 0.5) instead of FH_DFI_RADAR=(0.0,0.25,0.5) (the latter is a one element array, i.e. "0.0,0.25,0.5" ) ?

It has to include comma as it will be used to fill in namelist variable.

Okay, I see. In this case, to avoid confusion, would it be enough to just let FH_DFI_RADAR="0.0,0.25,0.5" and hence no need to deal with an array which contains only one item?

That doesn't not work when python generates the namelist for FV3.

guoqing-noaa · 2021-05-17T23:53:03Z

 LAYOUT_X=""
 LAYOUT_Y=""
 BLOCKSIZE=""
+FH_DFI_RADAR=(-20000000000)


Is it possible to leave FH_DFI_RADAR as an empty array if expected to run without radar tten?

We cannot set empty value to namelist parameters. This is default value for this namelist variable.

suggest changing to FH_DFI_RADAR="-20000000000" so as to match another suggested change FH_DFI_RADAR="0.0,0.25,0.5"

Does not work that way.

guoqing-noaa · 2021-05-17T23:56:01Z

+#
+#  for retro, the external boundary could come from other location.
+#
+if [[ "${DO_RETRO}" = "true" && ! -z "${EXTRN_MDL_SOURCE_BASEDIR_ICS}" ]]; then


To make a consistent style, suggest to use double equal sign here: if [[ "${DO_RETRO}" == "true" && ! -z "${EXTRN_MDL_SOURCE_BASEDIR_ICS}" ]]; then

guoqing-noaa · 2021-05-17T23:57:31Z

 fi
+
+
+if [[ "${DO_RETRO}" = "true" && ! -z "${EXTRN_MDL_SOURCE_BASEDIR_LBCS}" ]]; then


Similarly, suggest to use double equal sign in this if statement

christinaholtNOAA · 2021-05-17T20:20:07Z

 #
+# FH_DFI_RADAR (hour):
+# The forecast period of using radar tten.
+#


When does the period start and end? Is this 0-4 if FH_DFI_RADAR=4? Should it be an array? If so, how many entries, and what do they correspond to?

Added more details.

christinaholtNOAA · 2021-05-17T20:21:56Z


 FV3_HRRR:
  fv_core_nml:
+    dz_min: 6.0


Is this an intended change for tten?

This is bug fix from Tanya.

christinaholtNOAA · 2021-05-17T20:22:24Z

+    OBSPATH_NSSLMOSIAC=/mnt/lfs4/BMC/wrfruc/Ruifang.Li/data/reflectivity
+    LIGHTNING_ROOT=/mnt/lfs4/BMC/wrfruc/Ruifang.Li/data/lightning/vaisala/netcdf/
+    ENKF_FCST=/mnt/lfs4/BMC/wrfruc/Ruifang.Li/data/enkf/atm
+  fi


Please remove user-specific paths.

This is the location we stage the retro data now. Will be used by all other users to run the same retro.

christinaholtNOAA · 2021-05-17T20:22:42Z

 FIX_CRTM=/home/rtrr/FIX_EXEC_MODULE/crtm/CRTM_v2.3.0
 OBSPATH_NSSLMOSIAC=/public/data/radar/nssl/mrms/conus
 RADARREFL_TIMELEVEL=(0 15 30 45)
+#FH_DFI_RADAR=(0.0,0.25,0.5)


Should this setting be on? If not, please remove commented code.

Some will be on and some will be off. It will be used in 3km CONUS if the parallel test in the next couple of week show positive impact.

christinaholtNOAA

Looks good. Just a few comments/questions.

guoqing-noaa · 2021-05-18T05:03:23Z

@hu5970 Thanks for the update. I suggest changing FH_DFI_RADAR=(0.0,0.25,0.5) to FH_DFI_RADAR="0.0,0.25,0.5" to avoid possible confusions. In the bash script, FH_DFI_RADAR is not an array of 3 or 5 items, but an array of one string "0.0,0.25,0.5".

hu5970 · 2021-05-18T15:18:57Z

@guoqing-noaa string does not work. That is why I ended up with an array.

guoqing-noaa

@hu5970 Sorry that I forgot to note that we need to make a minor update at ush/generate_FV3LAM_wflow.sh to allow to use FH_DFI_RADAR as a string. I just tested it and it worked. See my comment at ush/generate_FV3LAM_wflow.sh for the suggested change. I think a string is still preferred over an array for this situation. Thanks!

guoqing-noaa · 2021-05-18T16:36:17Z

-   }"
+   }
+ 'gfs_physics_nml': {
+    'fh_dfi_radar': [${FH_DFI_RADAR[@]}],


@hu5970 To accommodate changing FH_DFI_RADAR to a string, line 845 will need to be updated as:

- 'fh_dfi_radar': [${FH_DFI_RADAR[@]}], + 'fh_dfi_radar': [${FH_DFI_RADAR}],

String works now and it is better solution. I have committed the changes. Thanks.

guoqing-noaa

Thanks for making the updates for my picky comments. I will go ahead to approve the PR but hope the comments for FH_DFI_RADAR in ush/config_defaults.sh can be updated before the final merge. Thanks!

guoqing-noaa · 2021-05-18T20:13:25Z

+# FH_DFI_RADAR:
+# The forecast period of using radar tten. Array of 5:
+# (0.0,0.25,0.5,0.75,1.0) to use radar tten in the first hour of the
+# forecast with reading new observed tten every 15 minutes.


Suggest to change the comments here to something as follows to adapt to the change of FH_DFI_RADAR to a string

# FH_DFI_RADAR: # the forecast hour to use radar tten, this is used to set the fh_dfi_radar parameter in input.nml* # e.g. FH_DFI_RADAR="0.0,0.25,0.5,0.75,1.0" will set fh_dfi_radar = 0.0,0.25,0.5,0.75,1.0 in input.nml* and # it tells the model to read and apply radar tten at the 0, 15, 30, 45, 60 minutes of forecasts respectively. #

* Feature/rrfs dev1: add retrospective function and changes for running on WCOSS (#104) * Cold start RRFS_dev1 3km CONUS hourly cycles from 09/21Z HRRR analysis. * Changes for running on RRFS cycles on WCOSS. * Copy addtional grid from UPP run directory to comout space. (#105) * Copy addtional grid from UPP run directory to comout space. * Improvements for 3-km NA (#103) * Extending walltime for "run_post" in 3-km NA * Updated settings for layout_x, layout_y to improve run time. * Increase RRFS_AK time step to match CONUS (#110) * Feature/rrfs dev1: add retro and radar tten functions (#108) * Add functions to run retro with staged data on disk. * Add namelist option for radar tten. * Bug fix for using GFS on JET for 3km NA. * Feature/rrfs dev1: delete radar tten option (#112) * delete tten option in namelist to match current USD model. * Turn off cloud analysis for future debug. (#113) * Change the "overwrite" option to "upgrade" (#111) * Change $EXPDIR backup from inside to outside (#117) * RRFS_dev1: Add python maps (#82) * Updating job script to work with maps. * Workflow for Python graphics. * Use a metatask to manage map subsets. Move skewt out of metatask. * Sorting out plotting the wgrib2-produced domains. * Updating configure scripts for appropriate domains. * Adding domains for 3DRTMA * Fixes for undefined variables and logic changes. * RRFS_dev1: Turn on radar tten option and use "upgrade" to deploy. (#121) * Add and turn on radar tten option * Change PREEXISTING_DIR_METHOD from "rename" to "upgrade" for all RRFS_dev runs. * Add configure options to decide how to clean the old cycles. (#123) Change the reference date for clean from current run time to current cycle time Co-authored-by: Jaymes Kenyon <jaymes.kenyon@noaa.gov> Co-authored-by: Trevor Alcott <69324222+TrevorAlcott-NOAA@users.noreply.github.com> Co-authored-by: Guoqing Ge <guoqing.ge@noaa.gov> Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>

hu5970 and others added 4 commits May 14, 2021 17:03

Add 3-km CONUS and AK configure files for dev2

00521b1

Feature/rrfs dev2: add configure files (#106)

8bae7f7

* Add 3-km CONUS and AK configure files for dev2

Add functions to run retro with staged data on disk.

b4d0d0e

Add namelist option for radar tten.

Merge remote-tracking branch 'upstream/feature/RRFS_dev2' into HEAD

adc6703

hu5970 requested review from CurtisAlexander-NOAA, JeffBeck-NOAA, TrevorAlcott-NOAA, christinaholtNOAA, chunhuazhou, jaymes-kenyon and terraladwig as code owners May 17, 2021 20:02

hu5970 requested a review from guoqing-noaa May 17, 2021 20:02

guoqing-noaa reviewed May 17, 2021

View reviewed changes

christinaholtNOAA reviewed May 18, 2021

View reviewed changes

Update based on review.

1b99c70

Bug fix for using GFS on JET for 3km NA.

8138ddb

guoqing-noaa reviewed May 18, 2021

View reviewed changes

Use string for FH_DFI_RADAR.

5de8919

guoqing-noaa approved these changes May 18, 2021

View reviewed changes

Update explanation for FH_DFI_RADAR.

3a85143

hu5970 merged commit 0ff99ef into NOAA-GSL:feature/RRFS_dev1 May 18, 2021

EdwardColon-NOAA pushed a commit to EdwardColon-NOAA/regional_workflow that referenced this pull request Sep 24, 2023

Update hash for using NOAA-GSL/GSI rrfs branch. (NOAA-GSL#108)

a70d4c3

		fi


		if [[ "${DO_RETRO}" = "true" && ! -z "${EXTRN_MDL_SOURCE_BASEDIR_LBCS}" ]]; then

Conversation

hu5970 commented May 17, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

christinaholtNOAA left a comment

Choose a reason for hiding this comment

Uh oh!

guoqing-noaa commented May 18, 2021

Uh oh!

hu5970 commented May 18, 2021

Uh oh!

guoqing-noaa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guoqing-noaa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants