Skip to content

Bug fix for QPFARI and QPFFFG encoding in RRFS#968

Merged
WenMeng-NOAA merged 10 commits into
NOAA-EMC:release/rrfs_v1from
EricJames-NOAA:qpfari_bugfix
Jun 12, 2024
Merged

Bug fix for QPFARI and QPFFFG encoding in RRFS#968
WenMeng-NOAA merged 10 commits into
NOAA-EMC:release/rrfs_v1from
EricJames-NOAA:qpfari_bugfix

Conversation

@EricJames-NOAA
Copy link
Copy Markdown
Contributor

@EricJames-NOAA EricJames-NOAA commented Jun 10, 2024 •
edited by WenMeng-NOAA
Loading

The main changes are:

  • Bug fix in prob values denoted in QPFARI and QPFFG
  • Bug fix in labeling of run total accumulated fields in sub-hourly
  • Bug fix in calculation of vertically summed smoke emissions
  • Change the path of g2tmpl/1.12.0 on WCOSS2 to the prod side
  • Change 1-h averaged MASSDEN to PM2.5 and PM10

prob_num and tot_num_prob were not set in grib2_module.f, resulting in seemingly random values showing up for the new GRIB2 records QPFARI and QPFFFG, as noted by Hui-Ya Chuang:

< 772:4824832913:vt=2024022023:surface:11-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat
=1 parm=197:prob >1:prob fcst 72/110
772:4824832913:vt=2024022023:surface:11-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat
=1 parm=197:prob >1:prob fcst 96/110
6177c6177
< 773:4824833141:vt=2024022023:surface:0-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat=
1 parm=197:prob >1:prob fcst 72/110

773:4824833141:vt=2024022023:surface:0-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat=
1 parm=197:prob >1:prob fcst 96/110

This small change fixes the issue, and should lead to bitwise identical results.

The change was tested for the RRFS_NA_3km system on Hera.

@EricJames-NOAA EricJames-NOAA changed the title QPFARI bugfix Bug fix for QPFARI and QPFFFG encoding in RRFS Jun 10, 2024
@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Jun 10, 2024

@EricJames-NOAA Can you sync your branch with the latest commit of the branch 'release/rrfs_v1'?

This was linked to issues Jun 10, 2024
Bug in calculation of vertically summed smoke emissions #969
Closed
Bug in prob values denoted in QPFARI and QPFFFG fields in RRFS #962
Closed
Bug in labeling of run total accumulated fields in sub-hourly output. #970
Closed
@EricJames-NOAA
Copy link
Copy Markdown
Contributor Author

EricJames-NOAA commented Jun 10, 2024

@WenMeng-NOAA welcome back! I hope your family are well.

Just to clarify, yes, these changes are needed for RRFSv1.

Smoke/dust folks have requested one more small change which I hope to include in this PR if that's OK.

@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Jun 11, 2024

@WenMeng-NOAA welcome back! I hope your family are well.

Just to clarify, yes, these changes are needed for RRFSv1.

Smoke/dust folks have requested one more small change which I hope to include in this PR if that's OK.

@EricJames-NOAA The modification in smoke/dust product calculation is fine with me unless @MatthewPyle-NOAA has concerns.

@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Jun 11, 2024

@EricJames-NOAA Can you take a look at my test results for the sub-hourly case at /home/Wen.Meng/stmp2/rrfs_2024050203 and let me know if there are any inconsistencies with your testing?

@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Jun 11, 2024

@EricJames-NOAA With the fix in grib2 meta data, the QPFARI and QPFFFG in rrfs PRSLEV dataset are encoded as:

772:714248470:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:9-10 hour acc fcst:prob >1:prob fcst 0/1
773:714248695:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:10-10 hour acc fcst:prob >1:prob fcst 0/1
774:714248920:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >2:prob fcst 0/1
775:714249145:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >2:prob fcst 0/1
776:714249370:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >5:prob fcst 0/1
777:714249595:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >5:prob fcst 0/1
778:714249820:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >10:prob fcst 0/1
779:714250045:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >10:prob fcst 0/1
780:714250270:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >100:prob fcst 0/1
781:714250495:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >100:prob fcst 0/1

Please confirm the above changes.

@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Jun 11, 2024

@EricJames-NOAA Can you also combine another change in your PR?
Update g2tmpl/1.12.0 location on WCOSS2 which was officially installed at the prod side:
in UPP/modulefiles/wcoss2.lua, remove line 48, and 49:

prepend_path("MODULEPATH"," /u/wen.meng/noscrub/ncep_post/g2tmpl/libs/modulefiles/compiler/intel/19.1.3.304")
load(pathJoin("g2tmpl", g2tmpl_ver))

@EricJames-NOAA
Copy link
Copy Markdown
Contributor Author

EricJames-NOAA commented Jun 11, 2024

@EricJames-NOAA Can you take a look at my test results for the sub-hourly case at /home/Wen.Meng/stmp2/rrfs_2024050203 and let me know if there are any inconsistencies with your testing?

@WenMeng-NOAA my tests were on Jet on /lfs4 which is not accessible at the moment. I am recreating my tests on Hera.

@EricJames-NOAA
Copy link
Copy Markdown
Contributor Author

EricJames-NOAA commented Jun 11, 2024

@EricJames-NOAA With the fix in grib2 meta data, the QPFARI and QPFFFG in rrfs PRSLEV dataset are encoded as:

772:714248470:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:9-10 hour acc fcst:prob >1:prob fcst 0/1
773:714248695:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:10-10 hour acc fcst:prob >1:prob fcst 0/1
774:714248920:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >2:prob fcst 0/1
775:714249145:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >2:prob fcst 0/1
776:714249370:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >5:prob fcst 0/1
777:714249595:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >5:prob fcst 0/1
778:714249820:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >10:prob fcst 0/1
779:714250045:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >10:prob fcst 0/1
780:714250270:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >100:prob fcst 0/1
781:714250495:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >100:prob fcst 0/1

Please confirm the above changes.

@WenMeng-NOAA those changes look correct and consistent with my results.

@WenMeng-NOAA WenMeng-NOAA linked an issue Jun 11, 2024 that may be closed by this pull request
Update UPP wcoss2 model to use recently implemented operational G2tmpl lib v1.12.0 #960
Closed
@EricJames-NOAA
Copy link
Copy Markdown
Contributor Author

EricJames-NOAA commented Jun 11, 2024

@WenMeng-NOAA your results are consistent with mine. I added the smoke change requested by NCEP, so this PR is ready from my perspective.

@WenMeng-NOAA WenMeng-NOAA linked an issue Jun 11, 2024 that may be closed by this pull request
1-h avg MASSDEN fields in RRFS need to be PM2.5 and PM10 #971
Closed
@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Jun 11, 2024

@EricJames-NOAA The 1-h averaged PM2.5 and PM10 in PRSLEV dataset are labeled as:

751:704042206:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <2.5e-06
752:704280574:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <1e-05

Please confirm.

@EricJames-NOAA
Copy link
Copy Markdown
Contributor Author

EricJames-NOAA commented Jun 12, 2024

@EricJames-NOAA The 1-h averaged PM2.5 and PM10 in PRSLEV dataset are labeled as:

751:704042206:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <2.5e-06
752:704280574:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <1e-05

Please confirm.

@WenMeng-NOAA yes, that is correct. We have changed these fields to be PM2.5 and PM10 (including both smoke and dust), so I changed the aerosol type to "Missing". The size categorization is correct.

@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Jun 12, 2024

@EricJames-NOAA The 1-h averaged PM2.5 and PM10 in PRSLEV dataset are labeled as:

751:704042206:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <2.5e-06
752:704280574:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <1e-05

Please confirm.

@WenMeng-NOAA yes, that is correct. We have changed these fields to be PM2.5 and PM10 (including both smoke and dust), so I changed the aerosol type to "Missing". The size categorization is correct.

@EricJames-NOAA Thanks for clarifying.

@WenMeng-NOAA WenMeng-NOAA merged commit fc85241 into NOAA-EMC:release/rrfs_v1 Jun 12, 2024
WenMeng-NOAA added a commit to WenMeng-NOAA/UPP that referenced this pull request Jun 17, 2024
@WenMeng-NOAA
Sync the RRFS hotfix NOAA-EMC#967 and NOAA-EMC#968.
bd4f4f4
This was referenced Jun 17, 2024
Merged
Closed
WenMeng-NOAA added a commit that referenced this pull request Jun 18, 2024
@WenMeng-NOAA @FernandoAndrade-NOAA
Sync the RRFS hotfix #967 and #968. (#979)
d5a7944 
* Sync the RRFS hotfix #967 and #968.

* hera RT log

* hercules RT log

---------

Co-authored-by: FernandoAndrade-NOAA <fernando.andrade-maldonado@noaa.gov>
This was referenced Jun 18, 2024
Closed
Closed
Closed
Closed
Closed
Closed
DavidHuber-NOAA added a commit to DavidHuber-NOAA/UPP that referenced this pull request Sep 9, 2024
@DavidHuber-NOAA
Merge remote-tracking branch 'origin/develop' into feature/ss_180
9669aba 
* origin/develop:
  Updates to support MPAS applications (RRFS and HFIP) (NOAA-EMC#1029)
  Add 6-hrly max winds at 10m to SFS (NOAA-EMC#1019)
  Update jet.lua (NOAA-EMC#1027)
  Updates to support UPP use with MPAS (NOAA-EMC#1018)
  [develop]: Turn on WARN_AS_ERROR for Doxygen Documentation Builds (NOAA-EMC#1023)
  Sample PR for testing (NOAA-EMC#1007)
  Enable template 4-49 to obtain aerosol ensemble information (NOAA-EMC#1011)
  Relocate gfs/gefs/sfs control files (NOAA-EMC#990)
  update g2tmp on aws to 1.12.0 (NOAA-EMC#996)
  [develop]: Update requirements files (NOAA-EMC#994)
  Bump zipp from 3.17.0 to 3.19.1 in /doc/UserGuide (NOAA-EMC#991)
  remove stack-python load for Gaea build (NOAA-EMC#993)
  Bump certifi from 2024.2.2 to 2024.7.4 in /doc/UserGuide (NOAA-EMC#988)
  [develop]: Add Doxygen variable definitions to modules (NOAA-EMC#965)
  Read bucket either as an integer or float (NOAA-EMC#984)
  Update Orion module file for Rocky 9 upgrade (NOAA-EMC#987)
  UPP bugfixes for inline post and g2tmpl 1.12.0 compatibility (and number concentration updates) (NOAA-EMC#974)
  Sync the RRFS hotfix NOAA-EMC#967 and NOAA-EMC#968. (NOAA-EMC#979)
  Bump urllib3 from 2.2.1 to 2.2.2 in /doc/UserGuide (NOAA-EMC#981)
  Fix for hydrostatic runs - write out Omega and HGT from model output (NOAA-EMC#959)
  Bump requests from 2.31.0 to 2.32.0 in /doc/UserGuide (NOAA-EMC#963)
  Correct rt.sh paths (NOAA-EMC#957)
  Updates to use GRIB2 template 4.9 for QPFFFG and QPFARI variables in RRFS (NOAA-EMC#929)
  Add hourly averaged smoke PM2.5 and dust PM10 (NOAA-EMC#917)
  Update Namelist documentation for all namelist options (NOAA-EMC#940)
  update gtg code revision to v.4.16.3 (NOAA-EMC#954)
  Bump jinja2 from 3.1.3 to 3.1.4 in /doc/UserGuide (NOAA-EMC#953)
  Clean up labeling for 15-min accumulated fields (NOAA-EMC#948)
  Updates to the GSL exp1 ceiling diagnostic (NOAA-EMC#947)
  Additional RT log information (NOAA-EMC#941)
  add cmake load to modulefile/gaea.lua (NOAA-EMC#945)
  RRFS updates (NOAA-EMC#938)
  add module for AWS (NOAA-EMC#934)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@WenMeng-NOAA WenMeng-NOAA WenMeng-NOAA approved these changes

@MatthewPyle-NOAA MatthewPyle-NOAA Awaiting requested review from MatthewPyle-NOAA

Assignees

No one assigned

Labels

RRFS

Projects

None yet

Milestone

No milestone

2 participants

@EricJames-NOAA @WenMeng-NOAA