Skip to content

NSSL MP scheme updates + Stochastic Physics CI/CD & Unit Test#2661

Merged
FernandoAndrade-NOAA merged 20 commits into
ufs-community:developfrom
grantfirl:nssl3m-update2025a
May 1, 2025
Merged

NSSL MP scheme updates + Stochastic Physics CI/CD & Unit Test#2661
FernandoAndrade-NOAA merged 20 commits into
ufs-community:developfrom
grantfirl:nssl3m-update2025a

Conversation

@grantfirl
Copy link
Copy Markdown
Collaborator

@grantfirl grantfirl commented Mar 21, 2025

Commit Queue Requirements:

  • Fill out all sections of this template.
  • All sub component pull requests have been reviewed by their code managers.
  • Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
  • Commit 'test_changes.list' from previous step

Description:

This is work from @MicroTed

This NSSL-MP update makes some improvements under the hood. No changes to the public subroutine calls.

  • More accurate saturation mixing ratio calculation (iqvsopt=1)
  • Changed default droplet renucleation to irenuc=5, which allows extra nucleation at high supersaturation
  • A default explicit rain breakup for 3-moment rain (irainbreak=2) has been added to reduce excessive median drop diameters in rain cores, which in turn increases evaporation and cool pool temperature deficits (were too warm).
  • Imposed reflectivity conservation in graupel->hail conversion (ihlcnh=3) and Bigg freezing (both 2- and 3-moment)

This PR also contains changes to the CCPP scientific documentation for the SRW v3 release. Output is here: https://dtcenter.ucar.edu/GMTB/UFS_SRW_App_v3.0.0/sci_doc/index.html

Combined PR: #2718

This PR updates stochastic physics to enable the running of standalone tests along with the automation of standalone tests via Jenkins pipeline integration.

Commit Message:

* UFSWM - NSSL-MP scheme updates
  * FV3 - NSSL-MP scheme updates
    * ccpp-physics - NSSL-MP scheme updates + scientific documentation updates for SRW v3.0.0
  * stochastic_physics - enable standalone unit tests and Jenkins pipeline git integration
  * MOM6 hash update 25ed

Priority:

  • Normal

Git Tracking

UFSWM:

  • None

Sub component Pull Requests:

UFSWM Blocking Dependencies:

  • None

Changes

Regression Test Changes (Please commit test_changes.list):

  • PR Updates/Changes Baselines.

Input data Changes:

  • None.

Library Changes/Upgrades:

  • No Updates

Testing Log:

  • RDHPCS
    • Hera
    • Orion
    • Hercules
    • GaeaC5
    • GaeaC6
    • Derecho
  • WCOSS2
    • Dogwood/Cactus
    • Acorn
  • CI
  • opnReqTest (complete task if unnecessary)

@grantfirl
Copy link
Copy Markdown
Collaborator Author

UFS RTs running now on Hera. Will update logs when finished.

@grantfirl grantfirl marked this pull request as ready for review April 1, 2025 19:15
@grantfirl
Copy link
Copy Markdown
Collaborator Author

All tests that failed are expected except for test_cpld_control_pdlib_p8_gnu which timed out.

@jkbk2004
Copy link
Copy Markdown
Collaborator

@grantfirl it sounds like you are on leave. Anyway, we can schedule when you come back.

@grantfirl
Copy link
Copy Markdown
Collaborator Author

@grantfirl it sounds like you are on leave. Anyway, we can schedule when you come back.

@jkbk2004 OK, this has been updated to the latest develop branches.

@jkbk2004 jkbk2004 mentioned this pull request Apr 29, 2025
14 tasks
@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Apr 29, 2025

@grantfirl
Copy link
Copy Markdown
Collaborator Author

#2718

Is there a UWM PR associated with the MOM6 commit hash change?

@grantfirl grantfirl changed the title NSSL MP scheme updates NSSL MP scheme updates + Stochastic Physics CI/CD & Unit Test Apr 29, 2025
@jkbk2004
Copy link
Copy Markdown
Collaborator

@grantfirl Git issue only: #2717. I did sanity check ok to update the hash.

@jkbk2004 jkbk2004 added Baseline Updates Current baselines will be updated. Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked. jenkins-ort run ORT testing labels Apr 29, 2025
@jkbk2004 jkbk2004 added the In Testing The PR that is currently in testing stages label Apr 30, 2025
@BrianCurtis-NOAA
Copy link
Copy Markdown
Collaborator

Have we verified that test_cpld_control_pdlib_p8_gnu is passing or still timing out?

@grantfirl
Copy link
Copy Markdown
Collaborator Author

Have we verified that test_cpld_control_pdlib_p8_gnu is passing or still timing out?

The latest RT log from Hera showed no problems with cpld_control_pdlib_p8_gnu.

@BrianCurtis-NOAA
Copy link
Copy Markdown
Collaborator

Have we verified that test_cpld_control_pdlib_p8_gnu is passing or still timing out?

The latest RT log from Hera showed no problems with cpld_control_pdlib_p8_gnu.

The test_changes.list has that test listed, so the process would include regenerating baselines for that file. If it was just a timeout, and not due to baselines changing, then we would want to update the test_changes.list to make note that it's not timing out any more and it's not expected to change baselines. Looking back at this PR for any reason would make me think this PR caused changes to that test.

I just wanted to make sure that before starting testing the gnu test passed. Maybe I missed it?

@grantfirl
Copy link
Copy Markdown
Collaborator Author

Have we verified that test_cpld_control_pdlib_p8_gnu is passing or still timing out?

The latest RT log from Hera showed no problems with cpld_control_pdlib_p8_gnu.

The test_changes.list has that test listed, so the process would include regenerating baselines for that file. If it was just a timeout, and not due to baselines changing, then we would want to update the test_changes.list to make note that it's not timing out any more and it's not expected to change baselines. Looking back at this PR for any reason would make me think this PR caused changes to that test.

I just wanted to make sure that before starting testing the gnu test passed. Maybe I missed it?

I didn't re-run the pretest, but it definitely should not change the results for it. Would you like me to try to re-run the test to compare against the previous baseline?

@BrianCurtis-NOAA
Copy link
Copy Markdown
Collaborator

Have we verified that test_cpld_control_pdlib_p8_gnu is passing or still timing out?

The latest RT log from Hera showed no problems with cpld_control_pdlib_p8_gnu.

The test_changes.list has that test listed, so the process would include regenerating baselines for that file. If it was just a timeout, and not due to baselines changing, then we would want to update the test_changes.list to make note that it's not timing out any more and it's not expected to change baselines. Looking back at this PR for any reason would make me think this PR caused changes to that test.
I just wanted to make sure that before starting testing the gnu test passed. Maybe I missed it?

I didn't re-run the pretest, but it definitely should not change the results for it. Would you like me to try to re-run the test to compare against the previous baseline?

I don't wish to add anything to your plate, I'll run that test to confirm it compared to previous baselines

@grantfirl
Copy link
Copy Markdown
Collaborator Author

Have we verified that test_cpld_control_pdlib_p8_gnu is passing or still timing out?

The latest RT log from Hera showed no problems with cpld_control_pdlib_p8_gnu.

The test_changes.list has that test listed, so the process would include regenerating baselines for that file. If it was just a timeout, and not due to baselines changing, then we would want to update the test_changes.list to make note that it's not timing out any more and it's not expected to change baselines. Looking back at this PR for any reason would make me think this PR caused changes to that test.
I just wanted to make sure that before starting testing the gnu test passed. Maybe I missed it?

I didn't re-run the pretest, but it definitely should not change the results for it. Would you like me to try to re-run the test to compare against the previous baseline?

I don't wish to add anything to your plate, I'll run that test to confirm it compared to previous baselines

@BrianCurtis-NOAA Thanks very much, I appreciate it.

@jkbk2004
Copy link
Copy Markdown
Collaborator

I re-test cpld_debug_pdlib_p8 against 0416 baseline on hercules and confirm no problem.

@BrianCurtis-NOAA
Copy link
Copy Markdown
Collaborator

[16:59:43]Brian.Curtis@hfe08:/scratch1/NCEPDEV/nems/Brian.Curtis/git/grantfirl/ufs-weather-model/tests
-->cat logs/RegressionTests_hera.log 
====START OF HERA REGRESSION TESTING LOG====

UFSWM hash used in testing:
68e9d2eb362d20aa51d5183c0f9b520082deb2eb

Submodule hashes used in testing:
 d3c1bc45492f8b35948c97021e4e6ab4d4c203e4 AQM (v0.2.0-38-gd3c1bc4)
 132fa6963ba2f7103f711eb567e4fe97ed0804a8 CDEPS-interface/CDEPS (cdeps0.4.17-406-g132fa69)
 4411a158cc4839157e3087d7fe8e399f9685eb35 CICE-interface/CICE (CICE6.0.0-399-g4411a15)
 00cf7e02ab4cb3cd095ac033b245e9dce924efba CMEPS-interface/CMEPS (cmeps_v0.4.1-2316-g00cf7e0)
 cabd7753ae17f7bfcc6dad56daf10868aa51c3f4 CMakeModules (v1.0.0-28-gcabd775)
 41398b0226d53725a3db6f9eccb224562505427a FV3 (remotes/origin/nssl3m-update2025a)
 041422934cae1570f2f0e67239d5d89f11c6e1b7 GOCART (sdr_v2.1.2.6-119-g0414229)
 bcf7777bb037ae2feb2a8a8ac51aacb3511b52d9 HYCOM-interface/HYCOM (2.3.00-122-gbcf7777)
 c4d2337d8607ec994b3cd61179eb974e0a237841 LM4-driver (baseline_change_240904-6-gc4d2337)
 25ed632840044136ba5b9292c466a168cee2ea5b MOM6-interface/MOM6 (dev/master/repository_split_2014.10.10-10469-g25ed63284)
 1a2d2fcb02da38e975f252ef743af24b68b3cc7f NOAHMP-interface/noahmp (v3.7.1-455-g1a2d2fc)
 d09a7a97241a5c75b9c5452711f0080b2a2dba1b WW3 (6.07.1-438-gd09a7a97)
 05cad173feeb598431e3ef5f17c2df6562c8d101 fire_behavior (v0.2.0-1-g05cad17)
 b5cec8b4f945b3f12c105368f8577cbecbdb0502 stochastic_physics (remotes/origin/platform-1077)


NOTES:
[Times](Memory) are at the end of each compile/test in format [MM:SS](Size).
The first time is for the full script (prep+run+finalize).
The second time is specifically for the run phase.
Times/Memory will be empty for failed tests.

BASELINE DIRECTORY: /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/develop-20250416
COMPARISON DIRECTORY: /scratch1/NCEPDEV/stmp2/Brian.Curtis/FV3_RT/rt_3061166

RT.SH OPTIONS USED:
* (-a) - HPC PROJECT ACCOUNT: nems
* (-n) - RUN SINGLE TEST: cpld_control_pdlib_p8
* (-e) - USE ECFLOW

PASS -- COMPILE 's2sw_pdlib_gnu' [17:11, 15:45]
PASS -- TEST 'cpld_control_pdlib_p8_gnu' [24:28, 21:39](1465 MB)

SYNOPSIS:
Starting Date/Time: 20250430 15:52:27
Ending Date/Time: 20250430 16:35:20
Total Time: 00h:43m:04s
Compiles Completed: 1/1
Tests Completed: 1/1

NOTES:
A file 'test_changes.list' was generated but is empty.
If you are using this log as a pull request verification, please commit 'test_changes.list'.

Result: SUCCESS

====END OF HERA REGRESSION TESTING LOG====

@FernandoAndrade-NOAA FernandoAndrade-NOAA removed the jenkins-ort run ORT testing label May 1, 2025
@jkbk2004 jkbk2004 removed the In Testing The PR that is currently in testing stages label May 1, 2025
@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented May 1, 2025

We can start merging process.

@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented May 1, 2025

@grantfirl new fv3 hash is NOAA-EMC/ufsatm@524330e and new stochastic physics hash is NOAA-PSL/stochastic_physics@97b24f8.

@grantfirl
Copy link
Copy Markdown
Collaborator Author

@grantfirl new fv3 hash is NOAA-EMC/fv3atm@524330e and new stochastic physics hash is NOAA-PSL/stochastic_physics@97b24f8.

@jkbk2004 hashes updated. Ready for review.

@FernandoAndrade-NOAA FernandoAndrade-NOAA merged commit ff162f3 into ufs-community:develop May 1, 2025
4 checks passed
@grantfirl grantfirl mentioned this pull request May 1, 2025
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Baseline Updates Current baselines will be updated. Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants