Skip to content

Weekly update of JEDI hashes (20251103)#1967

Merged
RussTreadon-NOAA merged 6 commits into
developfrom
feature/stable-nightly
Nov 3, 2025
Merged

Weekly update of JEDI hashes (20251103)#1967
RussTreadon-NOAA merged 6 commits into
developfrom
feature/stable-nightly

Conversation

@RussTreadon-NOAA

@RussTreadon-NOAA RussTreadon-NOAA commented Oct 31, 2025

Copy link
Copy Markdown
Contributor

Description

This PR contains three sets of changes:

  • weekly update of select JEDI hashes
  • remove hardwired user name in Hera CI script
  • change cube_sphere_grid file names for atm_jjob ctests

Companion PRs

None

Issues

None

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmsnowDA
  • C96_gcafs_cycled
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@RussTreadon-NOAA RussTreadon-NOAA self-assigned this Oct 31, 2025
@RussTreadon-NOAA RussTreadon-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion hercules-GW-RT Queue for automated testing with global-workflow on Hercules ursa-GW-RT gaeac6-GW-RT labels Nov 1, 2025
@emcbot emcbot added hercules-GW-RT-Running Automated testing with global-workflow running on Hercules ursa-GW-RT-Running orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed hercules-GW-RT Queue for automated testing with global-workflow on Hercules ursa-GW-RT orion-GW-RT Queue for automated testing with global-workflow on Orion labels Nov 1, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Nov 1, 2025
@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor Author

Automated GW-GDASApp Testing Results:
Machine: gaeac6

Start: Sat Nov  1 06:05:35 AM EDT 2025 on gaea68
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Nov  1 06:34:04 AM EDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                  *Failed*
Tests: Failed at Sat Nov  1 07:13:18 AM EDT 2025
Tests: 89% tests passed, 14 tests failed out of 125
	1705 - test_gdasapp_C96C48_hybatmsnowDA_gdas_sfcanl_202112201800 (Failed)
	1706 - test_gdasapp_C96C48_hybatmsnowDA_gdas_analcalc_202112201800 (Failed)
	1713 - test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_esfc_202112201800 (Failed)
	1743 - test_gdasapp_atm_jjob_var_init (Failed)
	1744 - test_gdasapp_atm_jjob_var_run (Failed)
	1745 - test_gdasapp_atm_jjob_var_inc (Failed)
	1746 - test_gdasapp_atm_jjob_var_final (Failed)
	1747 - test_gdasapp_atm_jjob_ens_init (Failed)
	1748 - test_gdasapp_atm_jjob_ens_letkf (Failed)
	1749 - test_gdasapp_atm_jjob_ens_init_split (Failed)
	1750 - test_gdasapp_atm_jjob_ens_obs (Failed)
	1751 - test_gdasapp_atm_jjob_ens_sol (Failed)
	1752 - test_gdasapp_atm_jjob_ens_inc (Failed)
	1753 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /gpfs/f6/ira-sti/scratch/Russ.Treadon/CI/GDASApp/workflow/PR/1967/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot

emcbot commented Nov 1, 2025

Copy link
Copy Markdown

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Sat Nov  1 05:02:10 CDT 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Nov  1 05:38:12 CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled
Tests:                                  *Failed*
Tests: Failed at Sat Nov  1 06:21:28 CDT 2025
Tests: 97% tests passed, 3 tests failed out of 110
	1705 - test_gdasapp_C96C48_hybatmsnowDA_gdas_sfcanl_202112201800 (Failed)
	1706 - test_gdasapp_C96C48_hybatmsnowDA_gdas_analcalc_202112201800 (Failed)
	1713 - test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_esfc_202112201800 (Failed)
Tests: see output at /work2/noaa/da/role-da/CI/hercules/GDASApp/workflow/PR/1967/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Nov 1, 2025
@emcbot

emcbot commented Nov 1, 2025

Copy link
Copy Markdown

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Sat Nov  1 10:04:37 UTC 2025 on hfe11
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Nov  1 10:41:50 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                  *Failed*
Tests: Failed at Sat Nov  1 11:27:20 UTC 2025
Tests: 98% tests passed, 3 tests failed out of 134
	1714 - test_gdasapp_C96C48_hybatmsnowDA_gdas_sfcanl_202112201800 (Failed)
	1715 - test_gdasapp_C96C48_hybatmsnowDA_gdas_analcalc_202112201800 (Failed)
	1722 - test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_esfc_202112201800 (Failed)
Tests: see output at /scratch3/NCEPDEV/da/role.jedipara/CI/GDASApp/workflow/PR/1967/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added the hera-GW-RT-Failed Automated testing with global-workflow failed on Hera label Nov 1, 2025
@emcbot emcbot removed the hercules-GW-RT-Running Automated testing with global-workflow running on Hercules label Nov 1, 2025
@emcbot

emcbot commented Nov 1, 2025

Copy link
Copy Markdown

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Sat Nov  1 13:49:38 UTC 2025 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Nov  1 14:37:29 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Sat Nov  1 15:23:16 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 134

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Nov 1, 2025
@emcbot

emcbot commented Nov 1, 2025

Copy link
Copy Markdown

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Sat Nov  1 08:48:32 AM CDT 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Nov  1 09:44:36 AM CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled
Tests:                                 *SUCCESS*
Tests: Completed at Sat Nov  1 10:57:51 AM CDT 2025
Tests: 100% tests passed, 0 tests failed out of 110

@emcbot emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Nov 1, 2025
@emcbot

emcbot commented Nov 1, 2025

Copy link
Copy Markdown

Automated GW-GDASApp Testing Results:
Machine: ursa

Start: Sat Nov  1 13:49:08 UTC 2025 on ufe01
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Nov  1 14:11:53 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Sat Nov  1 16:39:22 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 134

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor Author

GDASApp ctests on WCOSS2 & Gaea C6

Install feature/stable-nightly at a46810f inside g-w develop at d190541 on WCOSS2 (Cactus) and Gaea C6.

Run GDASApp ctests, including select g-w CI, on WCOSS2 with the following results

Test project /lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build
        Start 1654: test_gdasapp_C96C48_ufs_hybatmDA
        Start 1635: test_gdasapp_C96C48_hybatmDA
        Start 1693: test_gdasapp_C96C48_hybatmsnowDA
        Start 1678: test_gdasapp_C96_gcafs_cycled
        Start 1725: test_gdasapp_C48mx500_hybAOWCDA
        Start 1714: test_gdasapp_C48mx500_3DVarAOWCDA
        Start 1741: test_gdasapp_setup_atm_jjob_cycled_exp
        Start 1740: test_gdasapp_convert_gsi_satbias
        Start 1633: test_gdasapp_snow_imsproc
        Start 1634: test_gdasapp_convert_bufr_adpsfc_snow
        Start 1739: test_gdasapp_convert_bufr_adpsfc
        Start 1632: test_gdasapp_snow_create_ens
  1/125 Test #1632: test_gdasapp_snow_create_ens .............................................   Passed    4.64 sec
        Start 1753: test_gdasapp_aero_gen_3dvar_yaml
  2/125 Test #1633: test_gdasapp_snow_imsproc ................................................   Passed    6.67 sec
...
124/125 Test #1669: test_gdasapp_C96C48_ufs_hybatmDA_gdas_fcst_202402240000 ..................   Passed  420.12 sec
125/125 Test #1677: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_fcst_202402240000 ..............   Passed  257.85 sec

100% tests passed, 0 tests failed out of 125

Label Time Summary:
gdasapp    = 25156.95 sec*proc (116 tests)
script     = 25156.95 sec*proc (116 tests)

Total Test time (real) = 3044.37 sec

All tests run on WCOSS2 Passed

Prior to running GDASApp ctests on Gaea C6, modify working copy of sorc/gdas.cd/modulefiles/GDAS/gaeac6.intel.lua as follows

@@ -88,9 +88,9 @@ local mpinproc = '-n'
 setenv('MPIEXEC_EXEC', mpiexec)
 setenv('MPIEXEC_NPROC', mpinproc)
 
-setenv("CRTM_FIX","/gpfs/f6/ira-sti/world-shared/GDASApp/fix/crtm/2.4.0")
-setenv("GDASAPP_TESTDATA","/gpfs/f6/ira-sti/world-shared/GDASApp/testdata")
-setenv("GDASAPP_UNIT_TEST_DATA_PATH", "/gpfs/f6/ira-sti/world-shared/GDASApp/unittestdata")
+setenv("CRTM_FIX","/gpfs/f6/ira-sti/scratch/Russ.Treadon/GDASApp/fix/crtm/2.4.0")
+setenv("GDASAPP_TESTDATA","/gpfs/f6/ira-sti/scratch/Russ.Treadon/GDASApp/testdata")
+setenv("GDASAPP_UNIT_TEST_DATA_PATH", "/gpfs/f6/ira-sti/scratch/Russ.Treadon/GDASApp/unittestdata")
 
 whatis("Name: ".. "pkgName")
 whatis("Version: ".. "pkgVersion")

The modified paths contain the backgrounds for test_gdasapp_atm_jjob with EE2 compliant filenames.

Run GDASApp ctests with the following results

Test project /gpfs/f6/ira-sti/scratch/Russ.Treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build
        Start 1655: test_gdasapp_C96C48_ufs_hybatmDA
        Start 1636: test_gdasapp_C96C48_hybatmDA
        Start 1679: test_gdasapp_C96_gcafs_cycled
        Start 1694: test_gdasapp_C96C48_hybatmsnowDA
        Start 1715: test_gdasapp_C48mx500_3DVarAOWCDA
        Start 1726: test_gdasapp_C48mx500_hybAOWCDA
        Start 1742: test_gdasapp_setup_atm_jjob_cycled_exp
        Start 1630: test_gdasapp_check_python_norms
        Start 1631: test_gdasapp_check_yaml_keys
        Start 1632: test_gdasapp_jcb_installation
        Start 1633: test_gdasapp_snow_create_ens
        Start 1634: test_gdasapp_snow_imsproc
  1/125 Test #1631: test_gdasapp_check_yaml_keys .............................................   Passed    0.15 sec
        Start 1635: test_gdasapp_convert_bufr_adpsfc_snow
  2/125 Test #1632: test_gdasapp_jcb_installation ............................................   Passed    0.52 sec
...
124/125 Test #1707: test_gdasapp_C96C48_hybatmsnowDA_gdas_fcst_202112201800 ..................   Passed  352.07 sec
125/125 Test #1714: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_fcst_202112201800 ..............   Passed  185.89 sec

100% tests passed, 0 tests failed out of 125

Label Time Summary:
gdasapp    = 13951.02 sec*proc (116 tests)
script     = 13951.02 sec*proc (116 tests)

Total Test time (real) = 1880.85 sec

All GDASApp tests, including select g-w CI, run on Gaea C6 Passed.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor Author

All GDASApp ctests, including select g-w CI, pass on Gaea C6, Hera, Hercules, Orion, Ursa, WCOSS2 (Cactus).

This PR is ready for review

@RussTreadon-NOAA RussTreadon-NOAA marked this pull request as ready for review November 2, 2025 00:11
@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor Author

@CoryMartin-NOAA and @DavidNew-NOAA : The only thing we need to decide is how we want to update the test_gdasapp_atm background filenames on Gaea C6.

Quick options are

  1. update modulefiles/GDAS/gaeac6.intel.lua to point at my Gaea C6 directories. This change can be committed to feature/stable-nightly, or
  2. update the filenames in /gpfs/f6/ira-sti/world-shared/GDASApp (owned by Cory). No change required to GDASApp.

A long term option, 3, is to place directories with the correct filenames in role.jedipara. Before do this we need to add role.jedipara to a few groups. This option will require a future update to paths in modulefiles/GDAS/gaeac6.intel.lua.

What's your preference?

@CoryMartin-NOAA

Copy link
Copy Markdown
Contributor

Let's do 2 and 3, I can change the names here in a few mins on F6.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor Author

Thank you @CoryMartin-NOAA

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Files should be renamed in /gpfs/f6/ira-sti/world-shared/GDASApp/testdata/lowres/

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor Author

Automated GW-GDASApp Testing Results:
Machine: gaeac6

Start: Mon Nov  3 09:05:30 AM EST 2025 on gaea65
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon Nov  3 09:35:15 AM EST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Mon Nov  3 10:05:06 AM EST 2025
Tests: 100% tests passed, 0 tests failed out of 125

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor Author

Gaea C6 GDASApp ctests Passed. Thanks @CoryMartin-NOAA .

Thank you @DavidNew-NOAA and @CoryMartin-NOAA for your reviews and approvals. Merging now.

@RussTreadon-NOAA RussTreadon-NOAA merged commit a0ae1da into develop Nov 3, 2025
65 checks passed
@RussTreadon-NOAA RussTreadon-NOAA deleted the feature/stable-nightly branch November 3, 2025 15:11
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
# Description

This PR contains three sets of changes:
- weekly update of select JEDI hashes
- remove hardwired user name in Hera CI script
- change cube_sphere_grid file names for atm_jjob ctests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gaeac6-GW-RT-Passed hera-GW-RT-Passed Automated testing with global-workflow successful on Hera hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules orion-GW-RT-Passed Automated testing with global-workflow successful on Orion ursa-GW-RT-Passed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants