Skip to content

Update JEDI hashes (20250225)#1475

Merged
RussTreadon-NOAA merged 28 commits into
developfrom
feature/stable-nightly
Feb 25, 2025
Merged

Update JEDI hashes (20250225)#1475
RussTreadon-NOAA merged 28 commits into
developfrom
feature/stable-nightly

Conversation

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

@RussTreadon-NOAA RussTreadon-NOAA commented Feb 2, 2025

Description

Update select JEDI hashes on 20250225

Companion PRs

none

Issues

Resolves #1474

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmaerosnowDA
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@RussTreadon-NOAA RussTreadon-NOAA self-assigned this Feb 2, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

This PR will be marked Ready for review once all GDASApp ctests have been run and pass on Hera, Hercules, and Orion and g-w CI passes on WCOSS2.

@RussTreadon-NOAA RussTreadon-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion hercules-GW-RT Queue for automated testing with global-workflow on Hercules labels Feb 2, 2025
@emcbot emcbot added hercules-GW-RT-Running Automated testing with global-workflow running on Hercules orion-GW-RT-Running Automated testing with global-workflow running on Orion hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hercules-GW-RT Queue for automated testing with global-workflow on Hercules orion-GW-RT Queue for automated testing with global-workflow on Orion hera-GW-RT Queue for automated testing with global-workflow on Hera labels Feb 2, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 2, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Sun Feb  2 14:48:35 CST 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sun Feb  2 15:24:43 CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Sun Feb  2 16:24:09 CST 2025
Tests: 100% tests passed, 0 tests failed out of 135

@emcbot emcbot added hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Feb 2, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 2, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Sun Feb  2 20:56:24 UTC 2025 on hfe09
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sun Feb  2 21:42:53 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Sun Feb  2 22:43:09 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 135

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Feb 2, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 2, 2025

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Sun Feb  2 02:51:15 PM CST 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sun Feb  2 03:55:49 PM CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Sun Feb  2 05:38:19 PM CST 2025
Tests: 100% tests passed, 0 tests failed out of 135

@emcbot emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Feb 2, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

WCOSS2 g-w CI

Clone g-w develop at 380946c on Cactus. Update sorc/gdas.cd to feature/stable-nightly at e6eafb0`.

All g-w components successfully build except GDASApp. The GDASApp build fails with

[100%] Linking CXX executable ../../bin/gdas_soca_error_covariance_toolbox.x
cd /lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build/gdas/mains && /apps/ops/test/spack-stack-1.6.0-nco/envs/nco-intel-19.1.3.304/install/intel/19.1.3.304/cmake-3.23.1-chpcsen/bin/cmake -E remove /lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build/bin/gdas_soca_error_covariance_toolbox.x
cd /lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build/gdas/mains && /apps/ops/test/\
spack-stack-1.6.0-nco/envs/nco-intel-19.1.3.304/install/intel/19.1.3.304/cmake-3.23.1-chpcsen/bin/cmake -E cmake_link_script CMakeFiles/gdas_soca_error_covariance_toolbox.x.dir/link.txt --verbose=NO
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: ../../lib/libsaber.so: undefined reference to `atlas::grid::detail::partitioner::TransPartitioner::TransPartitioner()'
make[2]: *** [gdas/mains/CMakeFiles/gdas_soca_error_covariance_toolbox.x.dir/build.make:174: bin/gdas_soca_error_covariance_toolbox.x] Error 1
make[2]: Leaving directory '/lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build'
make[1]: *** [CMakeFiles/Makefile2:28951: gdas/mains/CMakeFiles/gdas_soca_error_covariance_toolbox.x.dir/all] Error 2
make[1]: Leaving directory '/lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build'
make: *** [Makefile:166: all] Error 2

Notice that Hera, Hercules, and Orion use more recent versions of atlas

./hera.intel.lua:load("atlas/0.35.1")
./orion.intel.lua:load("atlas/0.35.1")
./wcoss2.intel.lua:load("atlas/0.35.0")
./hercules.gnu.lua:load("atlas/0.36.0")
./hercules.intel.lua:load("atlas/0.36.0")

Might a more recent version of atlas resolve the
undefined reference to atlas::grid::detail::partitioner::TransPartitioner::TransPartitioner()'
message in libsaber.so?

Manually load GDASApp wcoss2.intel modulefile on Cactus. module spider atlas returns

---------------------------------------------------------------------------------------------------------------------------------
  atlas:
---------------------------------------------------------------------------------------------------------------------------------
     Versions:
        atlas/0.33.0
        atlas/0.35.0 

Unfortunately, atlas/0.35.0 is the most recent version of atlas available in the current installation of spack-stack/1.6.0 on WCOSS2.

Note that saber PR #1001 contains references to TransPartitioner.h in WriteFields.cc.

As a test back up working copy of sorc/saber to hash d51284c6. This is the commit prior to saber PR #1001. feature/stable-nightly successfully builds on Cactus using this saber hash.

We need to reach out to the library time to request installation of a newer version of atlas on WCOSS2. This PR can not move forward until we can build and run feature/stable-nightly on WCOSS2.

Attention: @DavidNew-NOAA , @danholdaway , @CoryMartin-NOAA , @guillaumevernieres

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

WCOSS2 g-w CI
As a test revert sorc/saber back to d51284c6, build GDASApp, and run g-w CI on Cactus. All configurations successfully run to completion.

/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_ATM_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 03 2025 10:05:18    Feb 03 2025 11:15:25
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_3DVarAOWCDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 03 2025 10:05:22    Feb 03 2025 10:20:23
202103250000      Active    Feb 03 2025 10:05:22             -          
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_hybAOWCDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 03 2025 10:05:24    Feb 03 2025 10:20:30
202103250000        Done    Feb 03 2025 10:05:24    Feb 03 2025 11:25:21
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SWA_gefs_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 03 2025 10:05:39    Feb 03 2025 12:01:05
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SW_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 03 2025 10:05:26    Feb 03 2025 11:30:46
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_atm3DVar_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 03 2025 10:05:29    Feb 03 2025 10:20:40
202112210000        Done    Feb 03 2025 10:05:29    Feb 03 2025 12:30:37
202112210600        Done    Feb 03 2025 10:05:29    Feb 03 2025 12:15:35
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 03 2025 10:05:31    Feb 03 2025 10:25:41
202112201800        Done    Feb 03 2025 10:05:31    Feb 03 2025 12:30:41
202112210000        Done    Feb 03 2025 10:05:31    Feb 03 2025 12:20:45
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 03 2025 10:05:33    Feb 03 2025 10:20:49
202112210000        Done    Feb 03 2025 10:05:33    Feb 03 2025 12:05:39
202112210600        Done    Feb 03 2025 10:05:33    Feb 03 2025 12:10:44
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_ufs_hybatmDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202402231800        Done    Feb 03 2025 10:05:36    Feb 03 2025 10:20:53
202402240000        Done    Feb 03 2025 10:05:36    Feb 03 2025 12:53:23
202402240600        Done    Feb 03 2025 10:05:36    Feb 03 2025 12:52:37
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_S2SWA_gefs_replay_ics_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202011010000        Done    Feb 03 2025 10:05:43    Feb 03 2025 10:56:14

@RussTreadon-NOAA RussTreadon-NOAA added the DO NOT MERGE PR is not ready to be merged yet label Feb 3, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

While g-w based GDASAPpp ctests pass on Hera, Hercules, and Orion, we can not build feature/stable-nightly at e6eafb0 on WCOSS2. Starting with saber @ b85ece5 we need at least atlas/0.35.1. Currently the WCOSS2 spack-stack installation only has atlas versions up to 0.35.0.

Mark this PR DO NO MERGE until WCOSS2 spack stack is updated.

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Feb 25, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 25, 2025

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Tue Feb 25 07:10:02 AM CST 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Feb 25 08:20:17 AM CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                  *Failed*
Tests: Failed at Tue Feb 25 09:03:24 AM CST 2025
Tests: 
Tests: see output at /work2/noaa/da/role-da/CI/orion/GDASApp/workflow/PR/1475/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Feb 25, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

Failed to copy g-w PR #3295 config.aeroanlgenb into Orion $HOMEgfs when I reapplied the orion-GW-RT label. Therefore, the following jobs failed (as exepected)

 83/137 Test #2041: test_gdasapp_C96C48_hybatmaerosnowDA_gdas_aeroanlgenb_202112201200 .......***Failed  297.15 sec
 96/137 Test #2048: test_gdasapp_C96C48_hybatmaerosnowDA_gdas_aeroanlinit_202112201800 .......***Failed   96.04 sec
 99/137 Test #2049: test_gdasapp_C96C48_hybatmaerosnowDA_gdas_aeroanlvar_202112201800 ........***Failed  131.65 sec
101/137 Test #2050: test_gdasapp_C96C48_hybatmaerosnowDA_gdas_aeroanlfinal_202112201800 ......***Failed   63.43 sec

Remove orion-GW-RT-Failed label. Reapply orion-GW-RT label.

@RussTreadon-NOAA RussTreadon-NOAA added orion-GW-RT Queue for automated testing with global-workflow on Orion and removed orion-GW-RT-Failed Automated testing with global-workflow failed on Orion labels Feb 25, 2025
@emcbot emcbot added orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed orion-GW-RT Queue for automated testing with global-workflow on Orion labels Feb 25, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

WCOSS2 CI
g-w CI results on Cactus are as follows

/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_ATM_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 25 2025 13:10:26    Feb 25 2025 14:20:33
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_3DVarAOWCDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 25 2025 13:10:28    Feb 25 2025 13:30:37
202103250000        Done    Feb 25 2025 13:10:28    Feb 25 2025 15:07:54
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_hybAOWCDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 25 2025 13:10:30    Feb 25 2025 13:30:40
202103250000        Done    Feb 25 2025 13:10:30    Feb 25 2025 14:30:47
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SWA_gefs_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 25 2025 13:10:31    Feb 25 2025 15:05:32
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SW_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 25 2025 13:10:34    Feb 25 2025 14:29:04
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_atm3DVar_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 25 2025 13:10:35    Feb 25 2025 13:30:52
202112210000        Done    Feb 25 2025 13:10:35    Feb 25 2025 15:35:30
202112210600        Done    Feb 25 2025 13:10:35    Feb 25 2025 15:15:45
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 25 2025 13:10:37    Feb 25 2025 13:30:55
202112201800        Done    Feb 25 2025 13:10:37    Feb 25 2025 15:29:45
202112210000      Active    Feb 25 2025 13:10:37             -          
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 25 2025 13:10:39    Feb 25 2025 13:30:58
202112210000        Done    Feb 25 2025 13:10:39    Feb 25 2025 15:10:44
202112210600        Done    Feb 25 2025 13:10:39    Feb 25 2025 15:10:44
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_ufs_hybatmDA_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202402231800        Done    Feb 25 2025 13:10:41    Feb 25 2025 13:31:02
202402240000        Done    Feb 25 2025 13:10:41    Feb 25 2025 16:10:41
202402240600        Done    Feb 25 2025 13:10:41    Feb 25 2025 15:59:28
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96mx100_S2S_stable-nightly
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
199405010000        Done    Feb 25 2025 13:10:43    Feb 25 2025 14:01:18

All g-w CI cases ran to completion except C96C48_hybatmaerosnowDA. The following jobs died

/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_stable-nightly
202112210000            gdas_snowanl                   181611557                DEAD                   1         2          58.0
202112210000       enkfgdas_esnowanl                   181611559                DEAD                   1         2          79.0

Both jobs died with the error

Traceback (most recent call last):
  File "/lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/ush/imsfv3_scf2ioda.py", line 16, in <module>
    import pyiodaconv.ioda_conv_engines as iconv
  File "/lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/sorc/iodaconv/src/pyiodaconv/ioda_conv_engines.py", line 3, in <module>
    from pyioda import ioda_obs_space as ioda_os
  File "/lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build/lib/python3.10/pyioda/ioda_obs_space/__init__.py", line 4, in <module>
    from pyioda import ioda
  File "/lfs/h2/emc/da/noscrub/russ.treadon/git/global-workflow/stable-nightly/sorc/gdas.cd/build/lib/python3.10/pyioda/ioda/__init__.py", line 8, in <module>
    from ._ioda_python import *
ImportError: /opt/cray/pe/lib64/libmpifort_intel.so.12: undefined symbol: MPIX_Enqueue_send

This undefined error message occurred in the past. The following patch was added to ush/load_ufsda_modules.sh to get past this error

@@ -41,6 +41,7 @@ case "${MACHINE_ID}" in
       # TODO: Add path to GDASApp libraries and cray-mpich as temporary patches
       # TODO: Remove LD_LIBRARY_PATH lines as soon as permanent solutions are available
       export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${HOMEgfs}/sorc/gdas.cd/build/lib"
+      export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/opt/cray/pe/mpich/8.1.19/ofi/intel/19.0/lib"
     fi
     module load "${MODS}/${MACHINE_ID}"
     ncdump=$( command -v ncdump )

Add this patch to working copy of $HOMEgfs/ush/load_ufsda_modules.sh. Rewind and reboot 20211221 00Z gdas_snowanl and enkfgdas_esnowanl. Both jobs successfully ran to completion.

russ.treadon@clogin04:/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_stable-nightly> rocotostat -d C96C48_hybatmaerosnowDA_stable-nightly.db -w C96C48_hybatmaerosnowDA_stable-nightly.xml -c 202112210000 | grep snowanl
202112210000            gdas_snowanl                   181622125           SUCCEEDED                   0         1         182.0
202112210000       enkfgdas_esnowanl                   181623085           SUCCEEDED                   0         1         167.0

and g-w CI case C96C48_hybatmaerosnowDA completed all jobs

russ.treadon@clogin04:/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_stable-nightly> rocotostat -d C96C48_hybatmaerosnowDA_stable-nightly.db -w C96C48_hybatmaerosnowDA_stable-nightly.xml -c all -s
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 25 2025 13:10:37    Feb 25 2025 13:30:55
202112201800        Done    Feb 25 2025 13:10:37    Feb 25 2025 15:29:45
202112210000        Done    Feb 25 2025 13:10:37    Feb 25 2025 17:17:25

The load_ufsda_modules.sh patch was no longer needed after RFC 13644 was implemented 5-6 February 2025. Seems like something changed on Cactus which requires the patch to be added back ... at least for the snowanl jobs.

Attention: @CatherineThomas-NOAA , @ClaraDraper-NOAA , @CoryMartin-NOAA

@emcbot
Copy link
Copy Markdown

emcbot commented Feb 25, 2025

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Tue Feb 25 09:28:15 AM CST 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Feb 25 10:36:55 AM CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Tue Feb 25 12:27:38 PM CST 2025
Tests: 100% tests passed, 0 tests failed out of 137

@emcbot emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Feb 25, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 25, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Tue Feb 25 05:52:54 CST 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Feb 25 06:29:04 CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp
Tests:                                 *SUCCESS*
Tests: Completed at Tue Feb 25 13:49:41 CST 2025
Tests: 100% tests passed, 0 tests failed out of 137

@emcbot emcbot added hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Feb 25, 2025
@RussTreadon-NOAA RussTreadon-NOAA changed the title Update JEDI hashes (20250203) Update JEDI hashes (20250225) Feb 25, 2025
@RussTreadon-NOAA RussTreadon-NOAA marked this pull request as ready for review February 25, 2025 19:59
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

GDASApp CI passed on Hera, Hercules, and Orion. g-w CI passed on WCOSS2 (Cactus) with caveats noted above. This PR is ready for review.

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

Thank you @CoryMartin-NOAA and @DavidNew-NOAA for the quick approvals. Now merging.

@RussTreadon-NOAA RussTreadon-NOAA merged commit 583e630 into develop Feb 25, 2025
@RussTreadon-NOAA RussTreadon-NOAA deleted the feature/stable-nightly branch February 25, 2025 20:09
jspsl pushed a commit that referenced this pull request Mar 26, 2025
Co-authored-by: Anna Shlyaeva <anna.v.shlyaeva@noaa.gov>
danholdaway added a commit that referenced this pull request Mar 26, 2025
* develop:
  added original date (#1570)
  Update OCEAN_BASIN_FILE_ORION path for marine ctest (#1569)
  Fixing window_begin and end in yaml for redating ADT (#1565)
  Save LETKF mean analysis and mean increment for verification (#1566)
  Update JEDI hashes (20250324) (#1567)
  Removes obsolete json files from marine bufr2ioda (#1564)
  Add hybrid 4denvar as a phase3 validation configuration (#1552)
  Bugfix/ocn ecen (#1550)
  Bugfix SSS obs for LETKF (#1560)
  Bugfix ocean ensemble recentering (#1558)
  apply ci path patch only when running g-w ctests (#1554)
  Update JEDI hashes (20250317)  (#1531)
  add option to create_experiment.py in create_exp.sh  (#1546)
  Update jcb hashes and support initial IODA stats capability (#1547)
  Fix GW-CI CTest dependencies (#1534)
  Converts prepoceanobs to use JCB-style obs list and config files (#1527)
  JEDI-based ensemble recentering and analysis calculation (#1488)
  tools for phase3 validation (#1526)
  Update JEDI hashes (20250306) (#1520)
  New jcb-gdas # and insitu obs prep bug fix  (#1523)
  Marine in situ obs update (SSS, moorings, drifters) (#1511)
  Save marine background and analysis spread (#1517)
  Minor build and install updates and ctest cleanup (#1512)
  correct logic for upper loop limit in chem_diagb (#1516)
  Incorporate multi-cycle concatination and separation of variables in post-BUFR observations (#1504)
  Update JEDI hashes (20250225) (#1475)
  Add a depth dependent e-fold scale (#1494)
  Update jcb and da-utils hashes to support land DA. (#1509)
  Add options to inflate the ensemble while recentering (#1501)
  Add components to use jcb yamls for marine letkf job (#1486)
  Add GNSSRO bufr2ioda and json files (#1448)
  Removal of the marine verification tools (#1499)
  update logic to identify MSU machine (#1498)
  IODA Converter with In-Situ Observations Concatenation and ObsError Inflation (#1472)
  Add sea ice recentering (#1478)
  marine vrfy bugfix (#1483)
  add LD_LIBRARY_PATH patch for wcoss2 build (#1484)
  Bugfix for ensemble recenter + add a test (#1477)
  Add code changes for the implementation of the  low-resolution B (#1441)
  update spack-stack path in Hera modulefiles (#1471)
  fixed glider output filename (#1469)
  add gdas_waveinit to C48mx500_3DVarAOWCDA ctest suite (#1470)
  update jcb-gdas hash 20250128 (#1466)
  Time series of omb stats and html doc (#1463)
  Update marine vrfy HTML codes (#1464)
  Automate the verify tools to submit from start pdy to end pdy (#1458)
  Update an offline marine vrfy tool (#1447)
  Update JEDI hashes (20250127) (#1460)
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
Co-authored-by: Anna Shlyaeva <anna.v.shlyaeva@noaa.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hera-GW-RT-Passed Automated testing with global-workflow successful on Hera hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules orion-GW-RT-Passed Automated testing with global-workflow successful on Orion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gdas_aeroanlgenb failure in 20250222 stable-nightly testing Update JEDI hashes (20250225)

6 participants