Skip to content

Update JEDI hashes and land-imsproc repo and hash#1775

Merged
CoryMartin-NOAA merged 11 commits into
developfrom
feature/stable-nightly
Jun 30, 2025
Merged

Update JEDI hashes and land-imsproc repo and hash#1775
CoryMartin-NOAA merged 11 commits into
developfrom
feature/stable-nightly

Conversation

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

@RussTreadon-NOAA RussTreadon-NOAA commented Jun 27, 2025

Description

Weekly update of JEDI hashes.

Included in this week's update is an update to sorc/land-imsproc. The update to sorc/land-imsproc is twofold

  1. change source repository from https://github.com/NOAA-PSL/land-IMS_proc.git to https://github.com/NOAA-EMC/land-SCF_proc.git via a change to .gitmodules
  2. point at land-imsproc @ 19957e5

Companion PRs

none

Issues

none

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmsnowDA
  • C96_gcafs_cycled
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@RussTreadon-NOAA RussTreadon-NOAA self-assigned this Jun 27, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

Attention: @ClaraDraper-NOAA , @yuanxue2870 , @YoulongXia-NOAA, and @CoryMartin-NOAA

test_gdasapp_snow_imsproc fails in the 20250628 stable-nightly run with the message

1988: + python /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py -i IMSscf.20210323.oro_C48.mx100.nc -o ioda.IMSscf.20210323.oro_C48.mx100.nc
1988: Traceback (most recent call last):
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 198, in <module>
1988:     main()
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 188, in main
1988:     ims = imsFV3(args.input)
1988:           ^^^^^^^^^^^^^^^^^^
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 57, in __init__
1988:     self._read()
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 87, in _read
1988:     stid = ncd.variables['stid'][:].ravel()
1988:            ~~~~~~~~~~~~~^^^^^^^^
1988: KeyError: 'stid'
1/1 Test #1988: test_gdasapp_snow_imsproc ........***Failed    4.99 sec

The 20260628 stable-nightly run updated the sorc/iodaconv hash to ioda-converters at d0b4f97. This hash is associated with ioda-converters PR #1668. This PR adds the IMS stationIdentification.

Do we need to update the test data used by test_gdasapp_snow_imsproc or the processing of the data in order for imsfv3_scf2ioda.py to successfully run with ioda-converters at d0b4f97?

I'll poke around and see what I can figure out but you probably know exactly what needs to be fixed on our side so that test_gdasapp_snow_imsproc once again passes.

We can not update the JEDI hashes in GDASApp until this problem is resolved (or we turn off the failing test ... not the recommended solution).

@yuanxue2870
Copy link
Copy Markdown
Contributor

Attention: @ClaraDraper-NOAA , @yuanxue2870 , @YoulongXia-NOAA, and @CoryMartin-NOAA

test_gdasapp_snow_imsproc fails in the 20250628 stable-nightly run with the message

1988: + python /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py -i IMSscf.20210323.oro_C48.mx100.nc -o ioda.IMSscf.20210323.oro_C48.mx100.nc
1988: Traceback (most recent call last):
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 198, in <module>
1988:     main()
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 188, in main
1988:     ims = imsFV3(args.input)
1988:           ^^^^^^^^^^^^^^^^^^
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 57, in __init__
1988:     self._read()
1988:   File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/bundle/gdas/build/bin/imsfv3_scf2ioda.py", line 87, in _read
1988:     stid = ncd.variables['stid'][:].ravel()
1988:            ~~~~~~~~~~~~~^^^^^^^^
1988: KeyError: 'stid'
1/1 Test #1988: test_gdasapp_snow_imsproc ........***Failed    4.99 sec

The 20260628 stable-nightly run updated the sorc/iodaconv hash to ioda-converters at d0b4f97. This hash is associated with ioda-converters PR #1668. This PR adds the IMS stationIdentification.

Do we need to update the test data used by test_gdasapp_snow_imsproc or the processing of the data in order for imsfv3_scf2ioda.py to successfully run with ioda-converters at d0b4f97?

I'll poke around and see what I can figure out but you probably know exactly what needs to be fixed on our side so that test_gdasapp_snow_imsproc once again passes.

We can not update the JEDI hashes in GDASApp until this problem is resolved (or we turn off the failing test ... not the recommended solution).

Hi Russ, Thanks for the info. You are correct that this failed test is related to an update in the ioda-converter yesterday. I will submit a PR to GDASApp next week with new testcase input and output files for this failed test case. Thanks!

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

It looks like we need to pull changes from NOAA-EMC/land-SCF_proc PR #1 into GDASApp.

Currently GDASApp develop points at sorc/land-imsproc @ 7ca1c74. This hash is from https://github.com/NOAA-PSL/land-SCF_proc/tree/develop.

Do we need to update GDASApp to point sorc/land-imsproc at 19957e5 from https://github.com/NOAA-EMC/land-SCF_proc/tree/develop?

I did so as a test in a working copy of GDASApp develop on Hera. When we build GDASApp with sorc/IMSaggregate_mod.f90 from NOAA-EMC/land-SCF_proc PR #1, test_gdasapp_snow_imsproc Passed.

Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/build
    Start 2045: test_gdasapp_snow_imsproc
1/1 Test #2045: test_gdasapp_snow_imsproc ........   Passed    3.20 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   3.88 sec

Is it OK to update the GDASApp sorc/land-imsproc hash to the EMC repo at 19957e5? If yes, I'll do so in GDASApp branch feature/stable-nightly.

@yuanxue2870
Copy link
Copy Markdown
Contributor

It looks like we need to pull changes from NOAA-EMC/land-SCF_proc PR #1 into GDASApp.

Currently GDASApp develop points at sorc/land-imsproc @ 7ca1c74. This hash is from https://github.com/NOAA-PSL/land-SCF_proc/tree/develop.

Do we need to update GDASApp to point sorc/land-imsproc at 19957e5 from https://github.com/NOAA-EMC/land-SCF_proc/tree/develop?

I did so as a test in a working copy of GDASApp develop on Hera. When we build GDASApp with sorc/IMSaggregate_mod.f90 from NOAA-EMC/land-SCF_proc PR #1, test_gdasapp_snow_imsproc Passed.

Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/ss_191/sorc/gdas.cd/build
    Start 2045: test_gdasapp_snow_imsproc
1/1 Test #2045: test_gdasapp_snow_imsproc ........   Passed    3.20 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   3.88 sec

Is it OK to update the GDASApp sorc/land-imsproc hash to the EMC repo at 19957e5? If yes, I'll do so in GDASApp branch feature/stable-nightly.

Yes, exactly. The update in ioda-converter is built upon NOAA-EMC/land-SCF_proc@19957e5. Thanks for finding the solution quickly!

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

Thank you @yuanxue2870 for your replies and confirmation of an acceptable path forward.

@RussTreadon-NOAA RussTreadon-NOAA changed the title Weekly update of JEDI hashes Weekly update of JEDI hashes and land-imsporc repo and hash Jun 28, 2025
@RussTreadon-NOAA RussTreadon-NOAA changed the title Weekly update of JEDI hashes and land-imsporc repo and hash Update JEDI hashes and land-imsproc repo and hash Jun 28, 2025
@RussTreadon-NOAA RussTreadon-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion hercules-GW-RT Queue for automated testing with global-workflow on Hercules labels Jun 28, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

GDASApp CI

Did not include C96_gcafs_cycled in automated CI for two reasons

  1. C96_gcafs_cycled does not yet run on Hercules and Orion
  2. C96_gcafs_cycled fails on Hera due to the g-w configuration for gcafs_aeroanlvar. See comment here for details

@emcbot emcbot added hercules-GW-RT-Running Automated testing with global-workflow running on Hercules orion-GW-RT-Running Automated testing with global-workflow running on Orion hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hercules-GW-RT Queue for automated testing with global-workflow on Hercules orion-GW-RT Queue for automated testing with global-workflow on Orion hera-GW-RT Queue for automated testing with global-workflow on Hera labels Jun 28, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Jun 28, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Sat Jun 28 22:21:18 UTC 2025 on hfe02
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Jun 28 22:57:01 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled
Tests:                                 *SUCCESS*
Tests: Completed at Sat Jun 28 23:47:49 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 140

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Jun 28, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Jun 29, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Sat Jun 28 17:21:22 CDT 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Jun 28 17:54:42 CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled|C96_gcafs_cycled
Tests:                                 *SUCCESS*
Tests: Completed at Sat Jun 28 19:12:50 CDT 2025
Tests: 100% tests passed, 0 tests failed out of 132

@emcbot emcbot added hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Jun 29, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Jun 29, 2025

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Sat Jun 28 05:22:01 PM CDT 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Sat Jun 28 06:16:09 PM CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled|C96_gcafs_cycled
Tests:                                 *SUCCESS*
Tests: Completed at Sat Jun 28 07:39:31 PM CDT 2025
Tests: 100% tests passed, 0 tests failed out of 132

@emcbot emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Jun 29, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

Hera 20250630 stable-nightly run
Install feature/stable-nightly at 57640e3 on Hera inside g-w develop at b60d23e. Locally modify dev/parm/config/gfs/config.resources.HERA as shown below to allow gcafs_aeronanlvar to run on Hera

@@ -20,6 +20,11 @@ case ${step} in
     export tasks_per_node=12
     ;;
 
+  "aeroanlvar")
+    export tasks_per_node=20
+    export memory="96GB"
+    ;;
+  
   "eupd")
     case "${CASE}" in
       "C1152" | "C768")

Run GDASApp ctests with following results:

Test project /scratch1/NCEPDEV/da/Russ.Treadon/CI/GDASApp/stable/20250630/global-workflow/sorc/gdas.cd/build
        Start 2067: test_gdasapp_C96C48_ufs_hybatmDA
        Start 2048: test_gdasapp_C96C48_hybatmDA
        Start 2091: test_gdasapp_C96_gcafs_cycled
        Start 2106: test_gdasapp_C96C48_hybatmsnowDA
        Start 2127: test_gdasapp_C48mx500_3DVarAOWCDA
        Start 2137: test_gdasapp_C48mx500_hybAOWCDA
        Start 2152: test_gdasapp_setup_atm_jjob_cycled_exp
        Start 1649: test_gdasapp_utils_incrhandler
        Start 1650: test_gdasapp_utils_incrhandler_8pes
        Start 1658: test_gdasapp_util_prepdata
        Start 1652: test_gdasapp_utils_hybridweights
        Start 1653: test_gdasapp_utils_setcorscales
  1/155 Test #2152: test_gdasapp_setup_atm_jjob_cycled_exp ...................................   Passed    1.24 sec
        Start 2157: test_gdasapp_atm_jjob_ens_init
  2/155 Test #1658: test_gdasapp_util_prepdata ...............................................   Passed    1.39 sec
        Start 2153: test_gdasapp_atm_jjob_var_init
  3/155 Test #2091: test_gdasapp_C96_gcafs_cycled ............................................   Passed    5.98 sec
        Start 2092: test_gdasapp_C96_gcafs_cycled_gcdas_stage_ic_202112201200
  4/155 Test #2127: test_gdasapp_C48mx500_3DVarAOWCDA ........................................   Passed    7.89 sec
        Start 2128: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_stage_ic_202103241800
  5/155 Test #2137: test_gdasapp_C48mx500_hybAOWCDA ..........................................   Passed    8.89 sec
        Start 2129: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_waveinit_202103241800
  6/155 Test #2048: test_gdasapp_C96C48_hybatmDA .............................................   Passed    9.63 sec
        Start 2049: test_gdasapp_C96C48_hybatmDA_gdas_stage_ic_202112201800
  7/155 Test #2067: test_gdasapp_C96C48_ufs_hybatmDA .........................................   Passed    9.88 sec
        Start 2068: test_gdasapp_C96C48_ufs_hybatmDA_gdas_stage_ic_202402231800
  8/155 Test #2106: test_gdasapp_C96C48_hybatmsnowDA .........................................   Passed   10.07 sec
        Start 2052: test_gdasapp_C96C48_hybatmDA_enkfgdas_stage_ic_202112201800
  9/155 Test #1649: test_gdasapp_utils_incrhandler ...........................................   Passed   10.52 sec
        Start 2071: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_stage_ic_202402231800
 10/155 Test #1653: test_gdasapp_utils_setcorscales ..........................................   Passed   10.51 sec
        Start 2107: test_gdasapp_C96C48_hybatmsnowDA_gdas_stage_ic_202112201200
 11/155 Test #1652: test_gdasapp_utils_hybridweights .........................................   Passed   10.52 sec
        Start 2110: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_stage_ic_202112201200
 12/155 Test #1650: test_gdasapp_utils_incrhandler_8pes ......................................   Passed   10.54 sec
        Start 2138: test_gdasapp_C48mx500_hybAOWCDA_gdas_stage_ic_202103241800
 13/155 Test #2128: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_stage_ic_202103241800 .............   Passed   35.70 sec
        Start 2140: test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_stage_ic_202103241800
 14/155 Test #2049: test_gdasapp_C96C48_hybatmDA_gdas_stage_ic_202112201800 ..................   Passed   35.93 sec
        Start 2050: test_gdasapp_C96C48_hybatmDA_gdas_fcst_202112201800
 15/155 Test #2138: test_gdasapp_C48mx500_hybAOWCDA_gdas_stage_ic_202103241800 ...............   Passed   36.08 sec
        Start 2139: test_gdasapp_C48mx500_hybAOWCDA_gdas_fcst_202103241800
 16/155 Test #2107: test_gdasapp_C96C48_hybatmsnowDA_gdas_stage_ic_202112201200 ..............   Passed   36.28 sec
        Start 2108: test_gdasapp_C96C48_hybatmsnowDA_gdas_fcst_202112201200
 17/155 Test #2157: test_gdasapp_atm_jjob_ens_init ...........................................   Passed   45.62 sec
        Start 2158: test_gdasapp_atm_jjob_ens_letkf
 18/155 Test #2153: test_gdasapp_atm_jjob_var_init ...........................................   Passed   45.64 sec
        Start 2096: test_gdasapp_C96_gcafs_cycled_gcdas_fetch_202112201800
 19/155 Test #2092: test_gdasapp_C96_gcafs_cycled_gcdas_stage_ic_202112201200 ................   Passed   48.28 sec
        Start 2093: test_gdasapp_C96_gcafs_cycled_gcdas_fcst_202112201200
 20/155 Test #2129: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_waveinit_202103241800 .............   Passed   48.36 sec
        Start 2130: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_fcst_202103241800
 21/155 Test #2052: test_gdasapp_C96C48_hybatmDA_enkfgdas_stage_ic_202112201800 ..............   Passed   47.48 sec
        Start 2053: test_gdasapp_C96C48_hybatmDA_enkfgdas_fcst_202112201800
 22/155 Test #2068: test_gdasapp_C96C48_ufs_hybatmDA_gdas_stage_ic_202402231800 ..............   Passed   48.05 sec
        Start 2069: test_gdasapp_C96C48_ufs_hybatmDA_gdas_fcst_202402231800
 23/155 Test #2071: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_stage_ic_202402231800 ..........   Passed   47.49 sec
        Start 2072: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_fcst_202402231800
 24/155 Test #2110: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_stage_ic_202112201200 ..........   Passed   49.14 sec
        Start 2111: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_fcst_202112201200
 25/155 Test #2140: test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_stage_ic_202103241800 ...........   Passed   60.03 sec
        Start 2141: test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_fcst_202103241800
 26/155 Test #2096: test_gdasapp_C96_gcafs_cycled_gcdas_fetch_202112201800 ...................   Passed   69.71 sec
        Start 2104: test_gdasapp_C96_gcafs_cycled_gcdas_prep_emissions_202112201800
 27/155 Test #2104: test_gdasapp_C96_gcafs_cycled_gcdas_prep_emissions_202112201800 ..........   Passed   32.43 sec
        Start 2154: test_gdasapp_atm_jjob_var_run
 28/155 Test #2072: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_fcst_202402231800 ..............   Passed  292.21 sec
        Start 2073: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_epmn_202402231800
 29/155 Test #2053: test_gdasapp_C96C48_hybatmDA_enkfgdas_fcst_202112201800 ..................   Passed  306.31 sec
        Start 2055: test_gdasapp_C96C48_hybatmDA_enkfgdas_epmn_202112201800
 30/155 Test #2111: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_fcst_202112201200 ..............   Passed  315.83 sec
        Start 2113: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_epmn_202112201200
 31/155 Test #2073: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_epmn_202402231800 ..............   Passed   69.16 sec
        Start 1651: test_gdasapp_utils_incrhandler_reproducibility
 32/155 Test #1651: test_gdasapp_utils_incrhandler_reproducibility ...........................   Passed    0.12 sec
        Start 1654: test_gdasapp_utils_diagb
 33/155 Test #2055: test_gdasapp_C96C48_hybatmDA_enkfgdas_epmn_202112201800 ..................   Passed   64.09 sec
        Start 1655: test_gdasapp_utils_enshandler
 34/155 Test #2130: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_fcst_202103241800 .................   Passed  375.27 sec
        Start 2131: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_prepoceanobs_202103250000
 35/155 Test #1654: test_gdasapp_utils_diagb .................................................   Passed   27.88 sec
        Start 2132: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmat_202103250000
 36/155 Test #1655: test_gdasapp_utils_enshandler ............................................   Passed   19.77 sec
        Start 1656: test_gdasapp_utils_socatofv3
 37/155 Test #2139: test_gdasapp_C48mx500_hybAOWCDA_gdas_fcst_202103241800 ...................   Passed  408.34 sec
        Start 2142: test_gdasapp_C48mx500_hybAOWCDA_gdas_prepoceanobs_202103250000
 38/155 Test #1656: test_gdasapp_utils_socatofv3 .............................................   Passed    8.80 sec
        Start 1657: test_gdasapp_util_coding_norms
 39/155 Test #1657: test_gdasapp_util_coding_norms ...........................................   Passed    2.72 sec
        Start 1659: test_gdasapp_util_rads2ioda
 40/155 Test #1659: test_gdasapp_util_rads2ioda ..............................................   Passed    0.63 sec
        Start 1660: test_gdasapp_util_ghrsst2ioda
 41/155 Test #1660: test_gdasapp_util_ghrsst2ioda ............................................   Passed    0.09 sec
        Start 1661: test_gdasapp_util_rtofstmp
 42/155 Test #1661: test_gdasapp_util_rtofstmp ...............................................   Passed    0.98 sec
        Start 1662: test_gdasapp_util_rtofssal
 43/155 Test #1662: test_gdasapp_util_rtofssal ...............................................   Passed    0.75 sec
        Start 1663: test_gdasapp_util_smap2ioda
 44/155 Test #1663: test_gdasapp_util_smap2ioda ..............................................   Passed    0.09 sec
        Start 1664: test_gdasapp_util_smos2ioda
 45/155 Test #1664: test_gdasapp_util_smos2ioda ..............................................   Passed    0.13 sec
        Start 1665: test_gdasapp_util_viirsaod2ioda
 46/155 Test #1665: test_gdasapp_util_viirsaod2ioda ..........................................   Passed    0.46 sec
        Start 1666: test_gdasapp_util_icecabi2ioda
 47/155 Test #1666: test_gdasapp_util_icecabi2ioda ...........................................   Passed    0.10 sec
        Start 1667: test_gdasapp_util_icecamsr2ioda
 48/155 Test #1667: test_gdasapp_util_icecamsr2ioda ..........................................   Passed    0.09 sec
        Start 1668: test_gdasapp_util_icecmirs2ioda
 49/155 Test #1668: test_gdasapp_util_icecmirs2ioda ..........................................   Passed    0.48 sec
        Start 1669: test_gdasapp_util_icecjpssrr2ioda
 50/155 Test #1669: test_gdasapp_util_icecjpssrr2ioda ........................................   Passed    0.17 sec
        Start 1670: test_gdasapp_util_insituall2ioda
 51/155 Test #1670: test_gdasapp_util_insituall2ioda .........................................   Passed    0.13 sec
        Start 2042: test_gdasapp_check_python_norms
 52/155 Test #2042: test_gdasapp_check_python_norms ..........................................   Passed    2.90 sec
        Start 2043: test_gdasapp_check_yaml_keys
 53/155 Test #2043: test_gdasapp_check_yaml_keys .............................................   Passed    0.12 sec
        Start 2044: test_gdasapp_snow_create_ens
 54/155 Test #2044: test_gdasapp_snow_create_ens .............................................   Passed    1.02 sec
        Start 2045: test_gdasapp_snow_imsproc
 55/155 Test #2045: test_gdasapp_snow_imsproc ................................................   Passed    4.98 sec
        Start 2046: test_gdasapp_snow_apply_jediincr
 56/155 Test #2113: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_epmn_202112201200 ..............   Passed   99.88 sec
        Start 2047: test_gdasapp_convert_bufr_adpsfc_snow
 57/155 Test #2047: test_gdasapp_convert_bufr_adpsfc_snow ....................................   Passed    2.57 sec
        Start 2150: test_gdasapp_convert_bufr_adpsfc
 58/155 Test #2046: test_gdasapp_snow_apply_jediincr .........................................   Passed    7.86 sec
        Start 2151: test_gdasapp_convert_gsi_satbias
 59/155 Test #2150: test_gdasapp_convert_bufr_adpsfc .........................................   Passed    3.35 sec
        Start 2164: test_gdasapp_aero_gen_3dvar_yaml
 60/155 Test #2164: test_gdasapp_aero_gen_3dvar_yaml .........................................   Passed    0.35 sec
        Start 2165: test_gdasapp_bufr2ioda_insitu_profile_argo
 61/155 Test #2151: test_gdasapp_convert_gsi_satbias .........................................   Passed    4.82 sec
        Start 2166: test_gdasapp_bufr2ioda_insitu_profile_bathy
 62/155 Test #2166: test_gdasapp_bufr2ioda_insitu_profile_bathy ..............................   Passed    1.04 sec
        Start 2167: test_gdasapp_bufr2ioda_insitu_profile_glider
 63/155 Test #2165: test_gdasapp_bufr2ioda_insitu_profile_argo ...............................   Passed    6.08 sec
        Start 2168: test_gdasapp_bufr2ioda_insitu_profile_tesac
 64/155 Test #2167: test_gdasapp_bufr2ioda_insitu_profile_glider .............................   Passed    1.84 sec
        Start 2169: test_gdasapp_bufr2ioda_insitu_profile_tropical
 65/155 Test #2141: test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_fcst_202103241800 ...............   Passed  385.21 sec
        Start 2143: test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmat_202103250000
 66/155 Test #2169: test_gdasapp_bufr2ioda_insitu_profile_tropical ...........................   Passed    1.26 sec
        Start 2170: test_gdasapp_bufr2ioda_insitu_profile_xbtctd
 67/155 Test #2170: test_gdasapp_bufr2ioda_insitu_profile_xbtctd .............................   Passed    0.86 sec
        Start 2171: test_gdasapp_bufr2ioda_insitu_surface_drifter
 68/155 Test #2171: test_gdasapp_bufr2ioda_insitu_surface_drifter ............................   Passed    1.19 sec
        Start 2172: test_gdasapp_bufr2ioda_insitu_surface_trkob
 69/155 Test #2168: test_gdasapp_bufr2ioda_insitu_profile_tesac ..............................   Passed    4.07 sec
        Start 2173: test_gdasapp_bufr2ioda_insitu_surface_dbuoyb
 70/155 Test #2172: test_gdasapp_bufr2ioda_insitu_surface_trkob ..............................   Passed    1.04 sec
        Start 2174: test_gdasapp_bufr2ioda_insitu_surface_dbuoyb_drifter
 71/155 Test #2173: test_gdasapp_bufr2ioda_insitu_surface_dbuoyb .............................   Passed    1.50 sec
 72/155 Test #2174: test_gdasapp_bufr2ioda_insitu_surface_dbuoyb_drifter .....................   Passed    1.86 sec
 73/155 Test #2050: test_gdasapp_C96C48_hybatmDA_gdas_fcst_202112201800 ......................   Passed  481.18 sec
        Start 2051: test_gdasapp_C96C48_hybatmDA_gdas_atmos_prod_202112201800
        Start 2054: test_gdasapp_C96C48_hybatmDA_enkfgdas_echgres_202112201800
 74/155 Test #2069: test_gdasapp_C96C48_ufs_hybatmDA_gdas_fcst_202402231800 ..................   Passed  468.83 sec
        Start 2070: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmos_prod_202402231800
 75/155 Test #2132: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmat_202103250000 ...........   Passed   80.36 sec
 76/155 Test #2108: test_gdasapp_C96C48_hybatmsnowDA_gdas_fcst_202112201200 ..................   Passed  493.38 sec
        Start 2109: test_gdasapp_C96C48_hybatmsnowDA_gdas_atmos_prod_202112201200
        Start 2112: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_echgres_202112201200
 77/155 Test #2143: test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmat_202103250000 .............   Passed   76.70 sec
 78/155 Test #2054: test_gdasapp_C96C48_hybatmDA_enkfgdas_echgres_202112201800 ...............   Passed   51.74 sec
 79/155 Test #2112: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_echgres_202112201200 ...........   Passed   50.15 sec
 80/155 Test #2158: test_gdasapp_atm_jjob_ens_letkf ..........................................   Passed  554.29 sec
        Start 2159: test_gdasapp_atm_jjob_ens_init_split
 81/155 Test #2159: test_gdasapp_atm_jjob_ens_init_split .....................................   Passed   45.91 sec
        Start 2160: test_gdasapp_atm_jjob_ens_obs
 82/155 Test #2051: test_gdasapp_C96C48_hybatmDA_gdas_atmos_prod_202112201800 ................   Passed  126.90 sec
        Start 2056: test_gdasapp_C96C48_hybatmDA_gdas_prep_202112210000
 83/155 Test #2070: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmos_prod_202402231800 ............   Passed  126.99 sec
        Start 2074: test_gdasapp_C96C48_ufs_hybatmDA_gdas_prep_202402240000
 84/155 Test #2154: test_gdasapp_atm_jjob_var_run ............................................   Passed  522.25 sec
        Start 2155: test_gdasapp_atm_jjob_var_inc
 85/155 Test #2109: test_gdasapp_C96C48_hybatmsnowDA_gdas_atmos_prod_202112201200 ............   Passed  141.36 sec
        Start 2114: test_gdasapp_C96C48_hybatmsnowDA_gdas_prep_202112201800
 86/155 Test #2160: test_gdasapp_atm_jjob_ens_obs ............................................   Passed   42.16 sec
        Start 2161: test_gdasapp_atm_jjob_ens_sol
 87/155 Test #2155: test_gdasapp_atm_jjob_var_inc ............................................   Passed   42.15 sec
        Start 2156: test_gdasapp_atm_jjob_var_final
 88/155 Test #2142: test_gdasapp_C48mx500_hybAOWCDA_gdas_prepoceanobs_202103250000 ...........   Passed  273.94 sec
        Start 2145: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlinit_202103250000
        Start 2144: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlletkf_202103250000
 89/155 Test #2131: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_prepoceanobs_202103250000 .........   Passed  297.98 sec
        Start 2133: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlinit_202103250000
 90/155 Test #2161: test_gdasapp_atm_jjob_ens_sol ............................................   Passed   42.27 sec
        Start 2162: test_gdasapp_atm_jjob_ens_inc
 91/155 Test #2156: test_gdasapp_atm_jjob_var_final ..........................................   Passed   42.15 sec
 92/155 Test #2162: test_gdasapp_atm_jjob_ens_inc ............................................   Passed   42.15 sec
        Start 2163: test_gdasapp_atm_jjob_ens_final
 93/155 Test #2145: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlinit_202103250000 ..........   Passed   61.37 sec
        Start 2146: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlvar_202103250000
 94/155 Test #2133: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlinit_202103250000 ........   Passed   62.00 sec
        Start 2134: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlvar_202103250000
 95/155 Test #2093: test_gdasapp_C96_gcafs_cycled_gcdas_fcst_202112201200 ....................   Passed  741.41 sec
        Start 2094: test_gdasapp_C96_gcafs_cycled_gcdas_atmos_prod_202112201200
        Start 2095: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlgenb_202112201200
        Start 2097: test_gdasapp_C96_gcafs_cycled_gcdas_offlineanl_202112201800
 96/155 Test #2163: test_gdasapp_atm_jjob_ens_final ..........................................   Passed   42.14 sec
 97/155 Test #2146: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlvar_202103250000 ...........   Passed   75.11 sec
        Start 2148: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000
 98/155 Test #2134: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlvar_202103250000 .........   Passed   77.05 sec
        Start 2135: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000
 99/155 Test #2144: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlletkf_202103250000 .........   Passed  143.11 sec
        Start 2147: test_gdasapp_C48mx500_hybAOWCDA_gdas_ocnanalecen_202103250000
100/155 Test #2094: test_gdasapp_C96_gcafs_cycled_gcdas_atmos_prod_202112201200 ..............   Passed  104.24 sec
        Start 2103: test_gdasapp_C96_gcafs_cycled_gcdas_prepobsaero_202112201800
101/155 Test #2095: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlgenb_202112201200 .............   Passed  120.19 sec
102/155 Test #2148: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000 .........   Passed   62.52 sec
        Start 2149: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000
103/155 Test #2135: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000 .......   Passed   63.44 sec
        Start 2136: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000
104/155 Test #2147: test_gdasapp_C48mx500_hybAOWCDA_gdas_ocnanalecen_202103250000 ............   Passed   72.57 sec
105/155 Test #2103: test_gdasapp_C96_gcafs_cycled_gcdas_prepobsaero_202112201800 .............   Passed   59.49 sec
        Start 2100: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlinit_202112201800
106/155 Test #2114: test_gdasapp_C96C48_hybatmsnowDA_gdas_prep_202112201800 ..................   Passed  278.49 sec
        Start 2115: test_gdasapp_C96C48_hybatmsnowDA_gdas_anal_202112201800
        Start 2120: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_eobs_202112201800
        Start 2116: test_gdasapp_C96C48_hybatmsnowDA_gdas_snowanl_202112201800
107/155 Test #2056: test_gdasapp_C96C48_hybatmDA_gdas_prep_202112210000 ......................   Passed  319.22 sec
        Start 2061: test_gdasapp_C96C48_hybatmDA_enkfgdas_eobs_202112210000
        Start 2057: test_gdasapp_C96C48_hybatmDA_gdas_anal_202112210000
108/155 Test #2149: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000 .........   Passed   58.40 sec
109/155 Test #2136: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000 .......   Passed   62.29 sec
110/155 Test #2097: test_gdasapp_C96_gcafs_cycled_gcdas_offlineanl_202112201800 ..............   Passed  221.04 sec
        Start 2098: test_gdasapp_C96_gcafs_cycled_gcdas_sfcanl_202112201800
111/155 Test #2100: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlinit_202112201800 .............   Passed   98.80 sec
        Start 2101: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlvar_202112201800
112/155 Test #2116: test_gdasapp_C96C48_hybatmsnowDA_gdas_snowanl_202112201800 ...............   Passed  109.41 sec
        Start 2124: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_esnowanl_202112201800
113/155 Test #2098: test_gdasapp_C96_gcafs_cycled_gcdas_sfcanl_202112201800 ..................   Passed   58.54 sec
114/155 Test #2074: test_gdasapp_C96C48_ufs_hybatmDA_gdas_prep_202402240000 ..................   Passed  441.77 sec
        Start 2075: test_gdasapp_C96C48_ufs_hybatmDA_gdas_prepatmiodaobs_202402240000
115/155 Test #2124: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_esnowanl_202112201800 ..........   Passed  104.54 sec
116/155 Test #2101: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlvar_202112201800 ..............   Passed  249.44 sec
        Start 2102: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlfinal_202112201800
117/155 Test #2102: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlfinal_202112201800 ............   Passed   72.44 sec
        Start 2105: test_gdasapp_C96_gcafs_cycled_gcdas_fcst_202112201800
118/155 Test #2120: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_eobs_202112201800 ..............   Passed  465.70 sec
        Start 2121: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_ediag_202112201800
119/155 Test #2061: test_gdasapp_C96C48_hybatmDA_enkfgdas_eobs_202112210000 ..................   Passed  476.84 sec
        Start 2062: test_gdasapp_C96C48_hybatmDA_enkfgdas_ediag_202112210000
120/155 Test #2075: test_gdasapp_C96C48_ufs_hybatmDA_gdas_prepatmiodaobs_202402240000 ........   Passed  396.07 sec
        Start 2083: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlinit_202402240000
        Start 2076: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlinit_202402240000
121/155 Test #2083: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlinit_202402240000 .....   Passed   69.97 sec
        Start 2084: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlobs_202402240000
122/155 Test #2076: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlinit_202402240000 ............   Passed   70.88 sec
        Start 2077: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlvar_202402240000
123/155 Test #2121: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_ediag_202112201800 .............   Passed  171.50 sec
        Start 2122: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_eupd_202112201800
124/155 Test #2062: test_gdasapp_C96C48_hybatmDA_enkfgdas_ediag_202112210000 .................   Passed  184.58 sec
        Start 2063: test_gdasapp_C96C48_hybatmDA_enkfgdas_eupd_202112210000
125/155 Test #2122: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_eupd_202112201800 ..............   Passed  154.25 sec
126/155 Test #2063: test_gdasapp_C96C48_hybatmDA_enkfgdas_eupd_202112210000 ..................   Passed  168.88 sec
127/155 Test #2057: test_gdasapp_C96C48_hybatmDA_gdas_anal_202112210000 ......................   Passed  833.29 sec
        Start 2058: test_gdasapp_C96C48_hybatmDA_gdas_sfcanl_202112210000
128/155 Test #2058: test_gdasapp_C96C48_hybatmDA_gdas_sfcanl_202112210000 ....................   Passed  101.30 sec
        Start 2059: test_gdasapp_C96C48_hybatmDA_gdas_analcalc_202112210000
        Start 2060: test_gdasapp_C96C48_hybatmDA_gdas_fcst_202112210000
129/155 Test #2115: test_gdasapp_C96C48_hybatmsnowDA_gdas_anal_202112201800 ..................   Passed  966.08 sec
        Start 2117: test_gdasapp_C96C48_hybatmsnowDA_gdas_sfcanl_202112201800
130/155 Test #2117: test_gdasapp_C96C48_hybatmsnowDA_gdas_sfcanl_202112201800 ................   Passed   74.98 sec
        Start 2118: test_gdasapp_C96C48_hybatmsnowDA_gdas_analcalc_202112201800
        Start 2119: test_gdasapp_C96C48_hybatmsnowDA_gdas_fcst_202112201800
131/155 Test #2059: test_gdasapp_C96C48_hybatmDA_gdas_analcalc_202112210000 ..................   Passed   95.86 sec
        Start 2064: test_gdasapp_C96C48_hybatmDA_enkfgdas_ecmn_202112210000
        Start 2065: test_gdasapp_C96C48_hybatmDA_enkfgdas_esfc_202112210000
132/155 Test #2084: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlobs_202402240000 ......   Passed  511.09 sec
        Start 2085: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlsol_202402240000
133/155 Test #2064: test_gdasapp_C96C48_hybatmDA_enkfgdas_ecmn_202112210000 ..................   Passed   76.15 sec
134/155 Test #2118: test_gdasapp_C96C48_hybatmsnowDA_gdas_analcalc_202112201800 ..............   Passed   85.32 sec
        Start 2123: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_ecmn_202112201800
        Start 2125: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_esfc_202112201800
135/155 Test #2123: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_ecmn_202112201800 ..............   Passed   63.43 sec
136/155 Test #2065: test_gdasapp_C96C48_hybatmDA_enkfgdas_esfc_202112210000 ..................   Passed  172.74 sec
        Start 2066: test_gdasapp_C96C48_hybatmDA_enkfgdas_fcst_202112210000
137/155 Test #2085: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlsol_202402240000 ......   Passed  117.79 sec
        Start 2086: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlfv3inc_202402240000
138/155 Test #2125: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_esfc_202112201800 ..............   Passed  142.98 sec
        Start 2126: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_fcst_202112201800
139/155 Test #2086: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlfv3inc_202402240000 ...   Passed   75.45 sec
        Start 2087: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlfinal_202402240000
140/155 Test #2077: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlvar_202402240000 .............   Passed  708.05 sec
        Start 2078: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlfv3inc_202402240000
141/155 Test #2078: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlfv3inc_202402240000 ..........   Passed   60.83 sec
        Start 2079: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlfinal_202402240000
142/155 Test #2105: test_gdasapp_C96_gcafs_cycled_gcdas_fcst_202112201800 ....................   Passed  954.91 sec
        Start 2099: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlgenb_202112201800
143/155 Test #2099: test_gdasapp_C96_gcafs_cycled_gcdas_aeroanlgenb_202112201800 .............   Passed  123.50 sec
144/155 Test #2060: test_gdasapp_C96C48_hybatmDA_gdas_fcst_202112210000 ......................   Passed  575.76 sec
145/155 Test #2079: test_gdasapp_C96C48_ufs_hybatmDA_gdas_atmanlfinal_202402240000 ...........   Passed  153.38 sec
        Start 2080: test_gdasapp_C96C48_ufs_hybatmDA_gdas_sfcanl_202402240000
        Start 2081: test_gdasapp_C96C48_ufs_hybatmDA_gdas_analcalc_fv3jedi_202402240000
146/155 Test #2087: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_atmensanlfinal_202402240000 ....   Passed  249.32 sec
        Start 2088: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_ecen_fv3jedi_202402240000
        Start 2089: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_esfc_202402240000
147/155 Test #2066: test_gdasapp_C96C48_hybatmDA_enkfgdas_fcst_202112210000 ..................   Passed  361.85 sec
148/155 Test #2080: test_gdasapp_C96C48_ufs_hybatmDA_gdas_sfcanl_202402240000 ................   Passed   58.83 sec
        Start 2082: test_gdasapp_C96C48_ufs_hybatmDA_gdas_fcst_202402240000
149/155 Test #2081: test_gdasapp_C96C48_ufs_hybatmDA_gdas_analcalc_fv3jedi_202402240000 ......   Passed   74.26 sec
150/155 Test #2119: test_gdasapp_C96C48_hybatmsnowDA_gdas_fcst_202112201800 ..................   Passed  577.43 sec
151/155 Test #2126: test_gdasapp_C96C48_hybatmsnowDA_enkfgdas_fcst_202112201800 ..............   Passed  355.16 sec
152/155 Test #2088: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_ecen_fv3jedi_202402240000 ......   Passed   71.31 sec
153/155 Test #2089: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_esfc_202402240000 ..............   Passed  132.52 sec
        Start 2090: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_fcst_202402240000
154/155 Test #2090: test_gdasapp_C96C48_ufs_hybatmDA_enkfgdas_fcst_202402240000 ..............   Passed  300.19 sec
155/155 Test #2082: test_gdasapp_C96C48_ufs_hybatmDA_gdas_fcst_202402240000 ..................   Passed  465.14 sec

100% tests passed, 0 tests failed out of 155

Label Time Summary:
gdas-utils    =   8.21 sec*proc (14 tests)
gdasapp       = 21515.55 sec*proc (114 tests)
script        = 21523.76 sec*proc (128 tests)

Total Test time (real) = 3009.59 sec

All tests, including GCAFS, run to completion.

@RussTreadon-NOAA RussTreadon-NOAA marked this pull request as ready for review June 30, 2025 13:32
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

This PR is ready for review

Copy link
Copy Markdown
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @RussTreadon-NOAA, for updating the hashes, particularly the IMS snow changes

Copy link
Copy Markdown
Contributor

@yuanxue2870 yuanxue2870 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thank you, Russ!

@CoryMartin-NOAA CoryMartin-NOAA merged commit c70e0f9 into develop Jun 30, 2025
23 checks passed
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/stable-nightly branch June 30, 2025 13:48
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor Author

Thank you @CoryMartin-NOAA for merging!

@CoryMartin-NOAA CoryMartin-NOAA restored the feature/stable-nightly branch June 30, 2025 13:49
@RussTreadon-NOAA RussTreadon-NOAA deleted the feature/stable-nightly branch June 30, 2025 14:48
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
# Description

Weekly update of JEDI hashes.  

Included in this week's update is an update to `sorc/land-imsproc`. The
update to `sorc/land-imsproc` is twofold
1. change source repository from
https://github.com/NOAA-PSL/land-IMS_proc.git to
https://github.com/NOAA-EMC/land-SCF_proc.git via a change to
`.gitmodules`
2. point at [land-imsproc @
19957e5](https://github.com/NOAA-EMC/land-SCF_proc/tree/19957e5160f4793076d81549ffe88c17e1377583)

# Companion PRs

none

# Issues

none

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [x] atm_jjob <!-- JEDI atm single cycle DA !-->
- [x] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [x] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [x] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [x] C96C48_hybatmDA <!-- GSI atm cycled DA !-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hera-GW-RT-Passed Automated testing with global-workflow successful on Hera hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules orion-GW-RT-Passed Automated testing with global-workflow successful on Orion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants