Switch to using C++ IMS SCF preprocessor by CoryMartin-NOAA · Pull Request #4000 · NOAA-EMC/global-workflow

CoryMartin-NOAA · 2025-09-03T12:54:53Z

Description

This PR switches the snow_analysis and snowens_analysis tasks from using the Fortran + Python IMS SCF preprocessing workflow to a C++ OOPS executable.

Closes #3980

First step towards #3933

Type of change

Bug fix (fixes something broken)
New feature (adds functionality)
Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? YES
- EMC verif-global
- GDAS - Already merged
- GFS-utils
- GSI
- GSI-monitor
- GSI-utils
- UFS-utils
- UFS-weather-model
- wxflow

How has this been tested?

Cycled test on GaeaC6

Checklist

…oaa/global-workflow into feature/ims-scf-cpp

aerorahul

lgtm.
I haven't paid much attention here, but it seems to me that most methods in snow_analysis.py and snowens_analysis.py are repetitive, and could be combined/consolidated to reduce burden.

CoryMartin-NOAA · 2025-09-03T13:03:51Z

@aerorahul agreed, in the current configuration. If we do EDA or EnKF for snow in the future, would be better to have them remain separated though.

ClaraDraper-NOAA · 2025-09-03T16:10:52Z

@aerorahul agreed, in the current configuration. If we do EDA or EnKF for snow in the future, would be better to have them remain separated though.

ClaraDraper-NOAA · 2025-09-03T17:09:30Z

For people's awareness:

Note that I'm working on updating the GHCN snow depth obs to the global_workflow. The GHCN IODA converter will need a similar treatment.

Additionally, I'm also working on a simplification to the snow_analysis task, which will result in moving the prep_IMS call (and some changes in the paths).

DavidNew-NOAA

Looks good. Thanks for the changes

ClaraDraper-NOAA

lgtm

DavidNew-NOAA · 2025-09-04T15:16:23Z

@aerorahul @DavidHuber-NOAA If there are no objections, can we get CI going for this PR? I have a PR that needs to integrate the changes from this one to move forward.

aerorahul · 2025-09-04T15:20:40Z

No objections from me. Please go ahead.

DavidHuber-NOAA · 2025-09-04T16:34:23Z

Launching tests on C6.

DavidHuber-NOAA · 2025-09-04T19:17:21Z

I accidentally killed the GitLab pipeline. I'll finish running the tests manually on C6.

DavidNew-NOAA · 2025-09-04T22:06:59Z

CI passes on Hera

******** C48_ATM ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:54:04
******** C48mx500_3DVarAOWCDA ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:14:04
202103250000        Done    Sep 04 2025 18:55:13    Sep 04 2025 20:44:09
******** C48mx500_hybAOWCDA ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:14:04
202103250000        Done    Sep 04 2025 18:55:13    Sep 04 2025 20:04:03
******** C48_S2SWA_gefs ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:56:04
******** C48_S2SW ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:52:04
******** C96_atm3DVar ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:12:05
202112210000        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:16:03
202112210600        Done    Sep 04 2025 18:55:13    Sep 04 2025 20:52:07
******** C96C48_hybatmDA ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:14:04
202112210000        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:14:04
202112210600        Done    Sep 04 2025 18:55:13    Sep 04 2025 20:42:06
******** C96C48_hybatmsnowDA ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:12:04
202112201800        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:08:03
202112210000        Done    Sep 04 2025 18:55:13    Sep 04 2025 20:42:06
******** C96C48_hybatmsoilDA ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202205150600        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:14:04
202205151200        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:24:05
202205151800        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:10:12
******** C96C48mx500_S2SW_cyc_gfs ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:14:04
202112201800        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:20:12
202112210000        Done    Sep 04 2025 18:55:13    Sep 04 2025 22:02:04
202112211800        Done    Sep 04 2025 19:16:04    Sep 04 2025 21:58:04
******** C96C48_ufs_hybatmDA ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202402231800        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:14:04
202402240000        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:30:14
202402240600        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:02:04
******** C96_gcafs_cycled ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:14:04
202112201800        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:14:04
202112210000        Done    Sep 04 2025 18:55:13    Sep 04 2025 20:04:03
******** C96_gcafs_cycled_noDA ********
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:12:04
202112201800        Done    Sep 04 2025 18:55:13    Sep 04 2025 21:06:04
202112210000        Done    Sep 04 2025 18:55:13    Sep 04 2025 19:52:05

TerrenceMcGuinness-NOAA · 2025-09-04T23:39:36Z

@DavidHuber-NOAA I restarted the Pipeline. Do not update the labels.
@DavidNew-NOAA Nice formatting on your self test. We can add that to our output if you prefer.

Oh they finished right off. So you guys finished the cases in place?
/gpfs/f6/drsa-precip3/world-shared/global/CI/GITLAB/pr_cases_4000_8a9ee157_3963/RUNTESTS/EXPDIR

DavidHuber-NOAA · 2025-09-05T11:24:40Z

Oh they finished right off. So you guys finished the cases in place? /gpfs/f6/drsa-precip3/world-shared/global/CI/GITLAB/pr_cases_4000_8a9ee157_3963/RUNTESTS/EXPDIR

Yes: #4000 (comment)

DavidHuber-NOAA · 2025-09-05T11:25:29Z

Thanks for relaunching the pipeline @TerrenceMcGuinness-NOAA. I forgot we can do that now.

DavidHuber-NOAA · 2025-09-05T12:29:24Z

All tests passed. Merging.

PR #4000 switched to using the C++ IMS code and in that we changed how that part of the snow analysis job is called, in this, I introduced a bug that would arise if the IMS input file was missing. This fixes this. Resolves #3980

CoryMartin-NOAA added 6 commits August 27, 2025 09:59

commit before meeting

fcb8f1b

end of day commit, still need to do snowens

617773e

update before meeting

6b11139

bugfixes

2a0d3ba

Merge branch 'develop' into feature/ims-scf-cpp

25be9fc

Merge branch 'feature/ims-scf-cpp' of https://github.com/corymartin-n…

fd921e5

…oaa/global-workflow into feature/ims-scf-cpp

CoryMartin-NOAA requested review from ClaraDraper-NOAA and DavidNew-NOAA September 3, 2025 12:54

CoryMartin-NOAA self-assigned this Sep 3, 2025

CoryMartin-NOAA requested review from aerorahul and jiaruidong2017 as code owners September 3, 2025 12:54

CoryMartin-NOAA added JEDI Feature development to support JEDI-based DA GFS labels Sep 3, 2025

aerorahul previously approved these changes Sep 3, 2025

View reviewed changes

jiaruidong2017 previously approved these changes Sep 3, 2025

View reviewed changes

DavidNew-NOAA reviewed Sep 3, 2025

View reviewed changes

Comment thread ush/python/pygfs/task/snow_analysis.py

Comment thread ush/python/pygfs/task/snowens_analysis.py

ClaraDraper-NOAA closed this Sep 3, 2025

ClaraDraper-NOAA reopened this Sep 3, 2025

ClaraDraper-NOAA mentioned this pull request Sep 3, 2025

Update GHCN snow depth assimilation #4004

Open

CoryMartin-NOAA added 2 commits September 3, 2025 13:38

Merge branch 'NOAA-EMC:develop' into feature/ims-scf-cpp

ed6b6bc

address comments but not yet tested

603754b

CoryMartin-NOAA dismissed stale reviews from aerorahul and jiaruidong2017 via 603754b September 3, 2025 17:51

CoryMartin-NOAA requested a review from DavidNew-NOAA September 3, 2025 18:04

DavidNew-NOAA approved these changes Sep 3, 2025

View reviewed changes

ClaraDraper-NOAA approved these changes Sep 3, 2025

View reviewed changes

DavidNew-NOAA mentioned this pull request Sep 4, 2025

Move Global Workflow parm/gdas files into GDASApp parm NOAA-EMC/GDASApp#1878

Merged

jiaruidong2017 approved these changes Sep 4, 2025

View reviewed changes

emcbot added CI-Gaeac6-Passed **Bot use only** CI testing on Gaea C6 for this PR has completed successfully and removed CI-Gaeac6-Running labels Sep 4, 2025

DavidHuber-NOAA added the CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully label Sep 5, 2025

Merge branch 'develop' into feature/ims-scf-cpp

467be4f

DavidHuber-NOAA merged commit cf34da5 into NOAA-EMC:develop Sep 5, 2025
5 checks passed

CoryMartin-NOAA mentioned this pull request Sep 26, 2025

Bugfixes from IMS C++ code additions #4105

Merged

22 tasks

Conversation

CoryMartin-NOAA commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Change characteristics

How has this been tested?

Checklist

Uh oh!

aerorahul left a comment

Choose a reason for hiding this comment

Uh oh!

CoryMartin-NOAA commented Sep 3, 2025

Uh oh!

Uh oh!

Uh oh!

ClaraDraper-NOAA commented Sep 3, 2025

Uh oh!

ClaraDraper-NOAA commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidNew-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

ClaraDraper-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

DavidNew-NOAA commented Sep 4, 2025

Uh oh!

aerorahul commented Sep 4, 2025

Uh oh!

DavidHuber-NOAA commented Sep 4, 2025

Uh oh!

DavidHuber-NOAA commented Sep 4, 2025

Uh oh!

DavidNew-NOAA commented Sep 4, 2025

Uh oh!

TerrenceMcGuinness-NOAA commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidHuber-NOAA commented Sep 5, 2025

Uh oh!

DavidHuber-NOAA commented Sep 5, 2025

Uh oh!

DavidHuber-NOAA commented Sep 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

CoryMartin-NOAA commented Sep 3, 2025 •

edited

Loading

ClaraDraper-NOAA commented Sep 3, 2025 •

edited

Loading

TerrenceMcGuinness-NOAA commented Sep 4, 2025 •

edited

Loading