Skip to content

Refactor marine ensemble recentering script and move to GW from GDASApp#3882

Merged
DavidHuber-NOAA merged 51 commits into
NOAA-EMC:developfrom
DavidNew-NOAA:feature/marine-refactor
Jul 30, 2025
Merged

Refactor marine ensemble recentering script and move to GW from GDASApp#3882
DavidHuber-NOAA merged 51 commits into
NOAA-EMC:developfrom
DavidNew-NOAA:feature/marine-refactor

Conversation

@DavidNew-NOAA
Copy link
Copy Markdown
Contributor

@DavidNew-NOAA DavidNew-NOAA commented Jul 17, 2025

Description

This PR does a couple things:

  1. It moves the marine ensemble recentering script into the Global Workflow from GDASApp and refactors it to fully use the Jedi class, JCB, and wxflow FileHandler.
  2. The ocnanalecen and marineanlletkf will now be enkfgdas rather than gdas jobs.
  3. The ocnanalecen job is renamed marineanlecen
  4. The marineanlfinal will now save yaml files to COMOUT_CONF, like all other jobs do.

Resolves #3801
Resolves #3767

Type of change

  • Bug fix (fixes something broken)
  • [] New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? YES
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? YES (If YES, please add a link to any PRs that are pending.)

How has this been tested?

  • Clone and build on Hera
  • Run C96C48_ufs_hybatmDA, C96C48_hybatmDA, C48mx500_3DVarAOWCDA, C48mx500_hybAOWCDA, and C96C48_hybatmsnowDA

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

@DavidNew-NOAA DavidNew-NOAA changed the title Feature/marine refactor Refactor marine ensemble recentering script and move to GW from GDASApp Jul 17, 2025
@emcbot emcbot added CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed CI-Gaeac6-Running and removed CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 labels Jul 29, 2025
@TerrenceMcGuinness-NOAA
Copy link
Copy Markdown
Collaborator

TerrenceMcGuinness-NOAA commented Jul 29, 2025

The build for gfs_model on Hercules timed out (the uploaded gist of log file got truncated):

/work2/noaa/global/role-global/GFS_CI_CD/HERCULES/CI/3882/global-workflow/sorc/ufs_model.fd/WW3/model/src/w3wavemd.F90(458): remark #6536: All symbols from this module are already visible due to another USE; the ONLY clause will have no effect. Rename clauses, if any, will be honored.   [W3TIMEMD]
    USE W3TIMEMD
--------^
/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/unified-env/install/intel/2021.9.0/cmake-3.23.1-uqaeynr/bin/cmake -E cmake_copy_f90_mod WW3/model/src/mod/wminitmd.mod WW3/model/src/CMakeFiles/ww3_lib.dir/wminitmd.mod.stamp Intel
/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/unified-env/install/intel/2021.9.0/cmake-3.23.1-uqaeynr/bin/cmake -E touch WW3/model/src/CMakeFiles/ww3_lib.dir/wminitmd.F90.o.provides.build
slurmstepd: error: *** JOB 5955406 ON hercules-01-20 CANCELLED AT 2025-07-29T10:25:27 DUE TO TIME LIMIT ***

https://jenkins.epic.oarcloud.noaa.gov/job/global-workflow/job/EMC-Global-Pipeline/view/change-requests/job/PR-3882/4/artifact/logs/gfs_model.log

@DavidNew-NOAA
Copy link
Copy Markdown
Contributor Author

Thanks @TerrenceMcGuinness-NOAA for letting us know

@DavidHuber-NOAA
Copy link
Copy Markdown
Contributor

@DavidNew-NOAA Could you add the following patch to dev/workflow/build_opts.yaml?

diff --git a/dev/workflow/build_opts.yaml b/dev/workflow/build_opts.yaml
index 5f1e88f85..8a4e21953 100644
--- a/dev/workflow/build_opts.yaml
+++ b/dev/workflow/build_opts.yaml
@@ -36,7 +36,7 @@ build:
   gfs_model:
     command: "./build_ufs.sh -e gfs_model.x"
     cores: 12
-    walltime: "00:30:00"
+    walltime: "00:40:00"

   gfs_ww3prepost:
     command: "./build_ww3prepost.sh"
@@ -46,7 +46,7 @@ build:
   gefs_model:
     command: "./build_ufs.sh -w -e gefs_model.x"
     cores: 12
-    walltime: "00:30:00"
+    walltime: "00:40:00"

   gefs_ww3_prepost:
     command: "./build_ww3prepost.sh -w"
@@ -56,12 +56,12 @@ build:
   sfs_model:
     command: "./build_ufs.sh -y -e sfs_model.x"
     cores: 12
-    walltime: "00:30:00"
+    walltime: "00:40:00"

   gcafs_model:
     command: "./build_ufs.sh -a ATMAERO -e gcafs_model.x"
     cores: 12
-    walltime: "00:30:00"
+    walltime: "00:40:00"

   upp:
     command: "./build_upp.sh"

@DavidNew-NOAA
Copy link
Copy Markdown
Contributor Author

@DavidHuber-NOAA patch applied

@DavidHuber-NOAA DavidHuber-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Jul 29, 2025
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Gaeac6-Passed **Bot use only** CI testing on Gaea C6 for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Gaeac6-Running CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Jul 29, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

See that Gaea C6 and Hercules CI passed. Do you want me to set up and run WCOSS2 (Cactus) CI?

I ask because GDASApp develop contains change which require DavidNew-NOAA:feature/marine-refactor to run g-w. Developers don't know this and encounter problems running GDASApp develop with g-w develop. This inconsistency goes away (at least for now) once this PR is merged into g-w develop

@DavidNew-NOAA
Copy link
Copy Markdown
Contributor Author

@RussTreadon-NOAA I don't how much CI is sufficient. If this passes on Gaea and Hercules and manually on Hera, I don't see a reason not to merge. @aerorahul and @DavidHuber-NOAA ?

Copy link
Copy Markdown
Contributor

@DavidHuber-NOAA DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@DavidHuber-NOAA
Copy link
Copy Markdown
Contributor

@DavidNew-NOAA agreed. Merging now.

@DavidHuber-NOAA DavidHuber-NOAA merged commit 6227bc4 into NOAA-EMC:develop Jul 30, 2025
10 of 11 checks passed
@DavidNew-NOAA
Copy link
Copy Markdown
Contributor Author

Thanks @DavidHuber-NOAA !

@DavidNew-NOAA DavidNew-NOAA deleted the feature/marine-refactor branch July 30, 2025 21:12
ypwang19 pushed a commit to ypwang19/global-workflow that referenced this pull request Jul 30, 2025
…tprocess

* upstream/develop:
  Refactor marine ensemble recentering script and move to GW from GDASApp (NOAA-EMC#3882)
DavidNew-NOAA added a commit to NOAA-EMC/GDASApp that referenced this pull request Jan 6, 2026
This PR is a companion of NOAA-EMC/global-workflow#3882 and
#1803 which refactor the the marine ensemble recentering
script and moves it into Global Workflow from GDASApp. The only change
in this repository is the creation of a JCB template for the soca
ensemble handler application
DavidNew-NOAA added a commit to NOAA-EMC/GDASApp that referenced this pull request Jan 16, 2026
# Description

This PR is a companion to NOAA-EMC/global-workflow#3882, which refactors
the marine ensemble recentering script and moves it into Global Workflow
from GDASApp.

The changes in this repository are all updates or additions of YAML
files for staging/saving files required for marine jobs and configuring
JEDI.

# Companion PRs

NOAA-EMC/global-workflow#3882

# Issues

NOAA-EMC/global-workflow#3801
NOAA-EMC/global-workflow#3767

# Automated CI tests to run in Global Workflow

N/A (Already tested CI cases in Global Workflow)
DavidNew-NOAA added a commit to NOAA-EMC/GDASApp that referenced this pull request Jan 16, 2026
# Description

Quick bug fix based on
NOAA-EMC/global-workflow#3882 (comment)

# Companion PRs

NOAA-EMC/global-workflow#3882

# Issues

<!-- Enter any issues referenced or resolved by this PR here. Use
keywords "Resolves" or "Refs".
Resolves #1234
Refs #4321
Refs NOAA-EMC/repo#5678
-->

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [ ] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-Gaeac6-Passed **Bot use only** CI testing on Gaea C6 for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor JEDI marine analysis recentering script and move it into GW from GDASApp Marine LETKF needs to use enkfgdas/enkfgfs RUN, instead of gdas

7 participants