Skip to content

Correct parametric and ensemble background error statistics filenames in marine DA#4120

Merged
DavidHuber-NOAA merged 9 commits into
NOAA-EMC:developfrom
AndrewEichmann-NOAA:bugfix/update-marine-da-output
Oct 7, 2025
Merged

Correct parametric and ensemble background error statistics filenames in marine DA#4120
DavidHuber-NOAA merged 9 commits into
NOAA-EMC:developfrom
AndrewEichmann-NOAA:bugfix/update-marine-da-output

Conversation

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Contributor

@AndrewEichmann-NOAA AndrewEichmann-NOAA commented Oct 2, 2025 •
edited
Loading

Description

This PR will preserve correct filenames for the marine DA parametric and ensemble background error statistics files and archive both.

Resolves #4119

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

How has this been tested?

C48mx500_3DVarAOWCDA and C48mx500_hybAOWCDA in full cycle tests (not just GDASApp ctests), the latter with nens=3 on Ursa

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

@AndrewEichmann-NOAA AndrewEichmann-NOAA mentioned this pull request Oct 2, 2025
Merged
7 tasks
@AndrewEichmann-NOAA AndrewEichmann-NOAA marked this pull request as ready for review October 2, 2025 18:27
guillaumevernieres pushed a commit to NOAA-EMC/GDASApp that referenced this pull request Oct 2, 2025
@AndrewEichmann-NOAA
break out ens and parametric bkgerr stats (#1922)
2953f31 
# Description

Fixes scrambling of ens and parametric bkgerr stats

# Companion PRs

Requires NOAA-EMC/jcb-gdas#186 and pending g-w
PR (NOAA-EMC/global-workflow#4120)

<!-- Enter links to any companion PRs here. -->

# Issues

Resolves, with jcb-gdas and global-workflow PRs,
#1921

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [x] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@AndrewEichmann-NOAA AndrewEichmann-NOAA added the blocked Issue is currently being blocked by another issue. Include blocking issue # in description label Oct 3, 2025
@RussTreadon-NOAA RussTreadon-NOAA mentioned this pull request Oct 3, 2025
Merged
7 tasks
@AndrewEichmann-NOAA AndrewEichmann-NOAA mentioned this pull request Oct 3, 2025
Merged
7 tasks
@AndrewEichmann-NOAA AndrewEichmann-NOAA removed the blocked Issue is currently being blocked by another issue. Include blocking issue # in description label Oct 3, 2025
DavidNew-NOAA
DavidNew-NOAA previously approved these changes Oct 3, 2025
View reviewed changes
AndrewEichmann-NOAA added a commit to NOAA-EMC/GDASApp that referenced this pull request Oct 3, 2025
@AndrewEichmann-NOAA
Add jinja yaml template for moving (renaming) bkgerr stat files (#1928)
fe812db 
# Description

Adds jinja yaml template for moving (renaming) bkgerr stat files in bmat
task

# Companion PRs

Will need NOAA-EMC/global-workflow#4120 to
actually work

# Issues

Helps fix #1921

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [x] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
guillaumevernieres
guillaumevernieres approved these changes Oct 3, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

RussTreadon-NOAA commented Oct 3, 2025

GDASApp ctests

Installed AndrewEichmann-NOAA:bugfix/update-marine-da-output at 33fee72 with sorc/gdas.cd replaced by GDASApp branch feature/stable-nightly at a8d7dd8. This feature/stable-nightly hash is in sync with GDASApp develop. It differs from GDASApp develop in that it can build and run on Gaea C6 and it updates the hash for select JEDI submodules.

Run GDASApp ctests on Gaea C6, Hera, Hercules, Orion, Ursa, and WCOSS2 (Cactus). GDASApp ctests include select jobs from g-w CI. All GDASApp tests Passed on these machines.

Below are the g-w CI marine DA jobs run as part of the GDASApp ctest suite (output taken from Ursa)

  8/133 Test #1718: test_gdasapp_C48mx500_3DVarAOWCDA ........................................   Passed    3.78 sec
  9/133 Test #1729: test_gdasapp_C48mx500_hybAOWCDA ..........................................   Passed    4.25 sec
 20/133 Test #1719: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_stage_ic_202103241800 .............   Passed   28.49 sec
 21/133 Test #1730: test_gdasapp_C48mx500_hybAOWCDA_gdas_stage_ic_202103241800 ...............   Passed   28.24 sec
 24/133 Test #1720: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_waveinit_202103241800 .............   Passed   40.28 sec
 29/133 Test #1732: test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_stage_ic_202103241800 ...........   Passed   26.80 sec
 33/133 Test #1721: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_fcst_202103241800 .................   Passed  212.84 sec
 35/133 Test #1731: test_gdasapp_C48mx500_hybAOWCDA_gdas_fcst_202103241800 ...................   Passed  217.03 sec
 37/133 Test #1723: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmatinit_202103250000 .......   Passed   27.66 sec
 45/133 Test #1722: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_prepoceanobs_202103250000 .........   Passed   40.14 sec
 51/133 Test #1733: test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_fcst_202103241800 ...............   Passed  244.45 sec
 54/133 Test #1724: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmat_202103250000 ...........   Passed   40.50 sec
 55/133 Test #1735: test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmatinit_202103250000 .........   Passed   26.69 sec
 57/133 Test #1725: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlinit_202103250000 ........   Passed   28.24 sec
 58/133 Test #1739: test_gdasapp_C48mx500_hybAOWCDA_enkfgdas_marineanlecen_202103250000 ......   Passed   25.40 sec
 59/133 Test #1736: test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmat_202103250000 .............   Passed   37.25 sec
 65/133 Test #1734: test_gdasapp_C48mx500_hybAOWCDA_gdas_prepoceanobs_202103250000 ...........   Passed  168.87 sec
 70/133 Test #1737: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlinit_202103250000 ..........   Passed   26.75 sec
 71/133 Test #1726: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlvar_202103250000 .........   Passed  102.24 sec
 74/133 Test #1738: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlvar_202103250000 ...........   Passed   26.72 sec
 75/133 Test #1727: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000 .......   Passed   28.18 sec
 77/133 Test #1728: test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000 .......   Passed   40.46 sec
 78/133 Test #1740: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000 .........   Passed   50.68 sec
 79/133 Test #1741: test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000 .........   Passed   26.93 sec

The Passed results for GDASApp do not mean the entire suite of g-w CI jobs will be successful. Nonetheless, the above Passed results are encouraging.

@RussTreadon-NOAA RussTreadon-NOAA self-requested a review October 6, 2025 11:06
RussTreadon-NOAA
RussTreadon-NOAA approved these changes Oct 6, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@RussTreadon-NOAA RussTreadon-NOAA left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weekly update of GDASApp JEDI hashes run using AndrewEichmann-NOAA:bugfix/update-marine-da-output for g-w. GDASApp ctests, including select g-w CI, successfully ran to completion on Gaea C6, Hera, Hercules, Orion, Ursa, and WCOSS2 (Cactus). See GDASApp PR #1927 for details.

Note: GDASApp PR #1927 has been merged into GDASApp develop. Done at 811d86b

Approve.

@RussTreadon-NOAA RussTreadon-NOAA mentioned this pull request Oct 6, 2025
Merged
20 tasks
@DavidHuber-NOAA
Copy link
Copy Markdown
Contributor

DavidHuber-NOAA commented Oct 6, 2025

Launching CI on Hera.

@emcbot emcbot added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Oct 6, 2025
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Oct 6, 2025
@DavidHuber-NOAA
Copy link
Copy Markdown
Contributor

DavidHuber-NOAA commented Oct 7, 2025

All tests passed on Hera. Merging.

@DavidHuber-NOAA DavidHuber-NOAA merged commit bcd290b into NOAA-EMC:develop Oct 7, 2025
5 checks passed
weihuang-jedi added a commit to NOAA-EPIC/global-workflow-cloud that referenced this pull request Oct 8, 2025
@weihuang-jedi
Merge branch 'develop' of github.com:NOAA-EPIC/global-workflow-cloud …
2a1f9de 
…into feature/adjust_tasks_per_node_layout

* 'develop' of github.com:NOAA-EPIC/global-workflow-cloud:
  Correct parametric and ensemble background error statistics filenames in marine DA (NOAA-EMC#4120)
weihuang-jedi added a commit to NOAA-EPIC/global-workflow-cloud that referenced this pull request Nov 5, 2025
@weihuang-jedi
Merge branch 'feature/use_container_spack-stack-1.9.2' of github.com:…
b85bcb5 
…NOAA-EPIC/global-workflow-cloud into feature/use_container_spack-stack-1.9.2

* 'feature/use_container_spack-stack-1.9.2' of github.com:NOAA-EPIC/global-workflow-cloud:
  remove env/*.container
  testing on AWS
  no need to save to repo, as it is a link
  add PYCMD
  merge develop change in
  Consolidate JEDI-based atmospheric analysis task configuration YAMLs and create new Analysis class (NOAA-EMC#4080)
  Ctest case updates (NOAA-EMC#4118)
  using PYCMD
  fix archive script
  Consolidate load_*_modules scripts into a generic load_modules.sh script (NOAA-EMC#4126)
  Updates for Gaea C6 following OS upgrade (NOAA-EMC#4110)
  combine few scripts to decrease numbers
  reverse to GW repo code, and new way to handle jobs scripts
  Correct parametric and ensemble background error statistics filenames in marine DA (NOAA-EMC#4120)
DavidNew-NOAA pushed a commit to NOAA-EMC/GDASApp that referenced this pull request Jan 16, 2026
@AndrewEichmann-NOAA
break out ens and parametric bkgerr stats (#1922)
11af843 
# Description

Fixes scrambling of ens and parametric bkgerr stats

# Companion PRs

Requires NOAA-EMC/jcb-gdas#186 and pending g-w
PR (NOAA-EMC/global-workflow#4120)

<!-- Enter links to any companion PRs here. -->

# Issues

Resolves, with jcb-gdas and global-workflow PRs,
#1921

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [x] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
DavidNew-NOAA pushed a commit to NOAA-EMC/GDASApp that referenced this pull request Jan 16, 2026
@AndrewEichmann-NOAA
Add jinja yaml template for moving (renaming) bkgerr stat files (#1928)
6aa06c1 
# Description

Adds jinja yaml template for moving (renaming) bkgerr stat files in bmat
task

# Companion PRs

Will need NOAA-EMC/global-workflow#4120 to
actually work

# Issues

Helps fix #1921

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [x] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@guillaumevernieres guillaumevernieres guillaumevernieres approved these changes

@RussTreadon-NOAA RussTreadon-NOAA RussTreadon-NOAA approved these changes

@DavidHuber-NOAA DavidHuber-NOAA DavidHuber-NOAA approved these changes

@DavidNew-NOAA DavidNew-NOAA DavidNew-NOAA left review comments

@aerorahul aerorahul Awaiting requested review from aerorahul

Assignees

No one assigned

Labels

CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

In marine DA parametric background error statistics are saved as ensemble background error statistics

6 participants

@AndrewEichmann-NOAA @RussTreadon-NOAA @DavidHuber-NOAA @guillaumevernieres @DavidNew-NOAA @emcbot