Skip to content

break out ens and parametric bkgerr stats#1922

Merged
guillaumevernieres merged 3 commits into
developfrom
bugfix/update-marine-da-output
Oct 2, 2025
Merged

break out ens and parametric bkgerr stats#1922
guillaumevernieres merged 3 commits into
developfrom
bugfix/update-marine-da-output

Conversation

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator

@AndrewEichmann-NOAA AndrewEichmann-NOAA commented Oct 1, 2025

Description

Fixes scrambling of ens and parametric bkgerr stats

Companion PRs

Requires NOAA-EMC/jcb-gdas#186 and pending g-w PR (NOAA-EMC/global-workflow#4120)

Issues

Resolves, with jcb-gdas and global-workflow PRs, #1921

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmsnowDA
  • C96_gcafs_cycled
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@AndrewEichmann-NOAA AndrewEichmann-NOAA requested review from DavidNew-NOAA, Copilot and guillaumevernieres and removed request for Copilot October 1, 2025 15:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Separates ensemble and parametric background error statistics handling to prevent scrambling of these different error types in the marine background matrix save configuration.

  • Moves parametric background error files outside the conditional block
  • Keeps ensemble background error files within the hybrid/ensemble conditional block
  • Ensures proper file naming and organization for different error statistics types

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread parm/marine/marine_bmat_save.yaml.j2 Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
DavidNew-NOAA
DavidNew-NOAA previously approved these changes Oct 1, 2025
Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AndrewEichmann-NOAA , can you check if something needs to be done on the archiving side.

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

@guillaumevernieres It does, testing that now

@AndrewEichmann-NOAA AndrewEichmann-NOAA self-assigned this Oct 2, 2025
@AndrewEichmann-NOAA AndrewEichmann-NOAA added hercules-GW-RT Queue for automated testing with global-workflow on Hercules ursa-GW-RT labels Oct 2, 2025
@emcbot emcbot added ursa-GW-RT-Running hercules-GW-RT-Running Automated testing with global-workflow running on Hercules and removed ursa-GW-RT hercules-GW-RT Queue for automated testing with global-workflow on Hercules labels Oct 2, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Oct 2, 2025

Automated GW-GDASApp Testing Results:
Machine: ursa

Start: Thu Oct  2 16:20:13 UTC 2025 on ufe04
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Thu Oct  2 16:42:50 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmsnowDA|C96_gcafs_cycled|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Thu Oct  2 16:51:18 UTC 2025
Tests: 76% tests passed, 10 tests failed out of 42
	1722 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmat_202103250000 (Failed)
	1723 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlinit_202103250000 (Failed)
	1724 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlvar_202103250000 (Failed)
	1725 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	1726 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
	1734 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmat_202103250000 (Failed)
	1735 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlinit_202103250000 (Failed)
	1736 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlvar_202103250000 (Failed)
	1738 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	1739 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
Tests: see output at /scratch4/NCEPDEV/stmp/role.jedipara/CI/GDASApp/workflow/PR/1922/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 2, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Thu Oct  2 11:17:38 CDT 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Thu Oct  2 11:53:48 CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled|atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmsnowDA|C96_gcafs_cycled|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Thu Oct  2 12:06:49 CDT 2025
Tests: 70% tests passed, 10 tests failed out of 33
	1713 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmat_202103250000 (Failed)
	1714 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlinit_202103250000 (Failed)
	1715 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlvar_202103250000 (Failed)
	1716 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	1717 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
	1725 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmat_202103250000 (Failed)
	1726 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlinit_202103250000 (Failed)
	1727 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlvar_202103250000 (Failed)
	1729 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	1730 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
Tests: see output at /work2/noaa/da/role-da/CI/hercules/GDASApp/workflow/PR/1922/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Oct 2, 2025
@AndrewEichmann-NOAA AndrewEichmann-NOAA added hercules-GW-RT Queue for automated testing with global-workflow on Hercules and removed hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Oct 2, 2025
@emcbot emcbot added hercules-GW-RT-Running Automated testing with global-workflow running on Hercules and removed hercules-GW-RT Queue for automated testing with global-workflow on Hercules labels Oct 2, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Oct 2, 2025

Automated GW-GDASApp Testing Results:
Machine: ursa

Start: Thu Oct  2 17:08:37 UTC 2025 on ufe03
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Thu Oct  2 17:30:54 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmsnowDA|C96_gcafs_cycled|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Thu Oct  2 17:38:38 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 42

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 2, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Thu Oct  2 12:18:13 CDT 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Thu Oct  2 12:50:58 CDT 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E C96_gcafs_cycled|atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmsnowDA|C96_gcafs_cycled|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Thu Oct  2 12:58:30 CDT 2025
Tests: 100% tests passed, 0 tests failed out of 33

@emcbot emcbot added hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Oct 2, 2025
@AndrewEichmann-NOAA AndrewEichmann-NOAA marked this pull request as ready for review October 2, 2025 17:59
Copy link
Copy Markdown
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@guillaumevernieres guillaumevernieres merged commit 2953f31 into develop Oct 2, 2025
21 checks passed
@guillaumevernieres guillaumevernieres deleted the bugfix/update-marine-da-output branch October 2, 2025 18:28
@AndrewEichmann-NOAA AndrewEichmann-NOAA mentioned this pull request Oct 2, 2025
7 tasks
guillaumevernieres pushed a commit that referenced this pull request Oct 2, 2025
# Description

Update jcb-gdas hash

# Companion PRs

NA

# Issues

Cleanup of merge of #1922

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [ ] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

@AndrewEichmann-NOAA and @guillaumevernieres : After merging this PR into feature/stable-nightly, the following g-w CI jobs fail

        1724 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmat_202103250000 (Failed)
        1725 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlinit_202103250000 (Failed)
        1726 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlvar_202103250000 (Failed)
        1727 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
        1728 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
        1736 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmat_202103250000 (Failed)
        1737 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlinit_202103250000 (Failed)
        1738 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlvar_202103250000 (Failed)
        1740 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
        1741 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000 (Failed)

Do we need to run ctests using g-w branch AndrewEichmann-NOAA:bugfix/update-marine-da-output from g-w PR #4120?

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

Do we need to run ctests using g-w branch AndrewEichmann-NOAA:bugfix/update-marine-da-output from g-w PR #4120?

@RussTreadon-NOAA Yes. Is there a way I should indicate that?

@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

@AndrewEichmann-NOAA . You note this requirement above.

Companion PRs
Requires NOAA-EMC/jcb-gdas#186 and pending g-w PR (NOAA-EMC/global-workflow#4120)

The problem is that automated feature/stable-nightly doesn't read PRs.

DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
# Description

Fixes scrambling of ens and parametric bkgerr stats

# Companion PRs

Requires NOAA-EMC/jcb-gdas#186 and pending g-w
PR (NOAA-EMC/global-workflow#4120)

<!-- Enter links to any companion PRs here. -->

# Issues

Resolves, with jcb-gdas and global-workflow PRs,
#1921

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [x] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
# Description

Update jcb-gdas hash

# Companion PRs

NA

# Issues

Cleanup of merge of #1922

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmsnowDA <!-- JEDI snow cycled DA !-->
- [ ] C96_gcafs_cycled <!-- JEDI aerosol cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [ ] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules ursa-GW-RT-Passed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants