Skip to content

Add a depth dependent e-fold scale#1494

Merged
guillaumevernieres merged 7 commits into
developfrom
feature/shallow
Feb 24, 2025
Merged

Add a depth dependent e-fold scale#1494
guillaumevernieres merged 7 commits into
developfrom
feature/shallow

Conversation

@guillaumevernieres
Copy link
Copy Markdown
Contributor

Description

Not making use of the ensemble spread requires adding a few bells and whistles to the parametric B. This PR allows better control of the variance partitioning in shallow places, scaling e-fold to the local depth.

Draft PR until NOAA-EMC/global-workflow#3238 is merged

Companion PRs

Issues

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmaerosnowDA
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@guillaumevernieres
Copy link
Copy Markdown
Contributor Author

I don't think this PR works without the changes in NOAA-EMC/global-workflow#3238 . Keeping as draft for now.

Copy link
Copy Markdown
Collaborator

@shlyaeva shlyaeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't test, but looks reasonable to me. A couple comments/questions below.
With this change the vertical e-folding scale stays 500 (or vertical e-folding scale) where depth>1500m (or vertical e-folding scale * min efold depth ratio), and switches to depth/3 (depth/min efold depth ratio) otherwise (perhaps except for very shallow water where depth is below min depth), did I read it correct?

Comment thread utils/soca/gdas_soca_diagb.h
Comment thread utils/soca/gdas_soca_diagb.h Outdated
shlyaeva
shlyaeva previously approved these changes Feb 14, 2025
shlyaeva
shlyaeva previously approved these changes Feb 14, 2025
Copy link
Copy Markdown
Collaborator

@shlyaeva shlyaeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing my comments, and I'm impressed!

Comment thread utils/soca/gdas_soca_diagb.h
@guillaumevernieres guillaumevernieres added the DO NOT MERGE PR is not ready to be merged yet label Feb 14, 2025
Comment thread utils/soca/gdas_soca_diagb.h
@guillaumevernieres guillaumevernieres marked this pull request as ready for review February 20, 2025 16:21
@guillaumevernieres guillaumevernieres added the hera-GW-RT Queue for automated testing with global-workflow on Hera label Feb 20, 2025
@guillaumevernieres guillaumevernieres self-assigned this Feb 21, 2025
@guillaumevernieres guillaumevernieres added the hercules-GW-RT Queue for automated testing with global-workflow on Hercules label Feb 21, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Feb 21, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 21, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Fri Feb 21 13:26:22 UTC 2025 on hfe03
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Feb 21 14:10:19 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C48mx500_3DVarAOWCDA|C48mx500_hybAOWCDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Fri Feb 21 14:11:17 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 33

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera hercules-GW-RT-Running Automated testing with global-workflow running on Hercules and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera hercules-GW-RT Queue for automated testing with global-workflow on Hercules labels Feb 21, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 21, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Fri Feb 21 09:10:27 CST 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Feb 21 10:02:23 CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C48mx500_3DVarAOWCDA|C48mx500_hybAOWCDA|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Fri Feb 21 10:37:43 CST 2025
Tests: 97% tests passed, 1 tests failed out of 33
	1967 - test_gdasapp_snow_letkfoi_snowda (Failed)
Tests: see output at /work2/noaa/da/role-da/CI/hercules/GDASApp/workflow/PR/1494/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Feb 21, 2025
@RussTreadon-NOAA
Copy link
Copy Markdown
Contributor

test_gdasapp_snow_letkfoi_snowda failed on Hercules due to the job taking too long to complete.

33/33 Test #1967: test_gdasapp_snow_letkfoi_snowda .................***Failed  2109.77 sec
+ YY=2021
+ MM=03

...

whalo =    3, ehalo =    3, shalo =    3, nhalo =    3
  X-AXIS =   48
  Y-AXIS =   48
slurmstepd: error: *** STEP 4602822.0 ON hercules-03-29 CANCELLED AT 2025-02-21T10:37:31 DUE TO TIME LIMIT ***
srun: Job step aborted: Waiting up to 32 seconds for job step to finish.
srun: error: hercules-03-29: tasks 1-5: Terminated
srun: Terminating StepId=4602822.0
srun: error: hercules-03-29: task 0: Terminated

MSU Orion and Hercules are experiencing sluggish disk performance. Tickets with the Orion and Hercules help desks were opened earlier this morning (2/21/2025).

@guillaumevernieres
Copy link
Copy Markdown
Contributor Author

@RussTreadon-NOAA , it seems that hercules has been having issues for the past few days. I still need to merge things in jcb-gdas and update the # here before I can merge this. I'm still hoping to get to this before cob ...

@guillaumevernieres guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Passed Automated testing with global-workflow successful on Hera hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Feb 21, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Feb 21, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Feb 21, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Fri Feb 21 19:24:19 UTC 2025 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Feb 21 20:15:57 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C48mx500_3DVarAOWCDA|C48mx500_hybAOWCDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Fri Feb 21 20:16:49 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 33

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Feb 21, 2025
Copy link
Copy Markdown
Collaborator

@AndrewEichmann-NOAA AndrewEichmann-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok

@guillaumevernieres guillaumevernieres removed the DO NOT MERGE PR is not ready to be merged yet label Feb 24, 2025
@guillaumevernieres guillaumevernieres merged commit 6c31bb3 into develop Feb 24, 2025
@guillaumevernieres guillaumevernieres deleted the feature/shallow branch February 24, 2025 16:25
jspsl pushed a commit that referenced this pull request Mar 26, 2025
# Description
Not making use of the ensemble spread requires adding a few bells and
whistles to the parametric **B**. This PR allows better control of the
variance partitioning in shallow places, scaling e-fold to the local
depth.

_Draft PR until NOAA-EMC/global-workflow#3238 is
merged_

# Companion PRs
- NOAA-EMC/jcb-gdas#73

# Issues
- fixes #1491

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmaerosnowDA  <!-- JEDI aero/snow cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [ ] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->
danholdaway added a commit that referenced this pull request Mar 26, 2025
* develop:
  added original date (#1570)
  Update OCEAN_BASIN_FILE_ORION path for marine ctest (#1569)
  Fixing window_begin and end in yaml for redating ADT (#1565)
  Save LETKF mean analysis and mean increment for verification (#1566)
  Update JEDI hashes (20250324) (#1567)
  Removes obsolete json files from marine bufr2ioda (#1564)
  Add hybrid 4denvar as a phase3 validation configuration (#1552)
  Bugfix/ocn ecen (#1550)
  Bugfix SSS obs for LETKF (#1560)
  Bugfix ocean ensemble recentering (#1558)
  apply ci path patch only when running g-w ctests (#1554)
  Update JEDI hashes (20250317)  (#1531)
  add option to create_experiment.py in create_exp.sh  (#1546)
  Update jcb hashes and support initial IODA stats capability (#1547)
  Fix GW-CI CTest dependencies (#1534)
  Converts prepoceanobs to use JCB-style obs list and config files (#1527)
  JEDI-based ensemble recentering and analysis calculation (#1488)
  tools for phase3 validation (#1526)
  Update JEDI hashes (20250306) (#1520)
  New jcb-gdas # and insitu obs prep bug fix  (#1523)
  Marine in situ obs update (SSS, moorings, drifters) (#1511)
  Save marine background and analysis spread (#1517)
  Minor build and install updates and ctest cleanup (#1512)
  correct logic for upper loop limit in chem_diagb (#1516)
  Incorporate multi-cycle concatination and separation of variables in post-BUFR observations (#1504)
  Update JEDI hashes (20250225) (#1475)
  Add a depth dependent e-fold scale (#1494)
  Update jcb and da-utils hashes to support land DA. (#1509)
  Add options to inflate the ensemble while recentering (#1501)
  Add components to use jcb yamls for marine letkf job (#1486)
  Add GNSSRO bufr2ioda and json files (#1448)
  Removal of the marine verification tools (#1499)
  update logic to identify MSU machine (#1498)
  IODA Converter with In-Situ Observations Concatenation and ObsError Inflation (#1472)
  Add sea ice recentering (#1478)
  marine vrfy bugfix (#1483)
  add LD_LIBRARY_PATH patch for wcoss2 build (#1484)
  Bugfix for ensemble recenter + add a test (#1477)
  Add code changes for the implementation of the  low-resolution B (#1441)
  update spack-stack path in Hera modulefiles (#1471)
  fixed glider output filename (#1469)
  add gdas_waveinit to C48mx500_3DVarAOWCDA ctest suite (#1470)
  update jcb-gdas hash 20250128 (#1466)
  Time series of omb stats and html doc (#1463)
  Update marine vrfy HTML codes (#1464)
  Automate the verify tools to submit from start pdy to end pdy (#1458)
  Update an offline marine vrfy tool (#1447)
  Update JEDI hashes (20250127) (#1460)
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
# Description
Not making use of the ensemble spread requires adding a few bells and
whistles to the parametric **B**. This PR allows better control of the
variance partitioning in shallow places, scaling e-fold to the local
depth.

_Draft PR until NOAA-EMC/global-workflow#3238 is
merged_

# Companion PRs
- NOAA-EMC/jcb-gdas#73

# Issues
- fixes #1491

# Automated CI tests to run in Global Workflow
<!-- Which Global Workflow CI tests are required to adequately test this
PR? -->
- [ ] atm_jjob <!-- JEDI atm single cycle DA !-->
- [ ] C96C48_ufs_hybatmDA <!-- JEDI atm cycled DA !-->
- [ ] C96C48_hybatmaerosnowDA  <!-- JEDI aero/snow cycled DA !-->
- [ ] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [ ] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
- [ ] C96C48_hybatmDA <!-- GSI atm cycled DA !-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hera-GW-RT-Passed Automated testing with global-workflow successful on Hera

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the ensemble B from the marine hybrid B

6 participants