Skip to content

Fix wave restarts and GEFS FHOUT/FHMAX#3009

Merged
DavidHuber-NOAA merged 15 commits into
NOAA-EMC:developfrom
WalterKolczynski-NOAA:hotfix/segs
Oct 30, 2024
Merged

Fix wave restarts and GEFS FHOUT/FHMAX#3009
DavidHuber-NOAA merged 15 commits into
NOAA-EMC:developfrom
WalterKolczynski-NOAA:hotfix/segs

Conversation

@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA commented Oct 16, 2024

Description

Fixes some issues that were preventing wave restarts from operating correctly.

First, the wave restart files were not being correctly linked from $DATA to $DATArestart. The files are placed in the root of $DATA instead of in ${DATA}/WAVE_RESTART, so now links for the individual files are created.

Second, the incorrect filenames were being searches for and copied as part of a rerun. Filenames were geared towards multigrid waves, which use the grid names, but single grid just uses a ww3. Since multigrid waves are deprecated in workflow and will soon be removed (#2637), these were updated only supporting the single-grid option.

These fixes allow forecast segments (and emergency restarts) to work correctly when waves are on.

Additionally, the FHMAX (and FHOUT) for perturbed GEFS members was being overwritten by config.efcs due to remnant code from when it was copied from the EnKF version. This interfered with the segment setting for those GEFS members.

Resolves #3001

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

  • S2SW forecast-only test with segments on Hercules

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • I have made corresponding changes to the system documentation if necessary

@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor Author

ATTN: @weihuang-jedi

Comment thread ush/forecast_postdet.sh Outdated
Comment thread ush/forecast_postdet.sh Outdated
Copy link
Copy Markdown
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is an issue in the forecast_postdet.sh part. Also a suggestion to break that into 2 parts.

Fixes some issues that were preventing wave restarts from operating
correctly.

First, the wave restart files were not being correctly linked from
`$DATA` to `$DATArestart`. The files are placed in the root of
`$DATA` instead of in `${DATA}/WAVE_RESTART`, so now links for the
individual files are created.

Second, the incorrect filenames were being searches for and copied
as part of a rerun. Filenames were geared towards multigrid waves,
which use the grid names, but single grid just uses a `ww3`. Since
multigrid waves are deprecated in workflow and will soon be removed
(NOAA-EMC#2637), these were updated only supporting the single-grid option.

These fixes allow forecast segments (and emergency restarts) to
work correctly when waves are on.

Resolves NOAA-EMC#3001
Removes an unnecessary loop when staging wave restarts and stops
creating output links for restart files that were already copied
in as ICs.

Also updates FHMIN_WAV when running with segments to stop creating
unneeded links from earlier in the forecast period.
@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor Author

Will need to add in additional fixes for GEFS perturbed members, as they are ignoring the segments.

The gefs `config.efcs` still contained code from the enkf version
of the file that was overwriting the FHMAX set for segments.
@WalterKolczynski-NOAA WalterKolczynski-NOAA changed the title Fix wave restarts Fix wave restarts and GEFS FHOUT/FHMAX Oct 17, 2024
@weihuang-jedi
Copy link
Copy Markdown
Contributor

I have test the PR code on Hercules. I worked as expected. Fixed the segments issue.
Thanks @WalterKolczynski-NOAA !

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Oct 22, 2024
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules labels Oct 22, 2024
@TerrenceMcGuinness-NOAA
Copy link
Copy Markdown
Collaborator

Jenkins controller had technical difficulties and was off line. Clearing Hercules is under currently under maintenance.

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA removed the CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules label Oct 23, 2024
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Oct 23, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Oct 23, 2024
@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Oct 25, 2024
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Oct 25, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Oct 25, 2024

Experiment C96_atm3DVar FAILED on Hercules in Build# 7 in
/work2/noaa/stmp/CI/HERCULES/3009/RUNTESTS/EXPDIR/

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 25, 2024

Experiment C48_S2SW FAILED on Hercules in Build# 7 in
/work2/noaa/stmp/CI/HERCULES/3009/RUNTESTS/EXPDIR/

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 25, 2024

Experiment C96C48_hybatmDA FAILED on Hercules in Build# 7 in
/work2/noaa/stmp/CI/HERCULES/3009/RUNTESTS/EXPDIR/

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 25, 2024

Experiment C48_ATM FAILED on Hercules in Build# 7 in
/work2/noaa/stmp/CI/HERCULES/3009/RUNTESTS/EXPDIR/

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 25, 2024

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hercules in Build# 7 in
/work2/noaa/stmp/CI/HERCULES/3009/RUNTESTS/EXPDIR/

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 25, 2024

Experiment C48_S2SWA_gefs FAILED on Hercules in Build# 7 in
/work2/noaa/stmp/CI/HERCULES/3009/RUNTESTS/EXPDIR/

@emcbot emcbot removed the CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed label Oct 25, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Oct 28, 2024

CI Passed on Hercules in Build# 8
Built and ran in directory /work2/noaa/global/CI/HERCULES/3009


Experiment C48_ATM_17ee741f Completed 2 Cycles: *SUCCESS* at Mon Oct 28 03:11:28 CDT 2024
Experiment C96_S2SWA_gefs_replay_ics_17ee741f Completed 1 Cycles: *SUCCESS* at Mon Oct 28 03:17:33 CDT 2024
Experiment C96_atm3DVar_17ee741f Completed 3 Cycles: *SUCCESS* at Mon Oct 28 04:18:05 CDT 2024
Experiment C96C48_hybatmDA_17ee741f Completed 3 Cycles: *SUCCESS* at Mon Oct 28 04:18:15 CDT 2024
Experiment C48_S2SW_17ee741f Completed 2 Cycles: *SUCCESS* at Mon Oct 28 05:12:31 CDT 2024
Experiment C48_S2SWA_gefs_17ee741f Completed 1 Cycles: *SUCCESS* at Mon Oct 28 05:25:39 CDT 2024

@DavidHuber-NOAA
Copy link
Copy Markdown
Contributor

I'm going to manually run just the GEFS tests on Hera.

@DavidHuber-NOAA
Copy link
Copy Markdown
Contributor

All tests passed on Hera, but after a conversation with Walter and Kate, we will wait to merge this until an additional fix for the stage_ic job goes in.

@emcbot
Copy link
Copy Markdown

emcbot commented Oct 30, 2024

CI Passed on Hercules in Build# 1
Built and ran in directory /work2/noaa/global/CI/HERCULES/3009


Experiment C96_S2SWA_gefs_replay_ics_01b0c2f9 Completed 1 Cycles: *SUCCESS* at Wed Oct 30 07:06:20 CDT 2024
Experiment C48_ATM_01b0c2f9 Completed 2 Cycles: *SUCCESS* at Wed Oct 30 07:24:33 CDT 2024
Experiment C48_S2SW_01b0c2f9 Completed 2 Cycles: *SUCCESS* at Wed Oct 30 08:18:52 CDT 2024
Experiment C96_atm3DVar_01b0c2f9 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 08:31:05 CDT 2024
Experiment C96C48_hybatmDA_01b0c2f9 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 08:31:12 CDT 2024
Experiment C48_S2SWA_gefs_01b0c2f9 Completed 1 Cycles: *SUCCESS* at Wed Oct 30 09:26:17 CDT 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Possible bug on GEFS fcst segment

7 participants