Refactor gridded wave post by AntonMFernando-NOAA · Pull Request #3014 · NOAA-EMC/global-workflow

AntonMFernando-NOAA · 2024-10-18T01:19:15Z

Description

The gridded wave post (wavepostsbs) script currently loops over all forecast hours and acts as its own post manager. Other component post and product scripts operate on one forecast time. This PR will update the wave post to match other components, that creates a bunch of short jobs instead of one long job (per member).
Resolves #2290

Type of change

Bug fix (fixes something broken)
New feature (adds functionality)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? NO

How has this been tested?

Run CI tests in HERA, ORION, and HERCULES

Checklist

Gridded wave post scripts that process only one forecast hour specified by the workflow manager (rocoto/ecflow)
Rocoto workflow updated to create the corresponding tasks for the new paradigm for both GFS and GEFS
Output identical to current system for both GFS and GEFS

emcbot · 2024-11-23T17:49:39Z

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hercules in Build# 2 in
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/EXPDIR/C96_S2SWA_gefs_replay_ics_39da04da

emcbot · 2024-11-23T18:01:23Z

Experiment C48_S2SWA_gefs FAILED on Hercules in Build# 2 with error logs:

/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f052.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f053.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f054.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f056.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f057.log

Follow link here to view the contents of the above file(s): (link)

emcbot · 2024-11-23T18:01:35Z

Experiment C48_S2SWA_gefs FAILED on Hercules in Build# 2 in
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/EXPDIR/C48_S2SWA_gefs_39da04da

emcbot · 2024-11-23T18:01:43Z

CI Failed on Hercules in Build# 2
Built and ran in directory /work2/noaa/global/CI/HERCULES/3014


Experiment C48_S2SW_39da04da Terminated with 0
FAIL
FAIL tasks failed and 5 dead at Sat Nov 23 09:41:48 CST 2024
Experiment C48_S2SW_39da04da Terminated: *FAIL*
Error logs:
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SW_39da04da/logs/2021032312/gfs_wavepostsbs_f012.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SW_39da04da/logs/2021032312/gfs_wavepostsbs_f015.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SW_39da04da/logs/2021032312/gfs_wavepostsbs_f018.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SW_39da04da/logs/2021032312/gfs_wavepostsbs_f021.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SW_39da04da/logs/2021032312/gfs_wavepostsbs_f024.log
Experiment C48_ATM_39da04da Completed 2 Cycles: *SUCCESS* at Sat Nov 23 09:47:42 CST 2024
Experiment C96_atm3DVar_39da04da Completed 3 Cycles: *SUCCESS* at Sat Nov 23 11:06:32 CST 2024
Experiment C96C48_hybatmDA_39da04da Completed 3 Cycles: *SUCCESS* at Sat Nov 23 11:42:52 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_39da04da Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Sat Nov 23 11:49:23 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_39da04da Terminated: *FAIL*
Error logs:
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_39da04da/logs/2020110100/gefs_wave_post_pnt_mem001.log
Experiment C48_S2SWA_gefs_39da04da Terminated with 0
FAIL
FAIL tasks failed and 5 dead at Sat Nov 23 12:01:04 CST 2024
Experiment C48_S2SWA_gefs_39da04da Terminated: *FAIL*
Error logs:
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f052.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f053.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f054.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f056.log
/work2/noaa/global/CI/HERCULES/3014/RUNTESTS/COMROOT/C48_S2SWA_gefs_39da04da/logs/2021032312/gefs_wave_post_grid_mem001_f057.log

…nando-NOAA/global-workflow into feature/wave_post_grid

emcbot · 2024-11-26T08:52:42Z

CI Passed on Hercules in Build# 3
Built and ran in directory /work2/noaa/global/CI/HERCULES/3014


Experiment C48_ATM_4b0523b0 Completed 2 Cycles: *SUCCESS* at Mon Nov 25 22:54:44 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_4b0523b0 Completed 1 Cycles: *SUCCESS* at Mon Nov 25 23:55:50 CST 2024
Experiment C96_atm3DVar_4b0523b0 Completed 3 Cycles: *SUCCESS* at Tue Nov 26 00:01:20 CST 2024
Experiment C96C48_hybatmDA_4b0523b0 Completed 3 Cycles: *SUCCESS* at Tue Nov 26 00:07:32 CST 2024
Experiment C48_S2SW_4b0523b0 Completed 2 Cycles: *SUCCESS* at Tue Nov 26 00:32:03 CST 2024
Experiment C48_S2SWA_gefs_4b0523b0 Completed 1 Cycles: *SUCCESS* at Tue Nov 26 02:52:29 CST 2024

AntonMFernando-NOAA · 2024-11-26T15:35:19Z

@WalterKolczynski-NOAA @DavidHuber-NOAA I think all good now.

WalterKolczynski-NOAA · 2024-11-26T16:32:04Z

CI Tests set up to run in /lfs/h2/emc/ptmp/walter.kolczynski/PR/PR_3014/RUNTESTS on WCOSS

emcbot · 2024-11-26T23:06:57Z

CI Passed on Hera in Build# 4
Built and ran in directory /scratch1/NCEPDEV/global/CI_dh/3014


Experiment C48_ATM_232027f8 Completed 2 Cycles: *SUCCESS* at Tue Nov 26 19:52:38 UTC 2024
Experiment C48mx500_3DVarAOWCDA_232027f8 Completed 2 Cycles: *SUCCESS* at Tue Nov 26 20:16:59 UTC 2024
Experiment C96_S2SWA_gefs_replay_ics_232027f8 Completed 1 Cycles: *SUCCESS* at Tue Nov 26 20:48:26 UTC 2024
Experiment C96C48_hybatmaerosnowDA_232027f8 Completed 3 Cycles: *SUCCESS* at Tue Nov 26 21:48:52 UTC 2024
Experiment C96_atm3DVar_232027f8 Completed 3 Cycles: *SUCCESS* at Tue Nov 26 21:54:57 UTC 2024
Experiment C96C48_hybatmDA_232027f8 Completed 3 Cycles: *SUCCESS* at Tue Nov 26 21:54:58 UTC 2024
Experiment C48_S2SW_232027f8 Completed 2 Cycles: *SUCCESS* at Tue Nov 26 22:20:26 UTC 2024
Experiment C96C48_ufs_hybatmDA_232027f8 Completed 3 Cycles: *SUCCESS* at Tue Nov 26 22:37:31 UTC 2024
Experiment C48_S2SWA_gefs_232027f8 Completed 1 Cycles: *SUCCESS* at Tue Nov 26 23:06:46 UTC 2024

AntonMFernando-NOAA and others added 26 commits October 17, 2024 16:56

modified gefs_tasks.py

26e06a3

Merge branch 'NOAA-EMC:develop' into feature/wave_post_grid

99dea4a

modified gefs_tasks.py

97b5932

modifed gefs_tasks.py

64552f8

modifed gefs_tasks.py

ed3768c

modified gefs_tasks.py

9373b88

modified gefs_tasks.py

916c1fa

modified gefs_tasks.py

f018d2e

modify gefs_tasks.py

aede2e9

modified exgfs_wave_post_gridded_sbs.sh

f59a6c9

modified gefs_tasks.py

f07b0ec

modified gefs.py

6261b4f

modified gefs_tasks.py

b2b3078

modified gefs_tasks.py

ea023be

modified gefs_tasks.py

3c69e48

modified gefs_task.py

16205f1

modified global-workflow\scripts\exgfs_wave_post_gridded_sbs.sh

0c7cb50

modified global-workflow\scripts\exgfs_wave_post_gridded_sbs.sh

458b130

modifed gefs_tasks.py

027e79f

modified global-workflow\scripts\exgfs_wave_post_gridded_sbs.sh

f75bbf3

corrected a typo

b408e75

mdified global-workflow\scripts\exgfs_wave_post_gridded_sbs.sh

d96631e

modified gefs_tasks.py

08c4f12

modified gefs_tasks.py

bc74e49

updated wavepostsbs.sh

99869ed

Merge branch 'develop' into feature/wave_post_grid

8b9d5d1

AntonMFernando-NOAA marked this pull request as ready for review October 21, 2024 19:54

AntonMFernando-NOAA requested review from JessicaMeixner-NOAA, WalterKolczynski-NOAA and sbanihash as code owners October 21, 2024 19:54

emcbot added CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed and removed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Nov 23, 2024

AntonMFernando-NOAA and others added 7 commits November 23, 2024 15:13

modified parm\config\gfs\config.base

7bf1a83

Merge branch 'feature/wave_post_grid' of https://github.com/AntonMFer…

c8cdde1

…nando-NOAA/global-workflow into feature/wave_post_grid

modified config.base for gfs

bfbc940

modified config.base for gfs

5a899c8

Merge branch 'develop' into feature/wave_post_grid

d774de8

modified config.base for gefs and gfs

f1a7995

modified parm\config\gfs\config.base

4b0523b

WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Nov 26, 2024

DavidHuber-NOAA reviewed Nov 26, 2024

View reviewed changes

Comment thread workflow/rocoto/gfs_tasks.py Outdated

DavidHuber-NOAA reviewed Nov 26, 2024

View reviewed changes

Comment thread workflow/rocoto/gefs_tasks.py Outdated

AntonMFernando-NOAA and others added 2 commits November 26, 2024 10:15

modified gefs_tasks.py and gfs_tasks.py

f4d8b23

Merge branch 'develop' into feature/wave_post_grid

232027f

WalterKolczynski-NOAA approved these changes Nov 26, 2024

View reviewed changes

Merge branch 'develop' into feature/wave_post_grid

4c05ca7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor gridded wave post#3014

Refactor gridded wave post#3014
WalterKolczynski-NOAA merged 140 commits into
NOAA-EMC:developfrom
AntonMFernando-NOAA:feature/wave_post_grid

AntonMFernando-NOAA commented Oct 18, 2024

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 26, 2024

Uh oh!

Uh oh!

Uh oh!

AntonMFernando-NOAA commented Nov 26, 2024

Uh oh!

WalterKolczynski-NOAA commented Nov 26, 2024

Uh oh!

emcbot commented Nov 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

Conversation

AntonMFernando-NOAA commented Oct 18, 2024

Description

Type of change

Change characteristics

How has this been tested?

Checklist

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 23, 2024

Uh oh!

emcbot commented Nov 26, 2024

Uh oh!

Uh oh!

Uh oh!

AntonMFernando-NOAA commented Nov 26, 2024

Uh oh!

WalterKolczynski-NOAA commented Nov 26, 2024

Uh oh!

emcbot commented Nov 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants