Rename wave output and refactor some wave scripts#3406
Closed
WalterKolczynski-NOAA wants to merge 4 commits into
Closed
Rename wave output and refactor some wave scripts#3406WalterKolczynski-NOAA wants to merge 4 commits into
WalterKolczynski-NOAA wants to merge 4 commits into
Conversation
Wave products are renamed to be uniform with other components and
follow NCO implementation standards. This means all output is in
the format `${RUN}.wave.tCCz.product[.fHHH][.domain].suffix`.
Some wave scripts (init and gridded post) are refactored to make
them consistent with other parts of the workflow. All scripts for
both jobs now pass `shellcheck` without warnings.
The biggest changes in the refactor include beyond those necessary
for the file rename include:
- Using the `run_mpmd.sh` script
- Updated templates to use atparse instead of `sed`
- Removed redirection of output to logs that get deleted (NOAA-EMC#296)
One functional change is raw interpolated grids are no longer
copied to COM, only the resulting grib files. Need to confirm with
Jessica this is okay.
Refs NOAA-EMC#296
Refs NOAA-EMC#3270
The wave boundary point bulletin job (wavepostbndpntbull) was trying to load a value from the 'wave' dict instead of 'wavepostbndpntbull', causing a failure during XML generation.
Updates the boundary point j-jobs and the gempak scripts for the wave file rename. The gempak scripts are also refactored to meet style guidelines. As part of this, the job is changed to run one forecast hour at a time, with a rocoto metatask for development. Grouping is done for this metatask as with other product metatasks to reduce the job count. For dependency reasons, these settings must be the same as the wave gridded post, so the max tasks and fhrs are read from that config set in rocoto. Like the regular point job, the boundary point j-jobs are not refactored as the point job will be redone soon anyway. Only changes necessary for the new filenames are made.
Contributor
Author
|
Still a draft because:
|
Contributor
|
Too many conflicts here. Will close this PR and open another |
22 tasks
Contributor
|
OBE by #3517 |
DavidHuber-NOAA
pushed a commit
that referenced
this pull request
Apr 3, 2025
…x some bugzillas along the way (#3517) # Description [Copied and edited from #3406] Wave products are renamed to be uniform with other components and follow NCO implementation standards. This means all output is in the format ${RUN}.wave.tCCz.product[.fHHH][.domain].suffix. Some wave scripts are refactored to make them consistent with other parts of the workflow. All scripts for those jobs now pass shellcheck without warnings. Point job scripts were only updated for the file rename, as those scripts will be redone in the near future anyway. The biggest changes in the refactor include beyond those necessary for the file rename include: Using the run_mpmd.sh script Removed redirection of output to logs that get deleted (#296) The wave gempak job now only operates on one forecast hour at a time. The corresponding rocoto task has been made into a metatask, using the same grouping as wave gridded post. The gempak job must use the same fhr and max tasks settings to ensure the dependencies line up. One functional change is raw interpolated grids are no longer copied to COM, only the resulting grib files. [Need to confirm with Jessica this is okay] Resolves #296 Resolves #3270 This was work was largely done by @WalterKolczynski-NOAA # Type of change - [X] Bug fix (fixes something broken) - [X] New feature (adds functionality) - [X] Maintenance (code refactor, clean-up, new CI test, etc.) # Change characteristics <!-- Choose YES or NO from each of the following and delete the other --> - Is this a breaking change (a change in existing functionality)? YES/NO - Does this change require a documentation update? YES/NO - Does this change require an update to any of the following submodules? YES/NO (If YES, please add a link to any PRs that are pending.) - [ ] EMC verif-global <!-- NOAA-EMC/EMC_verif-global#1234 --> - [ ] GDAS <!-- NOAA-EMC/GDASApp#1234 --> - [ ] GFS-utils <!-- NOAA-EMC/gfs-utils#1234 --> - [ ] GSI <!-- NOAA-EMC/GSI#1234 --> - [ ] GSI-monitor <!-- NOAA-EMC/GSI-Monitor#1234 --> - [ ] GSI-utils <!-- NOAA-EMC/GSI-Utils#1234 --> - [ ] UFS-utils <!-- ufs-community/UFS_UTILS#1234 --> - [ ] UFS-weather-model <!-- ufs-community/ufs-weather-model#1234 --> - [ ] wxflow <!-- NOAA-EMC/wxflow#1234 --> # How has this been tested? - C48mx500 forecast-only on WCOS2. A experiment with `develop` at 563567d `dev1` at: `/lfs/h2/emc/ptmp/rahul.mahajan/RUNTESTS/EXPDIR/dev1` A experiment with this branch `wk1` at: `/lfs/h2/emc/ptmp/rahul.mahajan/RUNTESTS/EXPDIR/wk1` Links to respective `COM` can be found in the experiment directories. Below is a list of wave products from `wk1` ```bash ❯❯❯ tree COM/gfs.20210323/12/products/wave/ COM/gfs.20210323/12/products/wave/ ├── gempak │ ├── gfswaves200k_2021032312f000 │ ├── gfswaves200k_2021032312f001 │ ├── ... │ ├── ... │ └── gfswaves200k_2021032312f120 ├── gridded │ └── global.2p00 │ ├── gfs.wave.t12z.global.2p00.f000.grib2 │ ├── gfs.wave.t12z.global.2p00.f000.grib2.idx │ ├── ... │ ├── ... │ ├── gfs.wave.t12z.global.2p00.f120.grib2 │ └── gfs.wave.t12z.global.2p00.f120.grib2.idx └── station ├── gfs.wave.t12z.t12z.bull_tar ├── gfs.wave.t12z.t12z.cbull_tar ├── gfs.wave.t12z.t12z.ibpbull_tar ├── gfs.wave.t12z.t12z.ibpcbull_tar ├── gfs.wave.t12z.t12z.ibp_tar └── gfs.wave.t12z.t12z.spec_tar.gz 4 directories, 369 files ``` The forecast model produced wave model output as: ```bash ❯❯❯ tree COM/gfs.20210323/12/model/wave/ ├── history │ ├── gfs.wave.t12z.log.uglo_100km.2021032312 │ ├── gfs.wave.t12z.points.f000.bin │ ├── gfs.wave.t12z.points.f001.bin │ ├── ... │ ├── gfs.wave.t12z.points.f120.bin │ ├── gfs.wave.t12z.uglo_100km.f000.bin │ ├── ... │ ├── ... │ ├── ... │ └── gfs.wave.t12z.uglo_100km.f120.bin ├── prep │ ├── gfs.wave.t12z.mod_def.glo_200.bin │ └── gfs.wave.t12z.mod_def.uglo_100km.bin └── restart 3 directories, 245 files ``` # Checklist - [ ] Any dependent changes have been merged and published - [ ] My code follows the style guidelines of this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have documented my code, including function, input, and output descriptions - [ ] My changes generate no new warnings - [ ] New and existing tests pass with my changes - [ ] This change is covered by an existing CI test or a new one has been added - [ ] Any new scripts have been added to the .github/CODEOWNERS file with owners - [ ] I have made corresponding changes to the system documentation if necessary
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Wave products are renamed to be uniform with other components and follow NCO implementation standards. This means all output is in the format
${RUN}.wave.tCCz.product[.fHHH][.domain].suffix.Some wave scripts are refactored to make them consistent with other parts of the workflow. All scripts for those jobs now pass
shellcheckwithout warnings. Point job scripts were only updated for the file rename, as those scripts will be redone in the near future anyway.The biggest changes in the refactor include beyond those necessary for the file rename include:
run_mpmd.shscriptsedOne functional change is raw interpolated grids are no longer copied to COM, only the resulting grib files. Need to confirm with Jessica this is okay.
Resolves #296
Resolves #3270
Type of change
Change characteristics
How has this been tested?
Checklist