Add automated baseline jobs to automated regression testing#491
Closed
BrianCurtis-NOAA wants to merge 23 commits into
Closed
Add automated baseline jobs to automated regression testing#491BrianCurtis-NOAA wants to merge 23 commits into
BrianCurtis-NOAA wants to merge 23 commits into
Conversation
…s it should not before starting the job
Merged
22 tasks
DusanJovic-NOAA
pushed a commit
that referenced
this pull request
Mar 31, 2021
…sion test script updates (#498) - Updates to auto-rt in PR #491 from @BrianCurtis-NOAA - Fixes to the regression testing scripts in branch https://github.com/DusanJovic-NOAA/ufs-weather-model/tree/rt_log_dir_fix from @DusanJovic-NOAA - Change baseline location on gaea to where emc.nemspara can write - Update of submodule pointers for fv3atm, ccpp-framework and ccpp-physics, and new Thompson MP lookup tables for the changes described in the associated PRs below - Remove tests fv3_gfs_v15p2_RRTMGP and fv3_gfs_v15p2_RRTMGP_debug from `tests/rt_gnu.conf` (same errors as with other GFS v15p2 tests previously, `compute_qs: saturation vapor pressure table overflow`)
pjpegion
pushed a commit
to NOAA-PSL/ufs-weather-model
that referenced
this pull request
Apr 4, 2023
* add p8 suite files * fix 2phases intermediate restart Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov> Co-authored-by: Jun Wang <jun.wang@noaa.gov>
epic-cicd-jenkins
pushed a commit
that referenced
this pull request
Apr 17, 2023
…lds [into develop] (#491) * Add capability to pull MRMS echo top data from the NOAA HPSS and run echo top verification through grid-stat * Add seperate processing for C&V/cloud fields to use nearest neighbor interpoloation rather than bilinear. Also add forgotten RETOP conf file. * Updated approach for using nearest neighbor for CandV fields. * Fix syntax issues in PointStat.conf for new interpolation functionality. Increase maxtries for post task to 2 since it fails occasionally but usually runs the second time. * Move back to 1 day increment to match what is currently in develop * Add grid-stat task in xml to run RETOP verification
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Checklist
Ths PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.
This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR
An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
are specified below.
If new or updated input data is required by this PR, it is clearly stated in the text of the PR.
Instructions: All subsequent sections of text should be filled in as appropriate.
The information provided below allows the code managers to understand the changes relevant to this PR, whether those changes are in the ufs-weather-model repository or in a subcomponent repository. Ufs-weather-model code managers will use the information provided to add any applicable labels, assign reviewers and place it in the Commit Queue. Once the PR is in the Commit Queue, it is the PR owner's responsiblity to keep the PR up-to-date with the develop branch of ufs-weather-model.
Description
This PR adds automated baseline jobs to the automated regression testing code
I've separated the jobs into their own files. This allows for easier creation or addition of additional jobs in the future.
I've moved most of the work that rt-auto.sh was doing inside rt_auto.py and the job files. Renamed it start_rt_auto.sh so it can be used as a wrapper for cron jobs so a login shell environment can be used.
I've tested on Hera without emc.nemspara. the reasoning below is important
IMPORTANT NOTE
1: There is a change in rt.sh that moved the baseline date outside of the section it's hard coded into. This allows for me to find the date easily and edit it each time a new baseline test is run. I've talked with @MinsukJi-NOAA with his initial approval of the change not effecting the rest of the code.
2: The code checks the first comment (the body) of the PR for a line that must be by itself that says "BL_DATE: 20210331" and the code will check if its YYYYMMDD format. If there is already a directory that exists with that date the code will fail because it's not supposed to exist until the code creates it.
3: Before baseline code can be used, the new branch feature/autort-bl needs to be implemented on supported machines.
Issue(s) addressed
Closes #490
Testing
The automated RT an be run on everything but WCOSS
The automated BL can be run on: Hera, Jet, Orion. Waiting on emc.nemspara on Gaea, and waiting for account access on Cheyenne. No support for WCOSS currently. May be able to get Cheyenne set up temporarily.
Dependencies
None