[develop] Improvements for WE2E tests: script features, additional tests, remove unsupported domains#871

Merged

MichaelLueken merged 40 commits into

ufs-community:developfrom

mkavulich:feature/WE2E_test_improvement_round_3_rebased

Sep 8, 2023

Commits on Aug 15, 2023

Add functions to setup.py that remove metatask dependencies for any metatasks that do not exist or have been removed by other checks.
mkavulich
committed
Improving testing scripts
mkavulich
committed
- Lint run_WE2E_tests.py (not 10/10 yet), add checks to ensure that
mkavulich
committed
Add new custom domains of varying resolutions and locations. NZ grid is most expensive at ~60 core hours, rest are 30 or less.
mkavulich
committed
- Replace "use_cron_to_relaunch" argument with "launch" argument that can take the values "python", "cron", or "none"
mkavulich
committed
- Add new hi-res (1.1km) SF area case
mkavulich
committed
Explicitly look only on HPSS for HPSS tests
mkavulich
committed
Revert vx_only changes, this isn't a viable route to simplification
mkavulich
committed
Swap tests to get successes...UPP on gnu still seems buggy
mkavulich
committed
Add long forecast (over 100 hours) case
mkavulich
committed
Add long_fcst to machine test files
mkavulich
committed
- Move new snow verification test to "custom_grids" directory, leave
mkavulich
committed
Remove unused predefined domains
mkavulich
committed
- Add argparse for generate_FV3LAM_wflow.py; now includes "debug" flag
mkavulich
committed
Gaea needs to use the same comprehensive suite as Orion (which also does not have HPSS access)
mkavulich
committed
long_fcst test uses AWS data, so can not be run on Cheyenne
mkavulich
committed
Fix write component grids for new custom domains
mkavulich
committed
Add new '--mode' flag to monitor_jobs.py to allow for "advance" mode that will only run rocotorun once for each experiment rather than continuously monitoring
mkavulich
committed
Fix SF test write componant grid
mkavulich
committed
Add default (False) for new debug argument, should fix failing test
mkavulich
committed
Fix linting for generate_FV3LAM_wflow.py
mkavulich
committed

Commits on Aug 16, 2023

Commits on Aug 22, 2023

Move New Zealand test away from Hera GNU (GNU compilers run significantly slower), speed up the slower tests and expand walltime
mkavulich
committed

Commits on Sep 5, 2023