Update lmod, miniconda, task modulefiles for Gaea by EdwardSnyder-NOAA · Pull Request #353 · ufs-community/ufs-srweather-app

EdwardSnyder-NOAA · 2022-09-16T22:34:28Z

DESCRIPTION OF CHANGES:

Updated lmod, hpc-stack, miniconda, and task modulefiles so that we can run the WE2E tests on Gaea. Two PRs were created that address these issues in more detail for the release/public-v2 branch of the SRW App (SRW: #352 and Regional Workflow: #830). The quick tests and MET_verification tests will be conducted on Gaea via the Jenkins pipeline.

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

TESTS CONDUCTED:

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

CHECKLIST

My code follows the style guidelines in the Contributor's Guide
I have performed a self-review of my own code using the Code Reviewer's Guide
I have commented my code, particularly in hard-to-understand areas
My changes need updates to the documentation. I have made corresponding changes to the documentation
My changes do not require updates to the documentation (explain).
My changes generate no new warnings
Noticed a pmi/alps error in the make_orog and make_grid log files, however the tasks still complete.

Tue Sep 27 17:15:43 2022: [unset]:_pmi_alps_init:alps_get_placement_info returned with error -1
Tue Sep 27 17:15:43 2022: [unset]:_pmi_init:_pmi_alps_init returned -1

New and existing tests pass with my changes
Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

CONTRIBUTORS (optional):

@natalie-perlin for her help with the configuration and installations of: hpc-stack, lmod, and miniconda3 on Gaea, as well as testing and troubleshooting the MET_verification WE2E test.

danielabdi-noaa · 2022-09-20T22:21:20Z

@EdwardSnyder-NOAA I got a miniconda3/4.12.0 not found error when trying to build srw app on gaea with this PR, so I suppose there are still issues to resolve ?

natalie-perlin

May or may not need changes in load_modules_run_task.sh

EdwardSnyder-NOAA · 2022-09-21T15:17:52Z

@danielabdi-noaa that was a typo I fixed with yesterday's commit. Currently, this PR will crash in the middle of building the SRW App. It is an error building the UPP. I haven't had time to look into this error yet.

* Add preamble script from global workflow. * Call preamble script in j-jobs and ex-scripts * Call preamble in other scripts. * Make names of j-jobs and ex-scripts consistent. * Working towards nco vars in table 1. * Change default bin directory to exec * Appen FATAL ERROR to print_err_msg_exit. * Replace some cp, cd, mkdir calls with their corresponding _vrfy versions * Add job and jobid to the job-card. * Add cyc and subcyc to rocoto xml * Add a j-job preamble script for setpdy. * Add a j-job postamble as well. * Define some Table 1 vars in setup. * Remove unused SRC_DIR, and rename others * Rename CYCLE_BASEDIR to COMIN_BASEDIR * Create the NCO root directories in setup. * Remove source machine file wrapper. * Bug fix in job_preamble. * Make make_ics/lbcs use DATA directory properly. * Make run_fcst use DATA directory properly. * Made run_post use DATA directory properly. * Make make_grid use DATA properly (untested). * Make make_sfc_climo use DATA properly (untested). * Make make_orog use DATA properly (untested). * Bug fix for none-nco mode. * Don't pass arguments from j-jobs to ex-scripts. * Make forecast and post-output go to COMOUT. * Remove CYCLE_DIR and use COMIN instead. * Bug fix for community mode. * Append cyc to COMIN in NCO mode. * Fix rocoto run_post dependency with run_fcst issue. * Use OPSROOT instead of PTMP and STMP. * Move nco vars in config_defaults. * Move logdir location to COMROOT. * Set all root directories to EXPTDIR in community mode. * Use pgmout and pgmerr. * Fix inline post. * Make pgmout/err redirection work with community mode. * Use print_err in get_obs_mrms. * Add prep_step. * Add post_step. * Add dbn_alert to post-processed grib2 output. * Download extrn files directly to COMIN. * Make make_ics/lbcs directly output to COMIN. * Change names of extrn_mdl_var_defns files. * Name fixes for DO_ENSEMBLE=false, dyn/phy * Don't create symlinks to grib2 files in NCO mode. * Append rrfs to make_ics/lbcs output. * Modify extrn_mdl_var_defns names. * Move forecast output to DATA/RUN.PDY. This location can be used to store output of other tasks as well. * Move templates to parm. * Fix for new parm location. * Move metplus one level up. * Fixes for community mode. * Rename SCRIPTSDIR and JOBSDIR. * Move all FIX** directories in to a fix/ directory. * Make FIXrrfs be EXPTDIR for community mode. * Symlink upp and ufs_utils parm files to top level parm directory. * Remove UPP_DIR and UFS_UTILS_DIR. * Define cycle with subcyc when it is non-zero. * Don't delete COMIN_BASEDIR if it already exists. * Disassociate NCO mode from pre-generated grid. * Don't choose fix location based on RUN_ENVIR. * Bug fix in make_lbcs. * Add flag to symlink or copy fix files. * Change slurm log file locations * Minor fix for inline post in nco mode. * Add unique workflow ID to avoid clashes between different runs, while keeping the relation between different tasks, which PID can not do. * Make verification tasks NCO complaint. * Pass RUN_ENVIR to we2e script. * Fixes for merge conflicts. * Add versions for wcoss2. * Fix symlinks. * Minor changes. * Move grid/orog/sfcc completion files to EXPTDIR/grid/orog etc. * Output modified namelist file with seeds in current directory. * Fixes for unittests. * Bugfix wrf_io version * Fix CI issue with bin locations. * Allow NCO root directories to be set individually. * Don't append workflow id in community mode. * Add helper script to rename model e.g. rrfs->aqm * Bug fixes and naming changes for consitency. * Replace instances of USHrrfs etc with a generic USHdir etc. * Add unittest for whole workflow now that the merge made it possible. * Remove unused process_args utility. * Remove hard coded paths from configs. * Don't replace existing var value with None. * Add config.nco to unittest. * Fix for Orion issue. * Fix default OPSROOT location in run_we2e. * Modeify setup_we2e script to run fundamental tests on all machines. * Fix conflicting ics/lbcs temp location by moving to DATA. * Bug fix in load_modules taken from PR #353. * Specify default shell instead of symlinking. * Turn off grid/orog/sfc_climo tasks for NCO test cases. * Use PDY and cyc in ex-scripts. * Remove CDATE from xml and define int job_preamble. * Use machine specific list of tests if available. * Run all tests in community mode so that the last NCO test case gets reported as finished. * Minor changes * Avoid using preamble in functions. * Use preamble in function too. * Turn on debugging for utility functions. * Turn on debug & verbose in CI. * Turn off set -e for init_env

The list of modules to be loaded needs updates.

Fixed a typo

natalie-perlin · 2022-09-27T14:47:20Z

@danielabdi-noaa - the module build_gaea_intel needed updates, should be working now!

danielabdi-noaa · 2022-09-27T15:08:59Z

Thanks for fixing the problem. I am able to build the PR successfully on Gaea now, so approving.

natalie-perlin · 2022-09-27T18:19:09Z

Waiting for the PR-830 to get approved in a regional_workflow repo, before merging these changes into the develop branch.

Updated following recent tests of Met verification, as in ufs-srweather-app repository PR-353: ufs-community/ufs-srweather-app#353

@EdwardSnyder-NOAA

* Lmod/8.7.12 init, updated miniconda3 - for Gaea Added new (Lmod/8.7.12) initialization wrapper script to the ENV_INIT_SCRIPTS_FPS variable; added the PROJ_LIB and PATH variables referring to a PROJ package location in the updated miniconda3/4.12.0 and the regional_workflow environment that contain the PROJ package * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/. * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * update run_vx.local Updates to the MetPlus verification script; not yet officially supported in the release of public-v2. These changes and test of the MetPlus script were done by @EdwardSnyder-NOAA * added missing argument for the ./etc/lmod-setup.sh script A bug found by @EdwardSnyder-NOAA; a separate PR to be created into the develop branch * Gaea: Lmod/8.7.12 initialization using a wrapper script, under role.epic account * Gaea: initialize Lmod/8.7.12 using a wrapper script * update a new location of miniconda3/4.12.0, rocoto Specify a new location of the miniconda3/4.12.0 with the regional_workflow environment containing all the necessary packages, and the rocoto/1.3.3 module installed on Gaea under EPIC role account: /lustre/f2/dev/wpo/role.epic/contrib/ * Delete get_extrn_lbcs.local A redundant module; it is placed under ./regional_workflow/modulefiles/tasks/gaea/ instead. * Update load_modules_run_task.sh * Update run_vx.local Updated following recent tests of Met verification, as in ufs-srweather-app repository PR-353: ufs-community/ufs-srweather-app#353 * Update gaea.sh Updated Met Installation locations on Gaea

Parallel Works app-run user added 5 commits September 16, 2022 22:14

update lmod

41d81ef

update lmod

feb6e02

update hpc-stack and miniconda

01fbe01

fix lmod-setup.sh bug for Gaea

3fa91f2

update files to run with new miniconda and MET VX

77e17a1

EdwardSnyder-NOAA changed the title ~~Feature/update gaea lmod modulefile~~ Update lmod, miniconda, task modulefiles for Gaea Sep 16, 2022

climbfuji mentioned this pull request Sep 19, 2022

Explore option to use lmod on Gaea JCSDA/spack-stack#345

Closed

fix typo

9865010

fixed typo

710cd11

natalie-perlin requested changes Sep 21, 2022

View reviewed changes

Comment thread ush/load_modules_run_task.sh

danielabdi-noaa added a commit to danielabdi-noaa/ufs-srweather-app that referenced this pull request Sep 22, 2022

Bug fix in load_modules taken from PR ufs-community#353.

0521d4f

danielabdi-noaa added a commit to danielabdi-noaa/ufs-srweather-app that referenced this pull request Sep 22, 2022

Bug fix in load_modules taken from PR ufs-community#353.

c6f3926

danielabdi-noaa added a commit to danielabdi-noaa/ufs-srweather-app that referenced this pull request Sep 23, 2022

Bug fix in load_modules taken from PR ufs-community#353.

93dd254

danielabdi-noaa added a commit to danielabdi-noaa/ufs-srweather-app that referenced this pull request Sep 25, 2022

Bug fix in load_modules taken from PR ufs-community#353.

1ecee87

update vx task

9dbf2f8

EdwardSnyder-NOAA marked this pull request as ready for review September 26, 2022 22:29

EdwardSnyder-NOAA requested review from BenjaminBlake-NOAA, JeffBeck-NOAA, RatkoVasic-NOAA, chan-hoo, christinaholtNOAA, christopherwharrop-noaa, danielabdi-noaa, gsketefian, mkavulich, panll and ywangwof as code owners September 26, 2022 22:29

EdwardSnyder-NOAA requested review from MichaelLueken, dmwright526, gspetro-NOAA, jkbk2004, mark-a-potts and willmayfield as code owners September 26, 2022 22:29

natalie-perlin approved these changes Sep 27, 2022

View reviewed changes

natalie-perlin added 2 commits September 27, 2022 10:34

Update build_gaea_intel

d70adbc

The list of modules to be loaded needs updates.

Update load_modules_run_task.sh

4418589

Fixed a typo

danielabdi-noaa approved these changes Sep 27, 2022

View reviewed changes

Update load_modules_run_task.sh

67173b9

updated vx task

e22f705

natalie-perlin added a commit to natalie-perlin/regional_workflow that referenced this pull request Sep 27, 2022

Update run_vx.local

5de3fef

Updated following recent tests of Met verification, as in ufs-srweather-app repository PR-353: ufs-community/ufs-srweather-app#353

EdwardSnyder-NOAA merged commit ebe966c into ufs-community:develop Sep 28, 2022

EdwardSnyder-NOAA deleted the feature/update-gaea-lmod-modulefile branch September 28, 2022 14:53

This was referenced Oct 19, 2022

Updating hpc-stack modules and miniconda locations for Hera, Gaea, Cheyenne, Orion, Jet ufs-community/ufs-weather-model#1465

Closed

Update miniconda3/4.12.0 module and regional_workflow environment on Cheyenne, Hera, Jet, Orion #431

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update lmod, miniconda, task modulefiles for Gaea#353

Update lmod, miniconda, task modulefiles for Gaea#353
EdwardSnyder-NOAA merged 12 commits into
ufs-community:developfrom
EdwardSnyder-NOAA:feature/update-gaea-lmod-modulefile

EdwardSnyder-NOAA commented Sep 16, 2022 •

edited

Loading

Uh oh!

danielabdi-noaa commented Sep 20, 2022

Uh oh!

natalie-perlin left a comment

Uh oh!

Uh oh!

EdwardSnyder-NOAA commented Sep 21, 2022

Uh oh!

natalie-perlin commented Sep 27, 2022

Uh oh!

danielabdi-noaa commented Sep 27, 2022

Uh oh!

natalie-perlin commented Sep 27, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

EdwardSnyder-NOAA commented Sep 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DESCRIPTION OF CHANGES:

Type of change

TESTS CONDUCTED:

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

CHECKLIST

LABELS (optional):

CONTRIBUTORS (optional):

Uh oh!

danielabdi-noaa commented Sep 20, 2022

Uh oh!

natalie-perlin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

EdwardSnyder-NOAA commented Sep 21, 2022

Uh oh!

natalie-perlin commented Sep 27, 2022

Uh oh!

danielabdi-noaa commented Sep 27, 2022

Uh oh!

natalie-perlin commented Sep 27, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

EdwardSnyder-NOAA commented Sep 16, 2022 •

edited

Loading