Skip to content

[develop]: Remove shell workflow.#764

Merged
danielabdi-noaa merged 22 commits into
ufs-community:developfrom
danielabdi-noaa:feature/remove_shell_workflow
Jul 18, 2022
Merged

[develop]: Remove shell workflow.#764
danielabdi-noaa merged 22 commits into
ufs-community:developfrom
danielabdi-noaa:feature/remove_shell_workflow

Conversation

@danielabdi-noaa
Copy link
Copy Markdown
Collaborator

@danielabdi-noaa danielabdi-noaa commented May 11, 2022

DESCRIPTION OF CHANGES:

This PR removes the shell workflow. Almost all of the duplicate files under ush/ are removed, choosing either the shell or python version. Two files still need both versions since it turned out to be difficult to use the python versions in job scripts without regional_workflow environment activated there. The PR will make updates much easier because there is only one workflow generation path to worry about.

TESTS CONDUCTED:

I have run tests successfully on HERA, ORION and JET. The set of tests are taken from a recent run using the shell workflow generation path done in PR #744 slighltly modified to use the new "custom_ESG/GFDLgrid" names.

Out of 56 tests

  • community_ensemble_008mems
  • community_ensemble_2mems
  • deactivate_tasks
  • grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_CONUS_3km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_AK_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_2017_gfdlmp
  • grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_HRRR
  • grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_RRFS_v1beta
  • grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
  • grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15_thompson_mynn_lam3km
  • grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha
  • grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_RRFS_v1beta
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha
  • grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • grid_RRFS_CONUScompact_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha
  • grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • grid_RRFS_NA_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta
  • grid_RRFS_SUBCONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_RRFS_SUBCONUS_3km_ics_HRRR_lbcs_RAP_suite_GFS_v15p2
  • grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
  • grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_HRRR
  • grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
  • inline_post
  • MET_ensemble_verification
  • MET_verification
  • nco_ensemble
  • nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
  • custom_ESGgrid
  • custom_GFDLgrid
  • custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_FALSE
  • custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_TRUE
  • pregen_grid_orog_sfc_climo
  • specify_DOT_OR_USCORE
  • specify_DT_ATMOS_LAYOUT_XY_BLOCKSIZE
  • specify_RESTART_INTERVAL
  • specify_template_filenames

these are the failures where the reason is mentioned in brackets

HERA
0 failures

ORION
1 failure

nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR (run_fcst segfault)

JET
only 5 failures which is a lot of improvement since last time I tried.

MET_ensemble_verification (run_gridstatvx_)
MET_verification (run_gridstatvx)
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR (make_lbcs)
nco_ensemble (run_fcst)
pregen_grid_orog_sfc_climo (run_fcst)

DEPENDENCIES:

None

DOCUMENTATION:

None

ISSUE (optional):

None

CONTRIBUTORS (optional):

@christinaholtNOAA @venitahagerty

@JeffBeck-NOAA
Copy link
Copy Markdown
Collaborator

JeffBeck-NOAA commented May 22, 2022

WE2E using the config.community.sh file succeeded on Hera.

@JeffBeck-NOAA JeffBeck-NOAA added the Tested on Hera Tested successfully on Hera machine label May 22, 2022
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch 3 times, most recently from d43cfe2 to 36e42b7 Compare May 23, 2022 16:52
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch from 36e42b7 to 8659e07 Compare May 23, 2022 17:45
@danielabdi-noaa danielabdi-noaa added the Tested on Jet Successfully tested on Jet machine label Jun 16, 2022
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch 2 times, most recently from 7fc93ac to cd22251 Compare June 27, 2022 19:30
@danielabdi-noaa danielabdi-noaa force-pushed the feature/remove_shell_workflow branch from cd22251 to 9f88f60 Compare July 14, 2022 20:25
@JeffBeck-NOAA JeffBeck-NOAA added the Tested on Cheyenne Successfully tested on NCAR Cheyenne machine label Jul 15, 2022
Copy link
Copy Markdown
Contributor

@christinaholtNOAA christinaholtNOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@danielabdi-noaa danielabdi-noaa merged commit a1885f4 into ufs-community:develop Jul 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Tested on Cheyenne Successfully tested on NCAR Cheyenne machine Tested on Hera Tested successfully on Hera machine Tested on Jet Successfully tested on Jet machine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants