Skip to content

Update tests #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
robgpita merged 9 commits into from
Sep 13, 2021
Merged

Update tests #1

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
19e87cc
Merge pull request #3 from NOAA-EMC/develop
christinaholtNOAA Sep 22, 2020
3a45093
Merge remote-tracking branch 'dtc/develop' into develop
christinaholtNOAA Nov 6, 2020
4def0e5
Merge remote-tracking branch 'gsl/rrfs_ci' into update_tests
christinaholtNOAA Aug 19, 2021
e64f40a
Merge remote-tracking branch 'origin' into update_tests
christinaholtNOAA Aug 31, 2021
9d321a7
Merge remote-tracking branch 'gsl/rrfs_ci' into update_tests
christinaholtNOAA Aug 31, 2021
bb37616
Moving script consistent with new dir.
christinaholtNOAA Sep 2, 2021
0647f1f
Remove old script.
christinaholtNOAA Sep 2, 2021
0359954
Clean up a few minor things.
christinaholtNOAA Sep 2, 2021
b5c80fe
Merge branch 'rrfs_ci' of https://github.com/robgonzalezpita/regional…
christinaholtNOAA Sep 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 38 additions & 27 deletions tests/end_to_end_tests.sh → tests/WE2E/end_to_end_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,44 +3,55 @@
#----------------------------------------------------------------------
# Automation of UFS Short Range Weather App Worfklow End to End Tests.
#
# The script is dependant on a successful build of this repo.
# The script is dependant on a successful build of this repo using the
# test/build.sh script in the ufs-srweather-app repository.
# The UFS build must be completed in a particular manner for this
# script to function properly, notably the location of the build and
# bin directories:
# BUILD_DIR=${APP_DIR}/build_${compiler}
# BIN_DIR=${APP_DIR}/bin_${compiler}
#
# BUILD_DIR=${APP_DIR}/build_${compiler}
# BIN_DIR=${APP_DIR}/bin_${compiler}
#
# With the creating of a test_log directory and output files describing
# the result of each workflow status, a testing framework is established.
# The user must verify the contents of each experiment file in test_log/
# to check workflow success or failure.
# to check workflow success or failure.
#
# Example: . end_to_end_tests.sh hera zrtrr
# Example: ./end_to_end_tests.sh hera zrtrr

@christinaholtNOAA christinaholtNOAA Sep 3, 2021

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I changed the instructions to execute instead of source this scripts. It's a bit safer.

#----------------------------------------------------------------------

#-----------------------------------------------------------------------
# Set variables
#-----------------------------------------------------------------------

branches=( rrfs_ci )
branches=( rrfs_ci )

function usage {
echo
echo "Usage: $0 machine slurm_account | -h"
echo
echo " machine [required] is one of: ${machines[@]}"
echo " slurm_account [required] case sensitive name of the user-specific slurm account"
echo " -h display this help"
echo
exit 1

}

machines=( hera )

if [ "$1" = "-h" ] ; then usage ; fi
[[ $# -le 1 ]] && usage
Comment thread
robgpita marked this conversation as resolved.

export machine=$1
machine=`echo "${machine}" | tr '[A-Z]' '[a-z]'` # scripts in sorc need lower case machine name
machine=$(echo "${machine}" | tr '[A-Z]' '[a-z]') # scripts in sorc need lower case machine name
Comment thread
robgpita marked this conversation as resolved.

export account=$2
account=`echo "${account}"`

Comment thread
robgpita marked this conversation as resolved.

#-----------------------------------------------------------------------
# Choose experiment. ( Full list in ./baselines_list.txt)
# Choose experiment.
Comment thread
robgpita marked this conversation as resolved.
#-----------------------------------------------------------------------

#expts=( grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha )
expts=( grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2 )
#expts=( grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2 )
#expts=( grid_RRFS_CONUS_13km_ics_HRRR_lbcs_RAP_suite_HRRR )
#expts=( inline_post )
#expts=( grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2 grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha )
expts=( grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GSD_SAR )

@christinaholtNOAA christinaholtNOAA Sep 3, 2021

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just removing commented code here.

@robgpita robgpita Sep 8, 2021

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do agree with removing the commented code, but if we'd like to run more than one experiment in the future, I believe we should provide additional information or some sort of "skeleton".

I left the extras in here to change which experiments to run. It is easy to toggle the commented experiments and re-run this script. As there were errors with finding IC/LBCS with particular experiments, I wanted to have a couple different experiments on deck. I reckon it would be nice to have a way to submit multiple experiments easily.

Do we only want to have one 'test' (experiment) for now?

If we want to accommodate more than one experiment, do we want to provide those as a config file, as CLA, or uncommenting code?


#-----------------------------------------------------------------------
# Set directories
Expand All @@ -50,10 +61,11 @@ scrfunc_fp=$( readlink -f "${BASH_SOURCE[0]}" )
scrfunc_fn=$( basename "${scrfunc_fp}" )
scrfunc_dir=$( dirname "${scrfunc_fp}" )

REGIONAL_WORKFLOW_DIR=$( dirname "${scrfunc_dir}" )
WE2E_DIR=$( dirname "${scrfunc_dir}" )
TESTS_DIR=$( dirname "${WE2E_DIR}" )
REGIONAL_WORKFLOW_DIR=$( dirname "${TESTS_DIR}" )

@robgpita robgpita Sep 8, 2021

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for cleaning this up @christinaholtNOAA

SRW_APP_DIR=$( dirname "${REGIONAL_WORKFLOW_DIR}" )
TOP_DIR=$( dirname "${SRW_APP_DIR}" )
TESTS_DIR=${REGIONAL_WORKFLOW_DIR}/tests

#BRANCH_DIR_NAME=ufs-srweather-app-${branches}
APP_DIR=${TOP_DIR}/ufs-srweather-app
Expand All @@ -70,18 +82,17 @@ sed -i 's|EXECDIR="${SR_WX_APP_TOP_DIR}/bin"|EXECDIR="${SR_WX_APP_TOP_DIR}/bin_i
#-----------------------------------------------------------------------

# Load Python Modules
source ${APP_DIR}/env/wflow_${machine}.env
cmd="${APP_DIR}/env/wflow_${machine}.env"
source ${cmd}

@christinaholtNOAA christinaholtNOAA Sep 3, 2021

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reduced a bit of duplicate information here by using the variable cmd.

@robgpita robgpita Sep 8, 2021

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for cleaning this up as well.


echo "-- Python command =>" $cmd
echo "-- Load environment =>" $cmd

# If experiments list file exists, remove it, and add the experiemnts to a new expts list file
if [ -f "auto_expts_list.txt" ] ; then
rm "auto_expts_list.txt"
fi
echo ${expts} > auto_expts_list.txt
auto_file="auto_expts_list.txt"

@christinaholtNOAA christinaholtNOAA Sep 3, 2021

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of the file was specified in several locations. If we put it in a variable, we could move it anywhere, and rename and only have to change it in one spot.


rm -rf ${auto_file}

@christinaholtNOAA christinaholtNOAA Sep 3, 2021

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use flags on rm that don't require checking for the existence of the file -- one less file system check, which can be expensive.

@robgpita robgpita Sep 8, 2021

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. Don't know why I didn't simplify this to rm -rf.

echo ${expts} > ${auto_file}

# Run the E2E Workflow tests
#./run_experiments.sh expts_file=auto_expts_list.txt machine=${machine} account=${account} use_cron_to_relaunch=FALSE
./run_experiments.sh expts_file=auto_expts_list.txt machine=${machine} account=${account}
./run_WE2E_tests.sh tests_file=${auto_file} machine=${machine} account=${account}
Comment thread
robgpita marked this conversation as resolved.