Skip to content

Changes needed to run the regional workflow on WCOSS #310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BenjaminBlake-NOAA merged 2 commits into from
Oct 14, 2020
Merged

Changes needed to run the regional workflow on WCOSS #310

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
463f2b1
Changes needed to run the regional workflow on WCOSS
Oct 13, 2020
27b352f
Update exregional_make_sfc_climo.sh
BenjaminBlake-NOAA Oct 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion scripts/exregional_make_grid.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ export OMP_STACKSIZE=2048m
case $MACHINE in


"WCOSS_C" | "WCOSS")
"WCOSS_CRAY")
#
{ save_shell_opts; set +x; } > /dev/null 2>&1

Expand All @@ -121,6 +121,18 @@ case $MACHINE in
ulimit -a
;;

"WCOSS_DELL_P3")
#
{ save_shell_opts; set +x; } > /dev/null 2>&1

module list

{ restore_shell_opts; } > /dev/null 2>&1

export APRUN="mpirun"

ulimit -s unlimited
;;

"HERA")
#
Expand Down
22 changes: 20 additions & 2 deletions scripts/exregional_make_orog.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ export OMP_STACKSIZE=2048m
case $MACHINE in


"WCOSS_C" | "WCOSS")
"WCOSS_CRAY")
#
{ save_shell_opts; set +x; } > /dev/null 2>&1

Expand All @@ -117,6 +117,11 @@ case $MACHINE in
ulimit -a
;;

"WCOSS_DELL_P3")
ulimit -s unlimited
ulimit -a
APRUN="mpirun"
;;

"HERA")
ulimit -s unlimited
Expand Down Expand Up @@ -286,7 +291,7 @@ Starting orography file generation..."
case $MACHINE in


"WCOSS_C" | "WCOSS")
"WCOSS_CRAY")
#
# On WCOSS and WCOSS_C, use cfp to run multiple tiles simulatneously for
# the orography. For now, we have only one tile in the regional case,
Expand All @@ -312,6 +317,19 @@ ${tmp_dir}" \
;;


"WCOSS_DELL_P3")

ufs_utils_ushdir="${UFS_UTILS_DIR}/ush"
res="0" # What should this be set to???

"${exec_fp}" < "${input_redirect_fn}" || \
print_err_msg_exit "\
Call to executable (exec_fp) that generates the raw orography file returned
with nonzero exit code:
exec_fp = \"${exec_fp}\""
;;


"CHEYENNE" | "HERA" | "JET" | "ODIN" | "STAMPEDE")
$APRUN "${exec_fp}" < "${input_redirect_fn}" || \
print_err_msg_exit "\
Expand Down
16 changes: 12 additions & 4 deletions scripts/exregional_make_sfc_climo.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,14 +128,22 @@ EOF
#
case $MACHINE in

"WCOSS_C")
"WCOSS_CRAY")
# This could be wrong. Just a guess since I don't have access to this machine.

@JeffBeck-NOAA JeffBeck-NOAA Oct 13, 2020

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BenjaminBlake-NOAA, feel free to take out this comment now that things are correctly defined for the WCOSS_CRAY. Thanks!

@BenjaminBlake-NOAA BenjaminBlake-NOAA Oct 13, 2020

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, I missed that comment. I just removed it!

@JeffBeck-NOAA JeffBeck-NOAA Oct 13, 2020

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Approving PR now.

APRUN=${APRUN:-"aprun -j 1 -n 6 -N 6"}
;;

"WCOSS")
# This could be wrong. Just a guess since I don't have access to this machine.
APRUN=${APRUN:-"aprun -j 1 -n 6 -N 6"}
"WCOSS_DELL_P3")

# Specify computational resources.
export NODES=2
export ntasks=48
export ptile=24
export threads=1
export MP_LABELIO=yes
export OMP_NUM_THREADS=$threads

APRUN="mpirun"
;;

"HERA")
Expand Down
16 changes: 2 additions & 14 deletions ush/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -745,25 +745,13 @@ case $MACHINE in
"WCOSS_CRAY")
FIXgsm=${FIXgsm:-"/gpfs/hps3/emc/global/noscrub/emc.glopara/git/fv3gfs/fix/fix_am"}
TOPO_DIR=${TOPO_DIR:-"/gpfs/hps3/emc/global/noscrub/emc.glopara/git/fv3gfs/fix/fix_orog"}
SFC_CLIMO_INPUT_DIR=${SFC_CLIMO_INPUT_DIR:-""}
SFC_CLIMO_INPUT_DIR=${SFC_CLIMO_INPUT_DIR:-"/gpfs/hps3/emc/global/noscrub/emc.glopara/git/fv3gfs/fix/fix_sfc_climo"}
;;

"WCOSS_DELL_P3")
FIXgsm=${FIXgsm:-"/gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/fv3gfs/fix/fix_am"}
TOPO_DIR=${TOPO_DIR:-"/gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/fv3gfs/fix/fix_orog"}
SFC_CLIMO_INPUT_DIR=${SFC_CLIMO_INPUT_DIR:-""}
;;

"DELL")
FIXgsm=${FIXgsm:-"/gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/fv3gfs/fix/fix_am"}
TOPO_DIR=${TOPO_DIR:-"/gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/fv3gfs/fix/fix_orog"}
SFC_CLIMO_INPUT_DIR=${SFC_CLIMO_INPUT_DIR:-""}
;;

"THEIA")
FIXgsm=${FIXgsm:-"/scratch4/NCEPDEV/global/save/glopara/git/fv3gfs/fix/fix_am"}
TOPO_DIR=${TOPO_DIR:-"/scratch4/NCEPDEV/global/save/glopara/git/fv3gfs/fix/fix_orog"}
SFC_CLIMO_INPUT_DIR=${SFC_CLIMO_INPUT_DIR:-"/scratch4/NCEPDEV/da/noscrub/George.Gayno/climo_fields_netcdf"}
SFC_CLIMO_INPUT_DIR=${SFC_CLIMO_INPUT_DIR:-"/gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/fv3gfs/fix/fix_sfc_climo"}
;;

"HERA")
Expand Down