Skip to content

Use err_chk in GSI J-Jobs and scripts #3549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DavidHuber-NOAA merged 14 commits into from
Apr 9, 2025
Merged

Use err_chk in GSI J-Jobs and scripts #3549

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
aa0a2e7
Merge err_chk forecast branch
DavidHuber-NOAA Apr 3, 2025
02f0115
Initial stab at err_chk-ing GSI jobs
DavidHuber-NOAA Apr 3, 2025
dba190b
Merge remote-tracking branch 'origin/develop' into feature/gsi_err
DavidHuber-NOAA Apr 4, 2025
170e77e
Double quote err
DavidHuber-NOAA Apr 4, 2025
c2f3a87
Add missing true
DavidHuber-NOAA Apr 4, 2025
72018e7
Run err_chk on separate line
DavidHuber-NOAA Apr 4, 2025
2fc4a88
Apply suggestions from code review
DavidHuber-NOAA Apr 4, 2025
43ef9aa
Double quote pgmout
DavidHuber-NOAA Apr 4, 2025
e1a089c
Remove pipefail wrap that's no longer needed
DavidHuber-NOAA Apr 4, 2025
5162e7f
Merge branch 'develop' into feature/gsi_err
DavidHuber-NOAA Apr 7, 2025
17356cb
Merge remote-tracking branch 'origin/develop' into feature/gsi_err
DavidHuber-NOAA Apr 8, 2025
f665740
Do not log err_chk calls with error messages
DavidHuber-NOAA Apr 8, 2025
c30d08e
Merge branch 'develop' into feature/gsi_err
DavidHuber-NOAA Apr 9, 2025
13d49ba
Merge branch 'develop' into feature/gsi_err
DavidHuber-NOAA Apr 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions jobs/JGDAS_ATMOS_ANALYSIS_DIAG
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,9 @@ mkdir -m 775 -p "${COMOUT_ATMOS_ANALYSIS}"

###############################################################
# Run relevant script
${ANALDIAGSH:-${SCRgfs}/exglobal_diag.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ANALDIAGSH:-${SCRgfs}/exglobal_diag.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
9 changes: 3 additions & 6 deletions jobs/JGDAS_ATMOS_CHGRES_FORENKF
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,9 @@ MEMDIR="mem001" YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_ATMOS_HISTORY_MEM

###############################################################
# Run relevant script
${CHGRESFCSTSH:-${SCRgfs}/exgdas_atmos_chgres_forenkf.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${CHGRESFCSTSH:-${SCRgfs}/exgdas_atmos_chgres_forenkf.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
9 changes: 3 additions & 6 deletions jobs/JGDAS_ATMOS_VERFOZN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,9 @@ if [[ ! -d ${TANKverf_ozn} ]]; then mkdir -p ${TANKverf_ozn} ; fi
#-------------------------------------------------------
# Execute the script.
#
"${SCRgfs}/exgdas_atmos_verfozn.sh"
err=$?
if (( err != 0 )); then
exit "${err}"
fi

"${SCRgfs}/exgdas_atmos_verfozn.sh" && true
export err=$?
err_chk

################################
# Remove the Working Directory
Expand Down
8 changes: 3 additions & 5 deletions jobs/JGDAS_ATMOS_VERFRAD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,9 @@ if [[ ! -d ${TANKverf_rad} ]]; then mkdir -p ${TANKverf_rad} ; fi

########################################################
# Execute the script.
"${SCRgfs}/exgdas_atmos_verfrad.sh"
err=$?
if (( err != 0 )); then
exit "${err}"
fi
"${SCRgfs}/exgdas_atmos_verfrad.sh" && true
export err=$?
err_chk

################################
# Remove the Working Directory
Expand Down
13 changes: 5 additions & 8 deletions jobs/JGDAS_ENKF_DIAG
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ MEMDIR="ensstat" RUN=${GDUMP_ENS} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \

export ATMGES_ENSMEAN="${COMIN_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006.ensmean.nc"
if [ ! -f ${ATMGES_ENSMEAN} ]; then
echo "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
exit 1
export err=1
err_chk "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
fi

# Link observational data
Expand Down Expand Up @@ -99,12 +99,9 @@ done
###############################################################
# Run relevant script

${ANALDIAGSH:-${SCRgfs}/exglobal_diag.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ANALDIAGSH:-${SCRgfs}/exglobal_diag.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
9 changes: 3 additions & 6 deletions jobs/JGDAS_ENKF_ECEN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,9 @@ MEMDIR="ensstat" RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
###############################################################
# Run relevant script

${ENKFRECENSH:-${SCRgfs}/exgdas_enkf_ecen.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ENKFRECENSH:-${SCRgfs}/exgdas_enkf_ecen.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
9 changes: 3 additions & 6 deletions jobs/JGDAS_ENKF_POST
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,9 @@ export LEVS=$((LEVS-1))
###############################################################
# Run relevant script

${ENKFPOSTSH:-${SCRgfs}/exgdas_enkf_post.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ENKFPOSTSH:-${SCRgfs}/exgdas_enkf_post.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
27 changes: 12 additions & 15 deletions jobs/JGDAS_ENKF_SELECT_OBS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,18 +51,18 @@ mkdir -p "${COMOUT_ATMOS_ANALYSIS}"

export ATMGES_ENSMEAN="${COMIN_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006${GSUFFIX}"
if [[ ! -f ${ATMGES_ENSMEAN} ]]; then
echo "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
exit 1
export err=1
err_chk "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
fi

# Ignore masking of chained commands and possible misspelling warning
# shellcheck disable=SC2153,SC2312
set +eu
set -o pipefail
Comment thread
KateFriedman-NOAA marked this conversation as resolved.
Outdated
LEVS=$(${NCDUMP} -h "${ATMGES_ENSMEAN}" | grep -i "pfull" | head -1 | awk -F" = " '{print $2}' | awk -F" " '{print $1}') # get LEVS
Comment thread
DavidHuber-NOAA marked this conversation as resolved.
Outdated
# shellcheck disable=
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi
export err=$?

@aerorahul aerorahul Apr 4, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See if you can use getncdimlen?

@aerorahul aerorahul Apr 4, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Below in this PR, you will see

export LEVS=$(${NCLEN} ${ATMGES} pfull) && true

err_chk
set +o pipefail
set_strict

export LEVS

# Link observational data
Expand Down Expand Up @@ -121,12 +121,9 @@ done
###############################################################
# Run relevant script

${INVOBSSH:-${SCRgfs}/exgdas_enkf_select_obs.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${INVOBSSH:-${SCRgfs}/exgdas_enkf_select_obs.sh} && true
export err=$?
err_chk

##############################################
# Send Alerts
Expand Down
9 changes: 3 additions & 6 deletions jobs/JGDAS_ENKF_SFC
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,9 @@ RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
###############################################################
# Run relevant script

${ENKFRESFCSH:-${SCRgfs}/exgdas_enkf_sfc.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ENKFRESFCSH:-${SCRgfs}/exgdas_enkf_sfc.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
9 changes: 3 additions & 6 deletions jobs/JGDAS_ENKF_UPDATE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,9 @@ MEMDIR="ensstat" RUN="enkfgdas" YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \
###############################################################
# Run relevant script

${ENKFUPDSH:-${SCRgfs}/exgdas_enkf_update.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ENKFUPDSH:-${SCRgfs}/exgdas_enkf_update.sh} && true
export err=$?
err_chk

##############################################
# Send Alerts
Expand Down
26 changes: 10 additions & 16 deletions jobs/JGLOBAL_ATMOS_ANALYSIS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,23 +45,20 @@ mkdir -p "${COMOUT_ATMOS_ANALYSIS}"

export ATMGES="${COMIN_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006.nc"
if [ ! -f ${ATMGES} ]; then
echo "FATAL ERROR: FILE MISSING: ATMGES = ${ATMGES}"
exit 1
export err=1
err_chk "FATAL ERROR: FILE MISSING: ATMGES = ${ATMGES}"
fi

# Get LEVS
export LEVS=$(${NCLEN} ${ATMGES} pfull)
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

export LEVS=$(${NCLEN} ${ATMGES} pfull) && true
export err=$?
err_chk

if [ ${DOHYBVAR} = "YES" ]; then
export ATMGES_ENSMEAN="${COMIN_ATMOS_HISTORY_ENS_PREV}/${GPREFIX_ENS}atmf006.ensmean.nc"
if [ ! -f ${ATMGES_ENSMEAN} ]; then
echo "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
exit 2
export err=2
err_chk "FATAL ERROR: FILE MISSING: ATMGES_ENSMEAN = ${ATMGES_ENSMEAN}"
fi
fi

Expand Down Expand Up @@ -89,12 +86,9 @@ fi
###############################################################
# Run relevant script

${ANALYSISSH:-${SCRgfs}/exglobal_atmos_analysis.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ANALYSISSH:-${SCRgfs}/exglobal_atmos_analysis.sh} && true
export err=$?
err_chk

##############################################
# Send Alerts
Expand Down
22 changes: 8 additions & 14 deletions jobs/JGLOBAL_ATMOS_ANALYSIS_CALC
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,14 @@ RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \

export ATMGES="${COMIN_ATMOS_HISTORY_PREV}/${GPREFIX}atmf006.nc"
if [ ! -f ${ATMGES} ]; then
echo "FATAL ERROR: FILE MISSING: ATMGES = ${ATMGES}"
exit 1
export err=1
err_chk "FATAL ERROR: FILE MISSING: ATMGES = ${ATMGES}"
fi

# Get LEVS
export LEVS=$(${NCLEN} ${ATMGES} pfull)
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

export LEVS=$(${NCLEN} ${ATMGES} pfull) && true
export err=$?
err_chk

# Generate Gaussian surface analysis # TODO: Should this be removed now that sfcanl is its own job?
export DOGAUSFCANL=${DOGAUSFCANL:-"YES"}
Expand All @@ -60,12 +57,9 @@ export DOGAUSFCANL=${DOGAUSFCANL:-"YES"}
###############################################################
# Run relevant script

${ANALCALCSH:-${SCRgfs}/exglobal_atmos_analysis_calc.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${ANALCALCSH:-${SCRgfs}/exglobal_atmos_analysis_calc.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
9 changes: 3 additions & 6 deletions jobs/JGLOBAL_ATMOS_SFCANL
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,9 @@ mkdir -p "${COMOUT_ATMOS_RESTART}"
###############################################################
# Run relevant script

${SFCANALSH:-${SCRgfs}/exglobal_atmos_sfcanl.sh}
err=$?
if [[ ${err} -ne 0 ]]; then
exit "${err}"
fi

${SFCANALSH:-${SCRgfs}/exglobal_atmos_sfcanl.sh} && true
export err=$?
err_chk

##############################################
# End JOB SPECIFIC work
Expand Down
8 changes: 3 additions & 5 deletions jobs/JGLOBAL_ATMOS_VMINMON
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,9 @@ if [[ ! -d ${M_TANKverf} ]]; then mkdir -p "${M_TANKverf}" ; fi

########################################################
# Execute the script.
"${SCRgfs}/exglobal_atmos_vminmon.sh"
err=$?
if (( err != 0 )); then
exit "${err}"
fi
"${SCRgfs}/exglobal_atmos_vminmon.sh" && true
export err=$?
err_chk

################################
# Remove the Working Directory
Expand Down
4 changes: 2 additions & 2 deletions scripts/exgdas_atmos_chgres_forenkf.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ EOF
export pgm=$CHGRESNCEXEC
. prep_step

$APRUN_CHGRES $CHGRESNCEXEC chgres_nc_gauss0$FHR.nml
$APRUN_CHGRES $CHGRESNCEXEC chgres_nc_gauss0$FHR.nml && true
Comment thread
DavidHuber-NOAA marked this conversation as resolved.
Outdated
export err=$?; err_chk
Comment thread
DavidHuber-NOAA marked this conversation as resolved.
Outdated
fi
done
Expand All @@ -169,7 +169,7 @@ EOF
export pgm=$CHGRESNCEXEC
. prep_step

$APRUNCFP_CHGRES $DATA/mp_chgres.sh
$APRUNCFP_CHGRES $DATA/mp_chgres.sh && true
Comment thread
DavidHuber-NOAA marked this conversation as resolved.
Outdated
export err=$?; err_chk
Comment thread
DavidHuber-NOAA marked this conversation as resolved.
Outdated
fi
fi
Expand Down
12 changes: 6 additions & 6 deletions scripts/exgdas_atmos_verfozn.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# (OznMon) DA package.
#
################################################################################
err=0
export err=0

data_available=0

Expand Down Expand Up @@ -36,13 +36,13 @@ if [[ -s ${oznstat} ]]; then

export OZNMON_NETCDF=${netcdf}

"${USHgfs}/ozn_xtrct.sh"
err=$?
"${USHgfs}/ozn_xtrct.sh" && true
export err=$?

else
# oznstat file not found
err=1
export err=1
fi

exit ${err}

err_chk
exit 0
Loading