Skip to content

[develop] Fix issues on AQM and NCO mode caused by new YAML interface PR #676 #722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MichaelLueken merged 22 commits into from
Apr 25, 2023
Merged

[develop] Fix issues on AQM and NCO mode caused by new YAML interface PR #676 #722

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
1f8c5e7
fix time-offset issue on gefs lbcs
chan-hoo Apr 5, 2023
86558bf
remove old change
chan-hoo Apr 5, 2023
05db0c2
fix cyc_idx issue on varying fcst_len_hrs
chan-hoo Apr 6, 2023
44a74de
fix undefined cycled
chan-hoo Apr 7, 2023
e6f6ebc
fix wrong mv command for aqm
chan-hoo Apr 7, 2023
761b9e4
fix dependecy issue on xml yaml
chan-hoo Apr 7, 2023
2e6dca5
fix dependency issue on xml
chan-hoo Apr 7, 2023
54b9250
remove wrong space from path to comin dir for nco
chan-hoo Apr 8, 2023
df21e19
Merge branch 'develop' of https://github.com/chan-hoo/ufs-srweather-a…
chan-hoo Apr 8, 2023
a563c75
add comin_dir change for community to setup.py
chan-hoo Apr 8, 2023
34bffb9
fix tmp dir name issue of run_post in nco mode
chan-hoo Apr 10, 2023
715883a
mv complete flag to end of pre_post_stat
chan-hoo Apr 10, 2023
4b4ec26
fix wrong FCST_DIR in nco mode
chan-hoo Apr 10, 2023
58e61e4
add tpp to machine file for bias-correction on wcoss2
chan-hoo Apr 10, 2023
4a18dbc
modify run_post dependency
chan-hoo Apr 10, 2023
02be7bb
remove num_fcst_len_cycl
chan-hoo Apr 16, 2023
7776094
fix typo
chan-hoo Apr 16, 2023
5314fb7
move set -x in scripts
chan-hoo Apr 16, 2023
5301a5a
remove set -x from scripts
chan-hoo Apr 18, 2023
296fedb
remove set -x from j-job
chan-hoo Apr 18, 2023
bca23d9
move defs to yaml
chan-hoo Apr 20, 2023
c53cb9e
set incr_cycl_freq to 2 digits
chan-hoo Apr 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion etc/lmod-setup.csh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ else
set L_MACHINE=$1
endif

source /etc/csh.login
if ( "$L_MACHINE" != wcoss2 ) then
source /etc/csh.login
endif

if ( "$L_MACHINE" == macos ) then
arch=$(uname -m)
Expand Down Expand Up @@ -53,6 +55,9 @@ else if ( "$L_MACHINE" == odin ) then
module --initial_load --no_redirect restore
setenv MODULEPATH "/oldscratch/ywang/external/hpc-stack/modulefiles/mpi/intel/2020/cray-mpich/7.7.16:/oldscratch/ywang/external/hpc-stack/modulefiles/compiler/intel/2020:/oldscratch/ywang/external/hpc-stack/modulefiles/core:/oldscratch/ywang/external/hpc-stack/modulefiles/stack:/opt/cray/pe/perftools/21.02.0/modulefiles:/opt/cray/ari/modulefiles:/opt/cray/pe/craype-targets/default/modulefiles:/opt/cray/pe/modulefiles:/opt/cray/modulefiles:/opt/modulefiles"

else if ( "$L_MACHINE" = wcoss2 ) then
module reset

else
module purge
endif
Expand Down
19 changes: 12 additions & 7 deletions etc/lmod-setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ else
L_MACHINE=$1
fi

[[ ${SHELLOPTS} =~ nounset ]] && has_mu=true || has_mu=false
[[ ${SHELLOPTS} =~ errexit ]] && has_me=true || has_me=false
$has_mu && set +u
$has_me && set +e
source /etc/profile
$has_mu && set -u
$has_me && set -e
if [ "$L_MACHINE" != wcoss2 ]; then
[[ ${SHELLOPTS} =~ nounset ]] && has_mu=true || has_mu=false
[[ ${SHELLOPTS} =~ errexit ]] && has_me=true || has_me=false
$has_mu && set +u
$has_me && set +e
source /etc/profile
$has_mu && set -u
$has_me && set -e
fi
Comment thread
chan-hoo marked this conversation as resolved.

if [ "$L_MACHINE" = macos ]; then
arch=$(uname -m)
Expand Down Expand Up @@ -56,6 +58,9 @@ elif [ "$L_MACHINE" = odin ]; then
module --initial_load --no_redirect restore
export MODULEPATH="/oldscratch/ywang/external/hpc-stack/modulefiles/mpi/intel/2020/cray-mpich/7.7.16:/oldscratch/ywang/external/hpc-stack/modulefiles/compiler/intel/2020:/oldscratch/ywang/external/hpc-stack/modulefiles/core:/oldscratch/ywang/external/hpc-stack/modulefiles/stack:/opt/cray/pe/perftools/21.02.0/modulefiles:/opt/cray/ari/modulefiles:/opt/cray/pe/craype-targets/default/modulefiles:/opt/cray/pe/modulefiles:/opt/cray/modulefiles:/opt/modulefiles"

elif [ "$L_MACHINE" = wcoss2 ]; then
module reset

else
module purge
fi
11 changes: 11 additions & 0 deletions jobs/JREGIONAL_PRE_POST_STAT
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,17 @@ job_postamble
#
#-----------------------------------------------------------------------
#
# Remove post_complete flag file.
#
#-----------------------------------------------------------------------
#
post_complete_file="${COMIN}/post_${PDY}${cyc}_task_complete.txt"
if [ -f ${post_complete_file} ] ; then
rm_vrfy -f ${post_complete_file}
fi
#
#-----------------------------------------------------------------------
#
# Restore the shell options saved at the beginning of this script/func-
# tion.
#
Expand Down
14 changes: 6 additions & 8 deletions jobs/JREGIONAL_RUN_POST
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,16 +138,14 @@ Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
#-----------------------------------------------------------------------
#
if [ "${FCST_LEN_HRS}" = "-1" ]; then
for i_cdate in "${!ALL_CDATES[@]}"; do
if [ "${ALL_CDATES[$i_cdate]}" = "${PDY}${cyc}" ]; then
FCST_LEN_HRS="${FCST_LEN_CYCL_ALL[$i_cdate]}"
break
fi
done
if [ ${#FCST_LEN_CYCL[@]} -gt 1 ]; then
cyc_mod=$(( ${cyc} - ${DATE_FIRST_CYCL:8:2} ))
CYCLE_IDX=$(( ${cyc_mod} / ${INCR_CYCL_FREQ} ))
FCST_LEN_HRS=${FCST_LEN_CYCL[$CYCLE_IDX]}

@christinaholtNOAA christinaholtNOAA Apr 13, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this logic works just right.

If DATE_FIRST_CYCL starts at 18Z, and we're looking at the next 00Z cycle, we get:

cyc_mod=$(( 00 - 18 ))
CYCLE_IDX=$(( -18 / 6 ))

So we're left with a negative CYCLE_IDX and can't get the right information in the list. I'd suggest that we add the requirement on FCST_LEN_CYCL to define a list that starts from 00Z and increments by the FCST_LEN_CYCL if you need to define variable forecasts lengths.

@chan-hoo chan-hoo Apr 16, 2023 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christinaholtNOAA, I don't agree with you. {cyc} starts from a lower value. if {cyc} includes "06" and "18", its order will be ["06", "18"] (not ["18", "06"]). I think the 'cyc_mod' will not have a negative value in any cases because these variable forecast length hours are only set per day as you modified it in your former PR:

ufs-srweather-app/ush/setup.py

Line 681 in 662ff2a

# Check that the number of entries divides into a day

What do you think about this? In the above example of ["06", "18"], the current condition will fail because their indexes will be [ "06/12", "18/12"]. With a new condition, they will be ["0", "1"] (["(6-6)/12", "(18-6)/12"]).

@christinaholtNOAA christinaholtNOAA Apr 19, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still concerned about the use case given that there would be a strong coupling between the FCST_LEN_CYCL and the DATE_FIRST_CYCL and DATE_LAST_CYCL variables.

In the above example, I was suggesting that these might be the user-defined settings for two cycles, where the first is meant to be a short forecast and the second a long one:

DATE_FIRST_CYCL=2023030518
DATE_LAST_CYCL=2023030600
FCST_LEN_CYCL=["06", "18"]

This is a specific example of where I'm saying it might be better to require users to define FCST_LEN_CYCL as if it were all the possible daily cycles like this:

FCST_LEN_CYCL=["18", "06",  "06", "06"]

This means that we don't have to handle the "special" cases and aren't so heavily coupling the user-defined dates to the forecast lengths.

@chan-hoo chan-hoo Apr 20, 2023 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christinaholtNOAA, I understand your concern. However, the change in this PR is better than the current one you made because it at least works for the special case. The current version will not work for both cases (my case as well as your case). In my opinion, the best solution is to use my original version:

  for i_cdate in "${!ALL_CDATES[@]}"; do
    if [ "${ALL_CDATES[$i_cdate]}" = "${PDY}${cyc}" ]; then
      FCST_LEN_HRS="${FCST_LEN_CYCL_ALL[$i_cdate]}"
      break
    fi

I understand that you didn't want to use ALL_CDATES, but this one will not have any issues. What do you think about that?

@christinaholtNOAA christinaholtNOAA Apr 21, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaning on ALL_CDATES is not a valid solution for when we want to run a real-time run indefinitely. Do you mind sharing the use case that you are trying to get running? The start and end date, and the forecast lengths you'd like to support?

@chan-hoo chan-hoo Apr 23, 2023

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christinaholtNOAA:

  1. Official NRT (Near Real Time) run: 4 cycles per day ("00" "06" "12" "18") and varying forecast length hours = ("06" "72" "72" "06").
  2. Official Retro run: same 4 cycles per day and lengths as 1); 3 month period
  3. Non-official test run: 2 cycles per day ("06" "18") for 3 days ("INCR_CYCL_FREQ: 12). This case causes the error in the current status. We don't support this case officially. However, I remember someone asked me if this case was available in the workflow. This is the reason why I am trying to update this part in my PR.

@chan-hoo chan-hoo Apr 25, 2023

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christinaholtNOAA, if you don't agree with my change, please provide us a reasonable solution. This PR takes so long for review now. The AQM users are not able to use the develop branch now and they are waiting for this PR to be merged. @MichaelLueken, could you please ask other reviewers to review this PR?


fcst_len_hrs=$( printf "%03d" "${FCST_LEN_HRS}" )
if [ "${fhr}" = "${fcst_len_hrs}" ]; then
touch "${COMIN}/${TN_RUN_POST}_${PDY}${cyc}_task_complete.txt"
touch "${COMIN}/post_${PDY}${cyc}_task_complete.txt"
fi
fi
#
Expand Down
37 changes: 16 additions & 21 deletions parm/wflow/aqm_all.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ task_nexus_gfs_sfc:
datadep_gfs:
attrs:
age: 00:00:00:05
text: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}">&COMINgfs;/gfs.@Y@m@d/@H/atmos</cyclestr>'
text: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}:00:00">&COMINgfs;/gfs.@Y@m@d/@H/atmos</cyclestr>'
streq:
left: retro
right: '{% if not workflow.DO_REAL_TIME %}retro{% endif %}'
Expand Down Expand Up @@ -117,12 +117,12 @@ task_aqm_ics_ext:
task_aqm_ics:
<<: *default_aqm
attrs:
cycledefs: cycled
cycledefs: cycled_from_second
maxtries: '2'
command: '&LOAD_MODULES_RUN_TASK_FP; "aqm_ics" "&JOBSdir;/JREGIONAL_AQM_ICS"'
envars:
<<: *default_vars
PREV_CYCLE_DIR: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}">&COMIN_DIR;</cyclestr>'
PREV_CYCLE_DIR: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}:00:00">&COMIN_DIR;</cyclestr>'
join: !cycstr '&LOGDIR;/{{ jobname }}_@Y@m@d@H&LOGEXT;'
dependency:
and:
Expand All @@ -133,11 +133,11 @@ task_aqm_ics:
datadep_date_tag_tracer:
attrs:
age: 00:00:00:05
text: '<cyclestr offset="-{{- workflow.INCR_CYCL_FREQ -}}">&COMIN_DIR;</cyclestr>/RESTART/<cyclestr>@Y@m@d.@H@M@S.fv_tracer.res.tile1.nc</cyclestr>'
text: '<cyclestr offset="-{{- workflow.INCR_CYCL_FREQ -}}:00:00">&COMIN_DIR;</cyclestr>/RESTART/<cyclestr>@Y@m@d.@H@M@S.fv_tracer.res.tile1.nc</cyclestr>'
datadep_tracer:
attrs:
age: 00:00:00:05
text: <cyclestr offset="-{{- workflow.INCR_CYCL_FREQ -}}">&COMIN_DIR;</cyclestr>/RESTART/fv_tracer.res.tile1.nc
text: <cyclestr offset="-{{- workflow.INCR_CYCL_FREQ -}}:00:00">&COMIN_DIR;</cyclestr>/RESTART/fv_tracer.res.tile1.nc

task_aqm_lbcs:
<<: *default_aqm
Expand All @@ -154,22 +154,14 @@ task_pre_post_stat:
command: '&LOAD_MODULES_RUN_TASK_FP; "pre_post_stat" "&JOBSdir;/JREGIONAL_PRE_POST_STAT"'
join: !cycstr '&LOGDIR;/{{ jobname }}_@Y@m@d@H&LOGEXT;'
dependency:
taskdep:
attrs:
task: run_fcst_mem000
#or_do_post:
# and_inline_post: # If inline post ran, wait on the forecast task to complete
# not:
# taskvalid:
# attrs:
# task: run_post_mem000_f000
# and_run_post: # If post was meant to run, wait on the whole post metatask
# taskvalid:
# attrs:
# task: run_post_mem000_f000
# metataskdep:
# attrs:
# metatask: run_post_mem000
or:
datadep:
attrs:
age: 00:00:00:05
text: !cycstr '&COMIN_DIR;/post_@Y@m@d@H_task_complete.txt'
metataskdep:
attrs:
metatask: run_ens_post

task_post_stat_o3:
<<: *default_aqm
Expand All @@ -187,6 +179,7 @@ task_post_stat_pm25:
join: !cycstr '&LOGDIR;/{{ jobname }}_@Y@m@d@H&LOGEXT;'
memory: 120G
dependency:
taskdep:
attrs:
task: pre_post_stat

Expand All @@ -196,6 +189,7 @@ task_bias_correction_o3:
join: !cycstr '&LOGDIR;/{{ jobname }}_@Y@m@d@H&LOGEXT;'
memory: 120G
dependency:
taskdep:
attrs:
task: pre_post_stat

Expand All @@ -205,6 +199,7 @@ task_bias_correction_pm25:
join: !cycstr '&LOGDIR;/{{ jobname }}_@Y@m@d@H&LOGEXT;'
memory: 120G
dependency:
taskdep:
attrs:
task: pre_post_stat

4 changes: 2 additions & 2 deletions parm/wflow/coldstart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ task_get_extrn_ics:
datadep_gfs:
attrs:
age: 00:00:00:05
text: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}">&COMINgfs;/gfs.@Y@m@d/@H/atmos</cyclestr>'
text: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}:00:00">&COMINgfs;/gfs.@Y@m@d/@H/atmos</cyclestr>'
streq:
left: retro
right: '{% if not workflow.DO_REAL_TIME %}retro{% endif %}'
Expand All @@ -68,7 +68,7 @@ task_get_extrn_lbcs:
datadep_gfs:
attrs:
age: 00:00:00:05
text: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}">&COMINgfs;/gfs.@Y@m@d/@H/atmos</cyclestr>'
text: '<cyclestr offset="-{{ workflow.INCR_CYCL_FREQ }}:00:00">&COMINgfs;/gfs.@Y@m@d/@H/atmos</cyclestr>'
streq:
left: retro
right: '{% if not workflow.DO_REAL_TIME %}retro{% endif %}'
Expand Down
8 changes: 5 additions & 3 deletions parm/wflow/default_workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ rocoto:
entities:
ACCOUNT: '{{ user.ACCOUNT }}'
CCPA_OBS_DIR: '{{ platform.CCPA_OBS_DIR }}'
COMIN_DIR: '{% if user.RUN_ENVIR == "nco" %}{{ nco.COMIN_BASEDIR }}/{{ nco.RUN }}.@Y@m@d/@H {% else %}{{ nco.COMIN_BASEDIR }}/@Y@m@d@H{% endif %}'
COMIN_DIR: '{% if user.RUN_ENVIR == "nco" %}{{"{}/{}.@Y@m@d/@H".format(nco.COMIN_BASEDIR,nco.RUN)}}{% else %}{{"{}/@Y@m@d@H".format(workflow.EXPTDIR)}}{% endif %}'
COMINgfs: '{{ platform.get("COMINgfs") }}'
FCST_DIR: '{{ nco.DATAROOT }}/run_fcst.{{ workflow.WORKFLOW_ID }}_@Y@m@d@H'
FCST_DIR: '{% if user.RUN_ENVIR == "nco" %}{{"{}/run_fcst_mem#mem#.{}_@Y@m@d@H".format(nco.DATAROOT,workflow.WORKFLOW_ID)}}{% else %}{{"{}/@Y@m@d@H".format(workflow.EXPTDIR)}}{% endif %}'
GLOBAL_VAR_DEFNS_FP: '{{ workflow.GLOBAL_VAR_DEFNS_FP }}'
JOBSdir: '{{ user.JOBSdir }}'
LOAD_MODULES_RUN_TASK_FP: '{{ workflow.LOAD_MODULES_RUN_TASK_FP }}'
Expand All @@ -31,7 +31,7 @@ rocoto:
WARMSTART_CYCLE_DIR: '{{ workflow.WARMSTART_CYCLE_DIR }}'
WORKFLOW_ID: '{{ workflow.WORKFLOW_ID }}'
attrs:
cyclethrottle: "20"
cyclethrottle: "200"
realtime: "F"
scheduler: '{{ platform.SCHED }}'
taskthrottle: "1000"
Expand All @@ -40,6 +40,8 @@ rocoto:
- !startstopfreq ['{{workflow.DATE_FIRST_CYCL}}', '{{workflow.DATE_FIRST_CYCL}}', '{{workflow.INCR_CYCL_FREQ}}']
forecast:
- !startstopfreq ['{{workflow.DATE_FIRST_CYCL}}', '{{workflow.DATE_LAST_CYCL}}', '{{workflow.INCR_CYCL_FREQ}}']
cycled_from_second:
- !startstopfreq ['{%- if workflow.DATE_FIRST_CYCL != workflow.DATE_LAST_CYCL %}{{ [workflow.DATE_FIRST_CYCL[0:8], "{:02d}".format(workflow.INCR_CYCL_FREQ)]|join }}{%- else %}{{workflow.DATE_FIRST_CYCL}}{%- endif %}', '{{workflow.DATE_LAST_CYCL}}', '{{workflow.INCR_CYCL_FREQ}}']
log: !cycstr '&LOGDIR;/FV3LAM_wflow.{% if user.RUN_ENVIR == "nco" %}{{ workflow.WORKFLOW_ID + "." }}{% endif %}log'
tasks:
taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml"]|include }}'
41 changes: 23 additions & 18 deletions scripts/exregional_aqm_lbcs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@ else
print_info_msg "$VERBOSE" "
All executables will be submitted with command \'${RUN_CMD_AQMLBC}\'."
fi

#
#-----------------------------------------------------------------------
#
Expand All @@ -88,12 +87,14 @@ cd_vrfy $DATA
#
#-----------------------------------------------------------------------
#
yyyymmdd="${PDY}"
mm="${PDY:4:2}"

CDATE_MOD=$( $DATE_UTIL --utc --date "${PDY} ${cyc} UTC - ${EXTRN_MDL_LBCS_OFFSET_HRS} hours" "+%Y%m%d%H" )
yyyymmdd=${CDATE_MOD:0:8}
mm="${CDATE_MOD:4:2}"
hh="${CDATE_MOD:8:2}"

if [ "${FCST_LEN_HRS}" = "-1" ]; then
CYCLE_IDX=$(( ${cyc} / ${INCR_CYCL_FREQ} ))
if [ ${#FCST_LEN_CYCL[@]} -gt 1 ]; then
cyc_mod=$(( ${cyc} - ${DATE_FIRST_CYCL:8:2} ))
CYCLE_IDX=$(( ${cyc_mod} / ${INCR_CYCL_FREQ} ))
FCST_LEN_HRS=${FCST_LEN_CYCL[$CYCLE_IDX]}
fi
LBC_SPEC_FCST_HRS=()
Expand All @@ -119,7 +120,7 @@ The chemical LBC files do not exist:
for hr in 0 ${LBC_SPEC_FCST_HRS[@]}; do
fhr=$( printf "%03d" "${hr}" )
if [ -r ${INPUT_DATA}/${NET}.${cycle}${dot_ensmem}.gfs_bndy.tile7.f${fhr}.nc ]; then
ncks -A ${chem_lbcs_fn} ${INPUT_DATA}/${NET}.${cycle}${dot_ensmem}.gfs_bndy.tile7.f${fhr}.nc
ncks -A ${chem_lbcs_fn} ${INPUT_DATA}/${NET}.${cycle}${dot_ensmem}.gfs_bndy.tile7.f${fhr}.nc
fi
done

Expand All @@ -136,36 +137,40 @@ fi
#-----------------------------------------------------------------------
#
if [ ${DO_AQM_GEFS_LBCS} = "TRUE" ]; then

RUN_CYC="${cyc}"
CDATE_MOD=$( $DATE_UTIL --utc --date "${PDY} ${cyc} UTC - ${EXTRN_MDL_LBCS_OFFSET_HRS} hours" "+%Y%m%d%H" )
PDY_MOD=${CDATE_MOD:0:8}
AQM_GEFS_FILE_CYC=${AQM_GEFS_FILE_CYC:-"${CDATE_MOD:8:2}"}

AQM_GEFS_FILE_CYC=${AQM_GEFS_FILE_CYC:-"${hh}"}
AQM_GEFS_FILE_CYC=$( printf "%02d" "${AQM_GEFS_FILE_CYC}" )

GEFS_CYC_DIFF=$(( cyc - AQM_GEFS_FILE_CYC ))
if [ "${GEFS_CYC_DIFF}" -lt "0" ]; then
TSTEPDIFF=$( printf "%02d" $(( 24 + ${GEFS_CYC_DIFF} )) )
else
TSTEPDIFF=$( printf "%02d" ${GEFS_CYC_DIFF} )
fi

AQM_MOFILE_FN="${AQM_GEFS_FILE_PREFIX}.t${AQM_GEFS_FILE_CYC}z.atmf"
if [ ${DO_REAL_TIME} = "TRUE" ]; then
AQM_MOFILE_FP="${COMINgefs}/gefs.${PDY_MOD}/${AQM_GEFS_FILE_CYC}/chem/sfcsig/${AQM_MOFILE_FN}"
if [ "${DO_REAL_TIME}" = "TRUE" ]; then
AQM_MOFILE_FP="${COMINgefs}/gefs.${yyyymmdd}/${AQM_GEFS_FILE_CYC}/chem/sfcsig/${AQM_MOFILE_FN}"
else
AQM_MOFILE_FP="${AQM_GEFS_DIR}/${PDY}/${AQM_GEFS_FILE_CYC}/${AQM_MOFILE_FN}"
AQM_MOFILE_FP="${AQM_GEFS_DIR}/${yyyymmdd}/${AQM_GEFS_FILE_CYC}/${AQM_MOFILE_FN}"
fi

# Check if GEFS aerosol files exist
for hr in 0 ${LBC_SPEC_FCST_HRS[@]}; do
fhr=$( printf "%03d" "${hr}" )
hr_mod=$(( hr + EXTRN_MDL_LBCS_OFFSET_HRS ))
fhr=$( printf "%03d" "${hr_mod}" )
AQM_MOFILE_FHR_FP="${AQM_MOFILE_FP}${fhr}.nemsio"
if [ ! -e "${AQM_MOFILE_FHR_FP}" ]; then
print_err_msg_exit "The GEFS file (AQM_MOFILE_FHR_FP) for LBCs does not exist:
AQM_MOFILE_FHR_FP = \"${AQM_MOFILE_FHR_FP}\""
fi
done

GEFS_CYC_DIFF=$( printf "%02d" "$(( RUN_CYC - AQM_GEFS_FILE_CYC ))" )
NUMTS="$(( FCST_LEN_HRS / LBC_SPEC_INTVL_HRS + 1 ))"

cat > gefs2lbc-nemsio.ini <<EOF
&control
tstepdiff=${GEFS_CYC_DIFF}
tstepdiff=${TSTEPDIFF}
dtstep=${LBC_SPEC_INTVL_HRS}
bndname='aothrj','aecj','aorgcj','asoil','numacc','numcor'
mofile='${AQM_MOFILE_FP}','.nemsio'
Expand Down
5 changes: 3 additions & 2 deletions scripts/exregional_bias_correction_o3.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,9 @@ if [ "${PREDEF_GRID_NAME}" = "AQM_NA_13km" ]; then
id_domain=793
fi

if [ "${FCST_LEN_HRS}" = "-1" ]; then
CYCLE_IDX=$(( ${cyc} / ${INCR_CYCL_FREQ} ))
if [ ${#FCST_LEN_CYCL[@]} -gt 1 ]; then
cyc_mod=$(( ${cyc} - ${DATE_FIRST_CYCL:8:2} ))
CYCLE_IDX=$(( ${cyc_mod} / ${INCR_CYCL_FREQ} ))
FCST_LEN_HRS=${FCST_LEN_CYCL[$CYCLE_IDX]}
fi

Expand Down
7 changes: 4 additions & 3 deletions scripts/exregional_bias_correction_pm25.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,13 @@ yyyymm_m3=${PDYm3:0:6}
if [ "${PREDEF_GRID_NAME}" = "AQM_NA_13km" ]; then
id_domain=793
fi
if [ "${FCST_LEN_HRS}" = "-1" ]; then
CYCLE_IDX=$(( ${cyc} / ${INCR_CYCL_FREQ} ))

if [ ${#FCST_LEN_CYCL[@]} -gt 1 ]; then
cyc_mod=$(( ${cyc} - ${DATE_FIRST_CYCL:8:2} ))
CYCLE_IDX=$(( ${cyc_mod} / ${INCR_CYCL_FREQ} ))
FCST_LEN_HRS=${FCST_LEN_CYCL[$CYCLE_IDX]}
fi


#-----------------------------------------------------------------------------
# STEP 1: Retrieve AIRNOW observation data
#-----------------------------------------------------------------------------
Expand Down
6 changes: 4 additions & 2 deletions scripts/exregional_get_extrn_mdl_files.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,10 @@ if [ "${ICS_OR_LBCS}" = "ICS" ]; then
elif [ "${ICS_OR_LBCS}" = "LBCS" ]; then
file_set="fcst"
first_time=$((TIME_OFFSET_HRS + LBC_SPEC_INTVL_HRS))
if [ "${FCST_LEN_HRS}" = "-1" ]; then
CYCLE_IDX=$(( ${cyc} / ${INCR_CYCL_FREQ} ))

if [ ${#FCST_LEN_CYCL[@]} -gt 1 ]; then
cyc_mod=$(( ${cyc} - ${DATE_FIRST_CYCL:8:2} ))
CYCLE_IDX=$(( ${cyc_mod} / ${INCR_CYCL_FREQ} ))
FCST_LEN_HRS=${FCST_LEN_CYCL[$CYCLE_IDX]}
fi
last_time=$((TIME_OFFSET_HRS + FCST_LEN_HRS))
Expand Down
5 changes: 3 additions & 2 deletions scripts/exregional_make_lbcs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,9 @@ DATA="${DATA}/tmp_LBCS"
mkdir_vrfy -p "$DATA"
cd_vrfy $DATA

if [ "${FCST_LEN_HRS}" = "-1" ]; then
CYCLE_IDX=$(( ${cyc} / ${INCR_CYCL_FREQ} ))
if [ ${#FCST_LEN_CYCL[@]} -gt 1 ]; then
cyc_mod=$(( ${cyc} - ${DATE_FIRST_CYCL:8:2} ))
CYCLE_IDX=$(( ${cyc_mod} / ${INCR_CYCL_FREQ} ))
FCST_LEN_HRS=${FCST_LEN_CYCL[$CYCLE_IDX]}
fi
LBC_SPEC_FCST_HRS=()
Expand Down
5 changes: 3 additions & 2 deletions scripts/exregional_nexus_emission.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,9 @@ yyyymmdd="${PDY}"

NUM_SPLIT_NEXUS=$( printf "%02d" ${NUM_SPLIT_NEXUS} )

if [ "${FCST_LEN_HRS}" = "-1" ]; then
CYCLE_IDX=$(( ${cyc} / ${INCR_CYCL_FREQ} ))
if [ ${#FCST_LEN_CYCL[@]} -gt 1 ]; then
cyc_mod=$(( ${cyc} - ${DATE_FIRST_CYCL:8:2} ))
CYCLE_IDX=$(( ${cyc_mod} / ${INCR_CYCL_FREQ} ))
FCST_LEN_HRS=${FCST_LEN_CYCL[$CYCLE_IDX]}
fi

Expand Down
Loading