Skip to content

Fix for EnKF cycling #251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chunhuazhou merged 1 commit into from
Nov 17, 2021
Merged

Fix for EnKF cycling #251

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions jobs/JREGIONAL_RUN_PREPSTART_ENSMEAN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,7 @@ mkdir_vrfy -p ${modelinputdir}
#
$SCRIPTSDIR/exregional_run_prepstart_ensmean.sh \
cycle_type="${CYCLE_TYPE}" \
modelinputdir="${modelinputdir}" \
fg_root="${FG_ROOT}" || \
modelinputdir="${modelinputdir}" || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
26 changes: 12 additions & 14 deletions scripts/exregional_run_prepstart_ensmean.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ specified cycle.
#
#-----------------------------------------------------------------------
#
valid_args=( "cycle_type" "modelinputdir" "fg_root")
valid_args=( "cycle_type" "modelinputdir" )
process_args valid_args "$@"
#
#-----------------------------------------------------------------------
Expand Down Expand Up @@ -123,27 +123,25 @@ while [[ $imem -le ${NUM_ENS_MEMBERS} ]];
#
fg_restart_dirname=fcst_fv3lam

YYYYMMDDHHmInterv=$( date +%Y%m%d%H -d "${START_DATE} ${DA_CYCLE_INTERV} hours ago" )
bkpath=${fg_root}/${YYYYMMDDHHmInterv}/mem${ensmem}/${fg_restart_dirname}/RESTART # cycling, use background from RESTART
bkpath=${CYCLE_DIR}/mem${ensmem}/${fg_restart_dirname}/INPUT # cycling, use background from RESTART

# the restart file from FV3 has a name like: ${YYYYMMDD}.${HH}0000.fv_core.res.tile1.nc
# But the restart files for the forecast length has a name like: fv_core.res.tile1.nc
# So the defination of restart_prefix needs a "." at the end.
#
restart_prefix="${YYYYMMDD}.${HH}0000."

checkfile=${bkpath}/${restart_prefix}fv_core.res.tile1.nc
checkfile1=${bkpath}/${restart_prefix}fv_tracer.res.tile1.nc
checkfile=${bkpath}/fv_core.res.tile1.nc
checkfile1=${bkpath}/fv_tracer.res.tile1.nc
if [ -r "${checkfile}" ] && [ -r "${checkfile1}" ] ; then
ln_vrfy ${bkpath}/${restart_prefix}fv_core.res.tile1.nc fv_core.res.tile1.nc_mem${ensmem}
ln_vrfy ${bkpath}/${restart_prefix}fv_tracer.res.tile1.nc fv_tracer.res.tile1.nc_mem${ensmem}
ln_vrfy ${bkpath}/${restart_prefix}sfc_data.nc sfc_data.nc_mem${ensmem}
ln_vrfy ${bkpath}/${restart_prefix}fv_srf_wnd.res.tile1.nc fv_srf_wnd.res.tile1.nc_mem${ensmem}
ln_vrfy ${bkpath}/${restart_prefix}phy_data.nc phy_data.nc_mem${ensmem}
ln_vrfy ${bkpath}/fv_core.res.tile1.nc fv_core.res.tile1.nc_mem${ensmem}
ln_vrfy ${bkpath}/fv_tracer.res.tile1.nc fv_tracer.res.tile1.nc_mem${ensmem}
ln_vrfy ${bkpath}/sfc_data.nc sfc_data.nc_mem${ensmem}
ln_vrfy ${bkpath}/fv_srf_wnd.res.tile1.nc fv_srf_wnd.res.tile1.nc_mem${ensmem}
ln_vrfy ${bkpath}/phy_data.nc phy_data.nc_mem${ensmem}
if [ $imem == 1 ]; then
ln_vrfy ${bkpath}/${restart_prefix}coupler.res coupler.res
ln_vrfy ${bkpath}/${restart_prefix}fv_core.res.nc fv_core.res.nc
ln_vrfy ${fg_root}/${YYYYMMDDHHmInterv}/mem${ensmem}/${fg_restart_dirname}/INPUT/gfs_ctrl.nc gfs_ctrl.nc
ln_vrfy ${bkpath}/coupler.res coupler.res
ln_vrfy ${bkpath}/fv_core.res.nc fv_core.res.nc
ln_vrfy ${bkpath}/gfs_ctrl.nc gfs_ctrl.nc
fi
else
print_err_msg_exit "Error: cannot find background: ${checkfile}"
Expand Down
2 changes: 1 addition & 1 deletion ush/load_modules_run_task.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ workflow task specified by task_name failed:
task_name = \"${task_name}\"
env_fp = \"${env_fp}\""

if [ ${task_name} = "run_anal_gsi" ] || [ ${task_name} = "run_enkfupdt" || [ ${task_name} = "run_prepstart" ]; then
if [ ${task_name} = "run_anal_gsi" ] || [ ${task_name} = "run_enkfupdt" ] || [ ${task_name} = "run_prepstart" ]; then
source "${env_fp}_DA" || print_err_msg_exit "\
Sourcing platform- and compiler-specific environment file (env_fp) for the
workflow task specified by task_name failed:
Expand Down
20 changes: 11 additions & 9 deletions ush/templates/FV3LAM_wflow.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -724,7 +724,11 @@ MODULES_RUN_TASK_FP script.
{%- if do_retro %}
<and>
<datadep age="00:00:05:00"><cyclestr>&FG_ROOT;/@Y@m@d@H{{ slash_ensmem_subdir }}/ics/gfs_data.tile7.halo0.nc</cyclestr></datadep>
<datadep age="00:00:05:00"><cyclestr offset="-{{ extrn_mdl_ics_offset_hrs }}:00:00">&FG_ROOT;/@Y@m@d@H{{ slash_ensmem_subdir }}/lbcs/gfs_bndy.tile7.{{ "%03d" % boundary_len_hrs }}.nc</cyclestr></datadep>
<or>
Copy link
Copy Markdown
Collaborator Author

@chunhuazhou chunhuazhou Nov 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hu5970 Is this the section that you are concerned about? The reason I changed it is that when I initialize the cycle at 00Z and have the ics and lbcs tasks done for 00Z.But because I am having extrn_mdl_ics_offset_hrs=6 for GDASENKF, it can't find lbcs from 18Z and therefore won't start prep_cyc. By changing it to look for lbcs in the past few hours, it will kick off the task since I have lbcs at 00Z.

{%- for h in range(0, extrn_mdl_ics_offset_hrs+1) %}
<datadep age="00:00:05:00"><cyclestr offset="-{{ h }}:00:00">&FG_ROOT;/@Y@m@d@H{{ slash_ensmem_subdir }}/lbcs/gfs_bndy.tile7.{{ "%03d" % boundary_len_hrs }}.nc</cyclestr></datadep>
{%- endfor %}
</or>
Copy link
Copy Markdown

@hu5970 hu5970 Nov 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why we need this extension of checking for boundary files in retro?

Copy link
Copy Markdown
Collaborator Author

@chunhuazhou chunhuazhou Nov 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I wasn't so sure why it was set to find the lbcs from extrn_mdl_ics_offset_hrs ago. So to keep its original setting and expand it for checking lbcs for the start-up cycle, I extended this check for the time window (- extrn_mdl_ics_offset_hrs, 0). May we can talk more about this tomorrow at the 10am meeting.

</and>
{% else %}
<and>
Expand Down Expand Up @@ -1312,23 +1316,21 @@ MODULES_RUN_TASK_FP script.
<envar><name>PDY</name><value><cyclestr>@Y@m@d</cyclestr></value></envar>
<envar><name>CDATE</name><value><cyclestr>@Y@m@d@H</cyclestr></value></envar>
<envar><name>CYCLE_DIR</name><value><cyclestr>&CYCLE_BASEDIR;/@Y@m@d@H</cyclestr></value></envar>
<!--
<envar><name>FG_ROOT</name><value><cyclestr>&FG_ROOT;</cyclestr></value></envar>
<envar><name>LBCS_ROOT</name><value><cyclestr>&FG_ROOT;</cyclestr></value></envar>
-->
<envar><name>CYCLE_TYPE</name><value><cyclestr>prod</cyclestr></value></envar>
<envar><name>NUM_ENS_MEMBERS</name><value><cyclestr>#{{ num_ens_members }}#</cyclestr></value></envar>

<dependency>
<and>
{%- if do_retro %}
{%- for m in range(1, num_ens_members+1) %}
<datadep age="00:00:01:00"><cyclestr offset="-{{ da_cycle_interval_hrs }}:00:00">&FG_ROOT;/@Y@m@d@H/mem{{ "%04d" % m }}/fcst_fv3lam/RESTART/</cyclestr><cyclestr>@Y@m@d.@H0000.coupler.res</cyclestr></datadep>
{%- for h in cycl_hrs_prodstart %}
<strneq><left>{{ h }}</left><right><cyclestr>@H</cyclestr></right></strneq>
{%- endfor %}
{%- else %}
<timedep><cyclestr offset="&START_TIME_CONVENTIONAL;">@Y@m@d@H@M00</cyclestr></timedep>
{%- for m in range(1, num_ens_members+1) %}
<datadep age="00:00:01:00"><cyclestr offset="-{{ da_cycle_interval_hrs }}:00:00">&FG_ROOT;/@Y@m@d@H/mem{{ "%04d" % m }}/fcst_fv3lam/RESTART/</cyclestr><cyclestr>@Y@m@d.@H0000.coupler.res</cyclestr></datadep>
<taskdep task="&PREP_CYC_PROD_TN;_mem{{ "%04d" % m }}"/>
{%- endfor %}
{%- endif %}
</and>
</dependency>

Expand Down Expand Up @@ -1367,7 +1369,7 @@ MODULES_RUN_TASK_FP script.
<dependency>
<and>
<taskdep task="&PREP_CYC_ENSMEAN_TN;"/>
<datadep age="05:00"><cyclestr offset="{{ da_cycle_interval_hrs }}:00:00">&OBSPATH;/@Y@m@d@H.rap.t@Hz.prepbufr.tm00</cyclestr></datadep>
<datadep age="05:00"><cyclestr>&OBSPATH;/@Y@m@d@H.rap.t@Hz.prepbufr.tm00</cyclestr></datadep>
</and>
</dependency>

Expand Down