Skip to content

Prepare observations for the updates to the ensemble members. #2416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jiaruidong2017 wants to merge 13 commits into from
Closed

Prepare observations for the updates to the ensemble members. #2416

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
1745619
Add the initial configure to prepare obs for the ensemble members.
jiaruidong2017 Mar 18, 2024
5fef5ad
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Mar 23, 2024
b63a0e3
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Mar 28, 2024
fa05e5a
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Mar 30, 2024
146e861
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Apr 2, 2024
8ef5104
Change COM_OBS_IN to COMIN_OBS and change COM_OBS_OUT to COMOUT_OBS.
jiaruidong2017 Apr 6, 2024
6cc76e4
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Apr 8, 2024
a0f6ed2
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Apr 10, 2024
15bbd45
Change generate_com to declare_from_tmpl.
jiaruidong2017 Apr 10, 2024
009767b
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Apr 13, 2024
fc7cc84
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Apr 16, 2024
2557cb7
Update the GDASApp commit hash.
jiaruidong2017 Apr 16, 2024
8b986b0
Merge branch 'develop' into feature/prepsnowobs_ens
jiaruidong2017 Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions env/HERA.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanlrun aeroanlrun snowanl"
echo "atmanlrun atmensanlrun aeroanlrun prepsnowobs snowanl"
echo "anal sfcanl fcst post metp"
echo "eobs eupd ecen efcs epos"
echo "eobs eupd ecen efcs epos epsn"
echo "postsnd awips gempak"
exit 1

Expand Down Expand Up @@ -42,7 +42,7 @@ if [[ "${step}" = "prep" ]] || [[ "${step}" = "prepbufr" ]]; then
export sys_tp="HERA"
export launcher_PREP="srun"

elif [[ "${step}" = "prepsnowobs" ]]; then
elif [[ "${step}" = "prepsnowobs" ]] || [[ "${step}" = "epsn" ]]; then

export APRUN_CALCFIMS="${launcher} -n 1"

Expand Down
30 changes: 25 additions & 5 deletions jobs/JGLOBAL_PREP_SNOW_OBS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,20 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"
export DATA=${DATA:-${DATAROOT}/${RUN}snowanl_${cyc}}
source "${HOMEgfs}/ush/jjob_header.sh" -e "prepsnowobs" -c "base prepsnowobs"
if (( 10#${ENSMEM:-0} > 0 )); then
source "${HOMEgfs}/ush/jjob_header.sh" -e "epsn" -c "base prepsnowobs epsn"
export CASE=${CASE_ENS}
else
source "${HOMEgfs}/ush/jjob_header.sh" -e "prepsnowobs" -c "base prepsnowobs"
fi

# Restart conditions for GFS cycle come from GDAS

@CoryMartin-NOAA CoryMartin-NOAA Apr 3, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about enkfgfs?

@jiaruidong2017 jiaruidong2017 Apr 3, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea about enkfgfs.

rCDUMP=${RUN}
[[ ${RUN} == "gfs" ]] && export rCDUMP="gdas"

# Get OBS for ENKFGDAS run from GDAS
oCDUMP=${RUN}

@CoryMartin-NOAA CoryMartin-NOAA Apr 3, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we don't export oCDUMP if the run if gdas?

@jiaruidong2017 jiaruidong2017 Apr 3, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need export oCDUMP here? Both rCDUMP and oCDUMP are used just locally.

@CoryMartin-NOAA CoryMartin-NOAA Apr 3, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry I misread this line, but if they are only used internally then why export rCDUMP above on line 13?

@jiaruidong2017 jiaruidong2017 Apr 3, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reused this by following the old version jobs/JGLOBAL_FORECAST for the GFS cycle to get the restart conditions. But I just checked and found they were modified as below:

# Restart conditions for GFS cycle come from GDAS
rCDUMP=${RUN}
export rCDUMP="${RUN/gfs/gdas}"

Do I need to make the corresponding changes?

[[ ${RUN} == "enkfgdas" ]] && export oCDUMP="gdas"

##############################################
# Set variables used in the script
Expand All @@ -12,17 +24,19 @@ source "${HOMEgfs}/ush/jjob_header.sh" -e "prepsnowobs" -c "base prepsnowobs"
GDATE=$(date --utc +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
gPDY=${GDATE:0:8}
gcyc=${GDATE:8:2}
GDUMP="gdas"

##############################################
# Begin JOB SPECIFIC work
##############################################
# Generate COM variables from templates
YMD=${PDY} HH=${cyc} generate_com -rx COM_OBS
RUN=${oCDUMP} MEMDIR="" YMD=${PDY} HH=${cyc} generate_com -rx COM_OBS_IN:COM_OBS_TMPL

@CoryMartin-NOAA CoryMartin-NOAA Apr 1, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul @WalterKolczynski-NOAA are you both okay with these additional variables? Something about this I don't particularly like but I don't have a good suggestion right now either.

@WalterKolczynski-NOAA WalterKolczynski-NOAA Apr 3, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple things I don't like here, the main one being this will not be acceptable to operations where COM_OBS_TMPL points to the obsproc space and should be strictly input for GFS. The out needs to use a different template (probably a new COM_SNOW_PREP_TMPL).

Also, if IN and OUT are going to be used in the variable name, would prefer to have it be COMIN_* and COMOUT_*. That is the pattern we plan on using when we add IN and OUT to the rest of the workflow.

@jiaruidong2017 jiaruidong2017 Apr 6, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the good suggestions, and I have made changes from COM_OBS_IN/OUT to COMIN_OBS and COMOUT_OBS. I have no idea and wait for the suggestions on using a new COM_SNOW_PREP_TMPL or not.

RUN=${RUN} YMD=${PDY} HH=${cyc} generate_com -rx COM_OBS_OUT:COM_OBS_TMPL

RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} generate_com -rx \
RUN=${rCDUMP} YMD=${gPDY} HH=${gcyc} generate_com -rx \

@CoryMartin-NOAA CoryMartin-NOAA Apr 3, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be GDUMP?

@jiaruidong2017 jiaruidong2017 Apr 3, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rCDUMP=${RUN} is defined locally for the GFS cycle to get the restart conditions from GDAS. For the GDAS cycle, rCDUMP is the same as GDUMP.

COM_ATMOS_RESTART_PREV:COM_ATMOS_RESTART_TMPL

mkdir -m 775 -p "${COM_OBS_OUT}"

@CoryMartin-NOAA CoryMartin-NOAA Apr 3, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we know for sure COM_OBS_IN exists already?

@jiaruidong2017 jiaruidong2017 Apr 3, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is created in the gdasprep job.


###############################################################
# Run relevant script
EXSCRIPT=${GDASSNOWPREPPY:-${SCRgfs}/exglobal_prep_snow_obs.py}
Expand All @@ -41,4 +55,10 @@ if [[ -e "${pgmout}" ]] ; then
cat "${pgmout}"
fi

##########################################
# Remove the Temporary working directory
##########################################
cd "${DATAROOT}" || true
[[ ${KEEPDATA} = "NO" ]] && rm -rf "${DATA}"

exit 0
2 changes: 1 addition & 1 deletion parm/config/gfs/config.com
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ if [[ "${RUN_ENVIR:-emc}" == "nco" ]]; then
COM_OBS_TMPL=$(compath.py "${envir}/obsproc/${obsproc_ver}")'/${RUN}.${YMD}/${HH}/atmos'
COM_RTOFS_TMPL=$(compath.py "${envir}/${WAVECUR_DID}/${rtofs_ver}")
else
COM_OBS_TMPL='${ROTDIR}/${RUN}.${YMD}/${HH}/obs'
COM_OBS_TMPL='${ROTDIR}/${RUN}.${YMD}/${HH}/obs/${MEMDIR}'

@CoryMartin-NOAA CoryMartin-NOAA Apr 3, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to do this for all ob types, isn't it, is this okay @aerorahul ?

@CoryMartin-NOAA CoryMartin-NOAA Apr 22, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinging @aerorahul again. Are we okay with creating a $MEMDIR for all observations? Will this work as intended out of the box for the atmosphere @jiaruidong2017 ? I don't know if it is preferred to have 80 largely identical sets of directories or 1 directory for most obs and 80 directories with just snow in them?

@jiaruidong2017 jiaruidong2017 Apr 22, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want that the COM_OBS points only to the gdas directory and without memdir, we can simply add MEMDIR="" where the COM_OBS is defined in all other jobs. An example is below:

RUN=${CDUMP} MEMDIR="" YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_OBS:COM_OBS_TMPL

COM_RTOFS_TMPL='${DMPDIR}'
fi
declare -rx COM_OBS_TMPL COM_RTOFS_TMPL
Expand Down
13 changes: 13 additions & 0 deletions parm/config/gfs/config.epsn
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#! /usr/bin/env bash

########## config.epsn ##########
# Snow Obs Prep specific

echo "BEGIN: config.epsn"

source "${EXPDIR}/config.prepsnowobs"

# Get task specific resources
. "${EXPDIR}/config.resources" epsn

echo "END: config.epsn"
12 changes: 6 additions & 6 deletions parm/config/gfs/config.resources
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ if (( $# != 1 )); then
echo "upp atmos_products"
echo "tracker genesis genesis_fsu"
echo "verfozn verfrad vminmon fit2obs metp arch cleanup"
echo "eobs ediag eomg eupd ecen esfc efcs epos earc"
echo "eobs ediag eomg eupd ecen esfc efcs epos earc epsn"
echo "init_chem mom6ic oceanice_products"
echo "waveinit waveprep wavepostsbs wavepostbndpnt wavepostbndpntbll wavepostpnt"
echo "wavegempak waveawipsbulls waveawipsgridded"
Expand Down Expand Up @@ -83,11 +83,11 @@ case ${step} in
fi
;;

"prepsnowobs")
export wtime_prepsnowobs="00:05:00"
export npe_prepsnowobs=1
export nth_prepsnowobs=1
export npe_node_prepsnowobs=1
"prepsnowobs" | "epsn")
declare -x "wtime_${step}"="00:05:00"
declare -x "npe_${step}"=1
declare -x "nth_${step}"=1
declare -x "npe_node_${step}"=1
;;

"prepatmiodaobs")
Expand Down
17 changes: 9 additions & 8 deletions ush/python/pygfs/task/snow_analysis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ def __init__(self, config):
'npz': self.config.LEVS - 1,
'SNOW_WINDOW_BEGIN': _window_begin,
'SNOW_WINDOW_LENGTH': f"PT{self.config['assim_freq']}H",
'OPREFIX': f"{self.runtime_config.RUN}.t{self.runtime_config.cyc:02d}z.",
'OPREFIX_IN': f"{self.runtime_config.RUN.replace('enkf','')}.t{self.runtime_config.cyc:02d}z.",
'OPREFIX_OUT': f"{self.runtime_config.RUN}.t{self.runtime_config.cyc:02d}z.",
'APREFIX': f"{self.runtime_config.RUN}.t{self.runtime_config.cyc:02d}z.",
'jedi_yaml': _letkfoi_yaml
}
Expand Down Expand Up @@ -73,8 +74,8 @@ def prepare_GTS(self) -> None:

# create a temporary dict of all keys needed in this method
localconf = AttrDict()
keys = ['HOMEgfs', 'DATA', 'current_cycle', 'COM_OBS', 'COM_ATMOS_RESTART_PREV',
'OPREFIX', 'CASE', 'OCNRES', 'ntiles']
keys = ['HOMEgfs', 'DATA', 'current_cycle', 'COM_OBS_IN', 'COM_OBS_OUT', 'COM_ATMOS_RESTART_PREV',
'OPREFIX_IN', 'OPREFIX_OUT', 'CASE', 'OCNRES', 'ntiles']
for key in keys:
localconf[key] = self.task_config[key]

Expand Down Expand Up @@ -126,7 +127,7 @@ def _gtsbufr2iodax(exe, yaml_file):
# Ensure the IODA snow depth GTS file is produced by the IODA converter
# If so, copy to COM_OBS/
try:
FileHandler(prep_gts_config.gtsioda).sync()
FileHandler(prep_gts_config.gtsioda[self.runtime_config.RUN]).sync()
except OSError as err:
logger.exception(f"{self.task_config.BUFR2IODAX} failed to produce GTS ioda files")
raise OSError(err)
Expand All @@ -152,8 +153,8 @@ def prepare_IMS(self) -> None:

# create a temporary dict of all keys needed in this method
localconf = AttrDict()
keys = ['DATA', 'current_cycle', 'COM_OBS', 'COM_ATMOS_RESTART_PREV',
'OPREFIX', 'CASE', 'OCNRES', 'ntiles', 'FIXgfs']
keys = ['DATA', 'current_cycle', 'COM_OBS_IN', 'COM_OBS_OUT', 'COM_ATMOS_RESTART_PREV',
'OPREFIX_IN', 'OPREFIX_OUT', 'CASE', 'OCNRES', 'ntiles', 'FIXgfs']
for key in keys:
localconf[key] = self.task_config[key]

Expand Down Expand Up @@ -227,7 +228,7 @@ def prepare_IMS(self) -> None:
logger.exception(f"{self.task_config.IMS2IODACONV} failed to produce {output_file}")
raise FileNotFoundError(f"{os.path.join(localconf.DATA, output_file)}")
else:
logger.info(f"Copy {output_file} to {self.task_config.COM_OBS}")
logger.info(f"Copy {output_file} to {self.task_config.COM_OBS_OUT}")
FileHandler(prep_ims_config.ims2ioda).sync()

@logit(logger)
Expand All @@ -249,7 +250,7 @@ def initialize(self) -> None:
# create a temporary dict of all keys needed in this method
localconf = AttrDict()
keys = ['DATA', 'current_cycle', 'COM_OBS', 'COM_ATMOS_RESTART_PREV',
'OPREFIX', 'CASE', 'OCNRES', 'ntiles']
'OPREFIX_OUT', 'CASE', 'OCNRES', 'ntiles']
for key in keys:
localconf[key] = self.task_config[key]

Expand Down
4 changes: 4 additions & 0 deletions workflow/applications/gfs_cycled.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ def _get_app_configs(self):
configs += ['atmensanlinit', 'atmensanlrun', 'atmensanlfinal']
else:
configs += ['eobs', 'eomg', 'ediag', 'eupd']
if self.do_jedisnowda:
configs += ['epsn']
configs += ['ecen', 'esfc', 'efcs', 'echgres', 'epos', 'earc']

if self.do_fit2obs:
Expand Down Expand Up @@ -165,6 +167,8 @@ def get_task_names(self):
else:
hybrid_tasks += ['eobs', 'eupd', 'echgres']
hybrid_tasks += ['ediag'] if self.lobsdiag_forenkf else ['eomg']
if self.do_jedisnowda:
hybrid_tasks += ['epsn']
hybrid_after_eupd_tasks += ['ecen', 'esfc', 'efcs', 'epos', 'earc', 'cleanup']

# Collect all "gdas" cycle tasks
Expand Down
39 changes: 39 additions & 0 deletions workflow/rocoto/gfs_tasks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2430,6 +2430,45 @@ def atmensanlfinal(self):

return task

def epsn(self):

deps = []
dep_dict = {'type': 'task', 'name': f'{self.cdump.replace("enkf","")}prep'}
deps.append(rocoto.add_dependency(dep_dict))
dep_dict = {'type': 'metatask', 'name': 'enkfgdasepmn', 'offset': '-06:00:00'}
deps.append(rocoto.add_dependency(dep_dict))
dependencies = rocoto.create_dependency(dep_condition='and', dep=deps)

epsnenvars = self.envars.copy()
epsnenvars_dict = {'ENSMEM': '#member#',
'MEMDIR': 'mem#member#'
}
for key, value in epsnenvars_dict.items():
epsnenvars.append(rocoto.create_envar(name=key, value=str(value)))

resources = self.get_resource('epsn')
task_name = f'{self.cdump}prepsnowobs_mem#member#'
task_dict = {'task_name': task_name,
'resources': resources,
'dependency': dependencies,
'envars': epsnenvars,
'cycledef': self.cdump.replace('enkf', ''),
'command': f'{self.HOMEgfs}/jobs/rocoto/prepsnowobs.sh',
'job_name': f'{self.pslot}_{task_name}_@H',
'log': f'{self.rotdir}/logs/@Y@m@d@H/{task_name}.log',
'maxtries': '&MAXTRIES;'
}

member_var_dict = {'member': ' '.join([str(mem).zfill(3) for mem in range(1, self.nmem + 1)])}
metatask_dict = {'task_name': f'{self.cdump}prepsnowobs',
'var_dict': member_var_dict,
'task_dict': task_dict
}

task = rocoto.create_task(metatask_dict)

return task

def ecen(self):

def _get_ecengroups():
Expand Down