Skip to content

Save marine LETKF yaml #3489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DavidHuber-NOAA merged 5 commits into from
Jun 6, 2025
Merged

Save marine LETKF yaml #3489

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4dcc0b6
Save marine LETKF yaml
Mar 23, 2025
f9310f2
Merge branch 'develop' into feature/save_marine_letkf_yaml
Jun 3, 2025
ad2a5da
Updates to save in COM_CONF and to archive
Jun 3, 2025
f4a74d7
add TODO
Jun 5, 2025
7696765
Merge branch 'develop' into feature/save_marine_letkf_yaml
DavidHuber-NOAA Jun 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions jobs/JGLOBAL_MARINE_ANALYSIS_LETKF
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMOUT_OCEAN_LETKF:COM_OCEAN_LETKF_TMPL \
COMOUT_ICE_LETKF:COM_ICE_LETKF_TMPL

# TODO: remove explicit RUN specification once https://github.com/NOAA-EMC/global-workflow/issues/3767 is fixed
RUN=${GDUMP_ENS} YMD=${PDY} HH=${cyc} MEMDIR="ensstat" declare_from_tmpl -rx \
Comment thread
aerorahul marked this conversation as resolved.
COMOUT_CONF:COM_CONF_TMPL

if [[ ! -d ${COMOUT_CONF} ]]; then mkdir -p "${COMOUT_CONF}"; fi
##############################################
# Begin JOB SPECIFIC work
##############################################
Expand Down
2 changes: 1 addition & 1 deletion parm/archive/enkf_marine_grp.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ enkf_marine_grp:
{% set COMIN_ICE_LETKF_MEM = COMIN_ICE_LETKF_MEM_list[imem] %}
{% set COMIN_ICE_HISTORY_MEM = COMIN_ICE_HISTORY_MEM_list[imem] %}


# Forecast data
{% if is_gdas %}
{% for fhr in range(3, 10, 3) %}
Expand All @@ -36,5 +35,6 @@ enkf_marine_grp:
{% if DOLETKF_OCN %}
- "{{ COMIN_OCEAN_LETKF_MEM | relpath(ROTDIR) }}/{{ head_ocean }}analysis.nc"
- "{{ COMIN_ICE_LETKF_MEM | relpath(ROTDIR) }}/{{ head_ice }}analysis.nc"
- "{{ COMIN_CONF | relpath(ROTDIR) }}/soca_letkf.yaml"
{% endif %}
{% endfor %} # first_group_mem to last_group_mem
6 changes: 6 additions & 0 deletions parm/archive/master_enkf.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,12 @@ datasets:

{% endfor %}

{% set tmpl_dict = ({ '${ROTDIR}':ROTDIR,
'${RUN}':RUN,
'${YMD}':cycle_YMD,
'${HH}':cycle_HH,
'${MEMDIR}': "ensstat"}) %}
{% set COMIN_CONF = COM_CONF_TMPL | replace_tmpl(tmpl_dict) %}
Comment on lines +77 to +82

@DavidHuber-NOAA DavidHuber-NOAA Jun 6, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be moved to JGLOBAL_ENS_ARCHIVE_TARS by inserting a declaration here:

global-workflow/jobs/JGLOBAL_ENS_ARCHIVE_TARS

Lines 9 to 15 in e131fb1

MEMDIR="ensstat" YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMIN_ATMOS_ANALYSIS_ENSSTAT:COM_ATMOS_ANALYSIS_TMPL \
COMIN_ATMOS_HISTORY_ENSSTAT:COM_ATMOS_HISTORY_TMPL \
COMIN_ICE_ANALYSIS_ENSSTAT:COM_ICE_ANALYSIS_TMPL \
COMIN_OCEAN_ANALYSIS_ENSSTAT:COM_OCEAN_ANALYSIS_TMPL \
COMIN_SNOW_ANALYSIS_ENSSTAT:COM_SNOW_ANALYSIS_TMPL \
COMOUT_CONF:COM_CONF_TMPL

But I'm OK with this being handled as part of #3767.

# Archive member data
{% filter indent(width=4) %}
{% include "enkf_grp.yaml.j2" %}
Expand Down
7 changes: 6 additions & 1 deletion ush/python/pygfs/task/marine_letkf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ def initialize(self):
'COMIN_ICE_HISTORY_PREV',
'COMOUT_ICE_LETKF',
'COMOUT_OCEAN_LETKF',
'COMOUT_CONF',
'DATA',
'DIST_HALO_SIZE',
'ENSPERT_RELPATH',
Expand Down Expand Up @@ -205,7 +206,7 @@ def finalize(self):
keys = ['current_cycle', 'DATA', 'NMEM_ENS', 'WINDOW_BEGIN', 'GDUMP_ENS',
'PARMgfs', 'ROTDIR', 'COM_OCEAN_LETKF_TMPL', 'COM_ICE_LETKF_TMPL',
'COMOUT_OCEAN_LETKF', 'COMOUT_ICE_LETKF', 'WINDOW_MIDDLE',
'MARINE_OBS_LIST_YAML']
'MARINE_OBS_LIST_YAML', 'COMOUT_CONF', 'letkf_yaml_file']
for key in keys:
letkfsaveconf[key] = self.task_config[key]

Expand All @@ -223,5 +224,9 @@ def finalize(self):
obs_files_to_copy.append([obs_src, obs_dst])
FileHandler({'mkdir': [os.path.join(letkfsaveconf.COMOUT_OCEAN_LETKF, 'diags')]}).sync()
FileHandler({'copy': obs_files_to_copy}).sync()
# yaml configurations
yamls_to_copy = []
yamls_to_copy.append([letkfsaveconf.letkf_yaml_file, os.path.join(letkfsaveconf.COMOUT_CONF, 'soca_letkf.yaml')])
FileHandler({'copy': yamls_to_copy}).sync()
letkf_save_list = parse_j2yaml(self.task_config.MARINE_LETKF_SAVE_YAML_TMPL, letkfsaveconf)
FileHandler(letkf_save_list).sync()