-
Notifications
You must be signed in to change notification settings - Fork 214
Refactoring of the marine B-matrix job #2749
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 18 commits
17494de
58115a6
5438a47
ee36b5e
8ae7f79
ba665f4
6900fc4
2d21d3d
8e137d0
216df7f
612b456
c034653
2311b4f
faa84bf
4913ff9
c93a43b
0e2e05e
6ab9a5d
96db47e
f6aaf60
8a7b148
7cf347d
c8cbba0
75cfc93
112085f
4e69124
6e0e2dc
4da5791
a0a34d9
ee30556
a209510
65eb775
c05582b
bbc1fa0
017d191
4ba7dee
3e85a94
5a7dd31
6d3704c
507b62d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,9 +14,10 @@ base: | |
| DO_VERFRAD: "YES" | ||
| DO_VRFY_OCEANDA: "NO" | ||
| FHMAX_GFS: 240 | ||
| ACCOUNT: {{ 'HPC_ACCOUNT' | getenv }} | ||
|
|
||
| ocnanal: | ||
| SOCA_INPUT_FIX_DIR: /scratch2/NCEPDEV/ocean/Guillaume.Vernieres/data/static/1440x1080x75/soca | ||
| SOCA_INPUT_FIX_DIR: {{ FIXgfs }}/gdas/soca/1440x1080x75/soca | ||
| CASE_ANL: 'C24' | ||
| SOCA_OBS_LIST: {{ HOMEgfs }}/sorc/gdas.cd/parm/soca/obs/obs_list.yaml | ||
| SOCA_NINNER: 100 | ||
|
|
@@ -25,6 +26,6 @@ ocnanal: | |
| NICAS_GRID_SIZE: 15000 | ||
|
|
||
| prepoceanobs: | ||
| SOCA_OBS_LIST: {{ HOMEgfs }}/sorc/gdas.cd/parm/soca/obs/obs_list.yaml | ||
| SOCA_OBS_LIST: {{ HOMEgfs }}/sorc/gdas.cd/parm/soca/obs/obs_list.yaml | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should be moved to
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What's wrong with keeping this exposed in the yaml? |
||
| OBSPREP_YAML: {{ HOMEgfs }}/sorc/gdas.cd/parm/soca/obsprep/obsprep_config.yaml | ||
| DMPDIR: /scratch1/NCEPDEV/da/common/ | ||
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| #!/bin/bash | ||
|
|
||
| source "${HOMEgfs}/ush/preamble.sh" | ||
|
|
||
| export DATA=${DATA:-${DATAROOT}/${RUN}marinebmat_${cyc}} | ||
|
CoryMartin-NOAA marked this conversation as resolved.
Outdated
|
||
| # source config.base, config.ocnanal and config.marinebmat | ||
| # and pass marinebmat to ${machine}.env | ||
| source "${HOMEgfs}/ush/jjob_header.sh" -e "marinebmat" -c "base ocnanal marinebmat" | ||
|
|
||
| # If ensemble perturbations are used, save them in a DATAenspert directory that will | ||
| # exist throughout the DA cycle | ||
| if (( 10#${NMEM_ENS:-0} > 0 )); then | ||
| export DATAjob="${DATAROOT}/${RUN}marinebmat.${PDY:-}${cyc}" | ||
| export DATAenspert="${DATAjob}/enspert" | ||
| if [[ ! -d "${DATAenspert}" ]]; then mkdir -p "${DATAenspert}"; fi | ||
| fi | ||
|
guillaumevernieres marked this conversation as resolved.
Outdated
|
||
|
|
||
| ############################################## | ||
| # Set variables used in the script | ||
| ############################################## | ||
| # shellcheck disable=SC2153 | ||
| GDATE=$(date --utc +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours") | ||
| gPDY=${GDATE:0:8} | ||
| gcyc=${GDATE:8:2} | ||
| export GDUMP="gdas" | ||
| export GDUMP_ENS="enkf${GDUMP}" | ||
|
guillaumevernieres marked this conversation as resolved.
|
||
|
|
||
| ############################################## | ||
| # Begin JOB SPECIFIC work | ||
| ############################################## | ||
|
|
||
| # Generate COM variables from templates | ||
| RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \ | ||
| COMIN_OCEAN_HISTORY_PREV:COM_OCEAN_HISTORY_TMPL \ | ||
| COMIN_ICE_HISTORY_PREV:COM_ICE_HISTORY_TMPL | ||
|
|
||
| RUN=${GDUMP_ENS} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx \ | ||
| COMIN_OCEAN_HISTORY_ENS_PREV:COM_OCEAN_HISTORY_TMPL \ | ||
| COMIN_ICE_HISTORY_ENS_PREV:COM_ICE_HISTORY_TMPL | ||
|
|
||
| YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \ | ||
| COMOUT_OCEAN_BMATRIX:COM_OCEAN_BMATRIX_TMPL \ | ||
| COMOUT_ICE_BMATRIX:COM_ICE_BMATRIX_TMPL | ||
|
|
||
| mkdir -p "${COMOUT_OCEAN_BMATRIX}" | ||
| mkdir -p "${COMOUT_ICE_BMATRIX}" | ||
|
|
||
| ############################################################### | ||
| # Run relevant script | ||
|
|
||
| EXSCRIPT=${GDASMARINEBMATRUNPY:-${SCRgfs}/exglobal_marine_bmat_run.py} | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ok, will push the changes after I'm done testing |
||
| ${EXSCRIPT} | ||
| status=$? | ||
| [[ ${status} -ne 0 ]] && exit "${status}" | ||
|
|
||
| ############################################## | ||
| # End JOB SPECIFIC work | ||
| ############################################## | ||
|
|
||
| ############################################## | ||
| # Final processing | ||
| ############################################## | ||
| if [[ -e "${pgmout}" ]] ; then | ||
| cat "${pgmout}" | ||
| fi | ||
|
|
||
| exit 0 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,23 +3,30 @@ gdasocean_analysis: | |
| name: "GDASOCEAN_ANALYSIS" | ||
| target: "{{ ATARDIR }}/{{ cycle_YMDH }}/gdasocean_analysis.tar" | ||
| required: | ||
| # analysis and analysis increments | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}ocninc.nc' | ||
| {% set ocngrid_cycle = '%02d' % (((cycle_HH | int) - 3) % 24) %} | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/gdas.t{{ ocngrid_cycle }}z.ocngrid.nc' | ||
| {% for domain in ["ocn", "ice"] %} | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}{{domain}}.bkgerr_stddev.nc' | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}{{domain}}.incr.nc' | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}{{domain}}ana.nc' | ||
| {% if NMEM_ENS > 2 %} | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}{{domain}}.recentering_error.nc' | ||
| {% endif %} | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}{{domain}}.incr.nc' | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}{{domain}}ana.nc' | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will do this when refactoring the deterministic analysis. |
||
| {% endfor %} | ||
|
|
||
| # static background error | ||
| - '{{ COMIN_OCEAN_BMATRIX | relpath(ROTDIR) }}/{{ head }}ocean.bkgerr_stddev.nc' | ||
| - '{{ COMIN_ICE_BMATRIX | relpath(ROTDIR) }}/{{ head }}ice.bkgerr_stddev.nc' | ||
|
|
||
| # ensemble background error | ||
| {% if NMEM_ENS > 2 %} | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}ocn.ssh_steric_stddev.nc' | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}ocn.ssh_unbal_stddev.nc' | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}ocn.ssh_total_stddev.nc' | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}ocn.steric_explained_variance.nc' | ||
| - '{{ COMIN_ICE_BMATRIX | relpath(ROTDIR) }}/{{ head }}ice.ens_weights.nc' | ||
| - '{{ COMIN_OCEAN_BMATRIX | relpath(ROTDIR) }}/{{ head }}ocean.ens_weights.nc' | ||
| - '{{ COMIN_OCEAN_BMATRIX | relpath(ROTDIR) }}/{{ head }}ocean.recentering_error.nc' | ||
| {% for diag_type in ["ssh_steric_stddev", "ssh_unbal_stddev", "ssh_total_stddev", "steric_explained_variance"] %} | ||
| - '{{ COMIN_OCEAN_BMATRIX | relpath(ROTDIR) }}/{{ head }}ocean.{{ diag_type }}.nc' | ||
| {% endfor %} | ||
| {% endif %} | ||
|
|
||
| # obs space diags | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/{{ head }}ocn.*.stats.csv' | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/diags/*.nc4' | ||
|
|
||
| # runtime configs | ||
| - '{{ COMIN_OCEAN_ANALYSIS | relpath(ROTDIR) }}/yaml/*.yaml' | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| #!/bin/bash | ||
|
|
||
| ########## config.marinebmat ########## | ||
| # configuration for the marine B-matrix | ||
|
|
||
| echo "BEGIN: config.marinebmat" | ||
|
|
||
| # Get task specific resources | ||
| . "${EXPDIR}/config.resources" marinebmat | ||
|
|
||
| echo "END: config.marinebmat" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this work?
FIXgfsis not available at the setup level. It is available in the jobs. This should be moved to the config files.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just tested again, it does work.
{{ FIXgfs }}/gdas/soca/1440x1080x75/socabecomes${ FIXgfs}/gdas/soca/1440x1080x75/socain the config under exp dir.I want to keep the option exposed in the yaml file, since that fix dir is still fluctuating ...