-
Notifications
You must be signed in to change notification settings - Fork 217
Updates to enable soil DA #3452
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
aerorahul
merged 29 commits into
NOAA-EMC:develop
from
ClaraDraper-NOAA:feature/mv_regridding
Apr 23, 2025
Merged
Changes from 21 commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
cfa3041
Updates to
ClaraDraper-NOAA 104f0f3
Merge branch 'develop' into feature/mv_regridding
ClaraDraper-NOAA 9fa8c62
Tidy up.
ClaraDraper-NOAA edb2478
Updated spack-stack version so can load esmf/8.6.0
ClaraDraper-NOAA af97407
run soil DA test on all machines.
ClaraDraper-NOAA 2dfc8b9
Linking sfc_regrid executable into global_workflow exec.
ClaraDraper-NOAA 26920cd
Add new spack-stack to hercules, gaea[56], and added explicit
ClaraDraper-NOAA caeee2d
Amend previous commit.
ClaraDraper-NOAA fccda64
Merge branch 'develop' into feature/mv_regridding
ClaraDraper-NOAA e6326f5
Bugfix in GAEA env files.
ClaraDraper-NOAA 91d3a4e
bugfix to Gaea[56] env files.
ClaraDraper-NOAA 1dcecd4
remove errornous dependency of gfs_sfcanl on eupd
ClaraDraper-NOAA 7fa94ab
py norms.
ClaraDraper-NOAA dc8162f
Merge branch 'develop' into feature/mv_regridding
ClaraDraper-NOAA 2b26ddc
Updated modules on orion.
ClaraDraper-NOAA dce0350
script change to allow monitoring of 2m obs.
ClaraDraper-NOAA b27920b
Prevent regridding exec from being over-written.
ClaraDraper-NOAA d0aae40
Merge branch 'develop' into feature/mv_regridding
ClaraDraper-NOAA 1ff8710
Restricting regid_ntasks to 6, for reproducibility.
ClaraDraper-NOAA c53f0e9
Merge branch 'develop' into feature/mv_regridding
ClaraDraper-NOAA fb8824a
Updated UFS_UTILS hash.
ClaraDraper-NOAA 289c668
initialize in hofx_2m_sfcfile in exgdas_enkf_update.sh
ClaraDraper-NOAA f7ccd13
Merge branch 'develop' into feature/mv_regridding
ClaraDraper-NOAA 1643aa5
Merged soil DA into hybatm test, and removed separate soil DA test.
ClaraDraper-NOAA 7a07691
coding norms.
ClaraDraper-NOAA a8dd91a
Removed soilDA test yaml.
ClaraDraper-NOAA 7901ff7
Merge branch 'develop' into feature/mv_regridding
ClaraDraper-NOAA 237608b
Update dev/jobs/sfcanl.sh
aerorahul cbe0916
Merge branch 'develop' into feature/mv_regridding
aerorahul File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| experiment: | ||
| system: gfs | ||
| mode: cycled | ||
|
|
||
| arguments: | ||
| account: {{ 'HPC_ACCOUNT' | getenv }} | ||
| pslot: {{ 'pslot' | getenv }} | ||
| app: ATM | ||
| resdetatmos: 96 | ||
| resensatmos: 48 | ||
| comroot: {{ 'RUNTESTS' | getenv }}/COMROOT | ||
| expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR | ||
| icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96C48/20241120 | ||
| idate: 2021122012 | ||
| edate: 2021122100 | ||
| nens: 2 | ||
| interval: 24 | ||
| start: cold | ||
| yaml: {{ HOMEgfs }}/ci/cases/yamls/atmsoilDA_defaults_ci.yaml | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Submodule ufs_utils.fd
updated
49 files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| export stack_intel_ver=2023.2.0 | ||
| export stack_cray_mpich_ver=8.1.28 | ||
| export spack_env=gsi-addon-dev | ||
| export spack_env=gsi-addon-dev-fms-2024.01 | ||
| source "${HOMEgfs:-}/versions/spack.ver" | ||
| export spack_mod_path="/ncrc/proj/epic/spack-stack/spack-stack-${spack_stack_ver}/envs/${spack_env}/install/modulefiles/Core" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| export stack_intel_ver=2023.2.0 | ||
| export stack_cray_mpich_ver=8.1.29 | ||
| export spack_env=gsi-addon | ||
| export spack_env=gsi-addon-dev-fms-2024.01 | ||
|
DavidHuber-NOAA marked this conversation as resolved.
|
||
| source "${HOMEgfs:-}/versions/spack.ver" | ||
| export spack_mod_path="/ncrc/proj/epic/spack-stack/c6/spack-stack-${spack_stack_ver}/envs/${spack_env}/install/modulefiles/Core" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| export stack_intel_ver=2021.5.0 | ||
| export stack_impi_ver=2021.5.1 | ||
| export spack_env=gsi-addon-dev-rocky8 | ||
| export spack_env=gsi-addon-dev-fms-2024.01 | ||
| source "${HOMEgfs:-}/versions/spack.ver" | ||
| export spack_mod_path="/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-${spack_stack_ver}/envs/${spack_env}/install/modulefiles/Core" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| export stack_intel_ver=2021.9.0 | ||
| export stack_impi_ver=2021.9.0 | ||
| export intel_mkl_ver=2023.1.0 | ||
| export spack_env=gsi-addon-env | ||
| export spack_env=gsi-addon-dev-fms-2024.01 | ||
| source "${HOMEgfs:-}/versions/spack.ver" | ||
| export spack_mod_path="/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-${spack_stack_ver}/envs/${spack_env}/install/modulefiles/Core" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| export stack_intel_ver=2021.9.0 | ||
| export stack_impi_ver=2021.9.0 | ||
| export spack_env=gsi-addon-env-rocky9 | ||
| export spack_env=gsi-addon-dev-fms-2024.01 | ||
| source "${HOMEgfs:-}/versions/spack.ver" | ||
| export spack_mod_path="/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-${spack_stack_ver}/envs/${spack_env}/install/modulefiles/Core" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we just combine this with the existing CI test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I second this. Could this be combined with the existing
ci/cases/pr/C96C48_hybatmDA.yaml?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I asked @CatherineThomas-NOAA about this, and I think she's planning to include it in the GFSv17 test. Would this suffice?
It's useful for me to have the test there, but if it's cludging up your tests I don't mind removing it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we just ensure that the hybatmDA test includes the soil DA work, no need to duplicate it then
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can turn on the soil DA in the hybatmDA test, and delete the current test. This will require establishing a new yaml for the combined test, since gfs_defaults_ci.yaml is used by multiple tests that don't have ensembles (so the soil DA will fail).
What does this do: DO_TEST_MODE: "YES" (This is currently NO for the snowDA test, and YES for the hybatm)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@DavidNew-NOAA can you give Clara an explanation of how this is used combined with GDASApp?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ClaraDraper-NOAA
DO_TEST_MODE: "YES"turns on JEDI application tests. It's referenced in the JCB algorithm YAMLs for various jobs, telling JCB what the parameters (reference file, test tolerances, etc) are for the associated JEDI application.For atm 3dvar for example:
https://github.com/NOAA-EMC/GDASApp/blob/91b6c1d25fabc3eb439ae66b925e0536b8f022ac/test/gw-ci/atm/jcb-prototype_3dvar_ufs_hybatmDA.yaml.j2#L36
For this example, those parameters would go in here:
https://github.com/NOAA-EMC/jcb-algorithms/blob/1c39f50c41a51e15e8e839ec86aff0cb71b7406f/3dvar.yaml.j2#L100
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @DavidNew-NOAA. Is this still relevant here, even though this test is not running JEDI?
If so, I assume I'll need to create new baselines once I add the soil DA to the test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, not really relevant here, though in one case it's used for testing in GSI, reducing the number of iterations in the minimizer:
global-workflow/parm/config/gfs/config.anal
Line 33 in 6021dd2
So you can more or less use it for whatever you want in the context of testing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks - I left DO_TEST_MODE: "YES" to keep the faster minimization.