Skip to content

SFS Runs at C96mx100 #2960

Merged
WalterKolczynski-NOAA merged 95 commits into
NOAA-EMC:developfrom
NeilBarton-NOAA:SFS
Nov 13, 2024
Merged

SFS Runs at C96mx100 #2960
WalterKolczynski-NOAA merged 95 commits into
NOAA-EMC:developfrom
NeilBarton-NOAA:SFS

Conversation

@NeilBarton-NOAA
Copy link
Copy Markdown
Contributor

@NeilBarton-NOAA NeilBarton-NOAA commented Sep 25, 2024

Description

Adds the ability to run C96mx100 with SFS options. Included in this PR is the ability to use the interpolated/coarse grained 1 degree MOM6 ICs. Staging the ATM and OCN perturbation files without using the +03 replay ICs.

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? YES
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

This has been tested in the gefs forecast-only configurations on hera and hercules.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • I have made corresponding changes to the system documentation if necessary

NeilBarton-NOAA and others added 30 commits July 2, 2024 15:04
@NeilBarton-NOAA
initial creation of CI using replay ICs
30cf5df
@NeilBarton-NOAA
change name
4b3ae39
@NeilBarton-NOAA
making an S2SWA case
fe2054e
@NeilBarton-NOAA
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
7b7ee9b
@NeilBarton-NOAA
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
225d280
@NeilBarton-NOAA
start of CI for GEFS replays # Please enter the commit message for your
b17765e 
changes. Lines starting
@NeilBarton-NOAA
Merge branch 'develop' into gefs_replay_ci
675cb77
@NeilBarton-NOAA
removing shell check errors
e49dbcf
@NeilBarton-NOAA
Merge branch 'gefs_replay_ci' of https://github.com/NeilBarton-NOAA/g…
ec4becc 
…lobal-workflow into gefs_replay_ci
@NeilBarton-NOAA
fixes for wave post processing
b69d653
@NeilBarton-NOAA
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
084d744
@NeilBarton-NOAA
Merge remote-tracking branch 'origin/develop' into gefs_replay_ci
a4bb8f3
@NeilBarton-NOAA
add BREAKPOINTS variable
4ff31fd
@NeilBarton-NOAA
fix shell checks
97b92ee
@NeilBarton-NOAA
edit defult config.efc to be close to re-forecasting
713f41d
@EricSinsky-NOAA
Fix bug in variable check
af31d2f 
Fix a bug in a variable check in the extractvars task where parenthesis in the variable name will prompt a false warning message.
@EricSinsky-NOAA
Add more settings to GEFS replay CI test
cec7b81
@EricSinsky-NOAA
Add atmos variables requested for reforeacst
4aaca9a
@EricSinsky-NOAA
Reorganize TSNOWP in paramlist
11e1957
@EricSinsky-NOAA
Extract ice variables from history file instead
a005e5f 
Ice variables are extracted from the history files instead of the product files. This is necessary because there are certain ice variables that are needed that are only available in the history files and not the product files.
@EricSinsky-NOAA
Fix typo
2dd9a7e
@EricSinsky-NOAA
Update TCDC variable to correct name
b132408
@NeilBarton-NOAA
Merge pull request #7 from EricSinsky-NOAA/gefs_replay_ci_pr
3d9455b 
Add suggested patches to gefs_replay_ci
@NeilBarton-NOAA
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
447224c
@NeilBarton-NOAA
edits to stage_ic for replay perturbations
c6b7eaa
@NeilBarton-NOAA
adding icsdir to pr CI yaml
51aa2dd
@NeilBarton-NOAA
Merge branch 'NOAA-EMC:develop' into gefs_replay_ci
a641591
@EricSinsky-NOAA
Fix ensstat failures
4985361
@NeilBarton-NOAA
updates for products
8eaef27
@EricSinsky-NOAA
Add shellcheck disable and skip f000 for ocn/ice
2e7241f 
Shellcheck disable has been added. Also, f000 is skipped in extractvars for ocn and ice since there are no products for f000 for ocn and ice. This change will remove warning messages.
@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Nov 7, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 7, 2024

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hera in Build# 3 in
/scratch1/NCEPDEV/global/CI/2960/RUNTESTS/EXPDIR/C96_S2SWA_gefs_replay_ics_4dcb3ac8

@emcbot emcbot added CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed and removed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Nov 8, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 8, 2024

CI Failed on Hercules in Build# 2
Built and ran in directory /work2/noaa/global/CI/HERCULES/2960


Experiment C96_S2SWA_gefs_replay_ics_4dcb3ac8 Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Thu Nov  7 15:59:03 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_4dcb3ac8 Terminated: *FAIL*
Error logs:
/work2/noaa/global/CI/HERCULES/2960/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_4dcb3ac8/logs/2020110100/gefs_fcst_mem001_seg0.log
/work2/noaa/global/CI/HERCULES/2960/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_4dcb3ac8/logs/2020110100/gefs_fcst_mem002_seg0.log
Experiment C48_ATM_4dcb3ac8 Completed 2 Cycles: *SUCCESS* at Thu Nov  7 18:48:29 CST 2024
Experiment C48_S2SW_4dcb3ac8 Completed 2 Cycles: *SUCCESS* at Thu Nov  7 20:13:30 CST 2024
Experiment C96_atm3DVar_4dcb3ac8 Completed 3 Cycles: *SUCCESS* at Thu Nov  7 20:37:48 CST 2024
Experiment C96C48_hybatmDA_4dcb3ac8 Completed 3 Cycles: *SUCCESS* at Thu Nov  7 20:37:48 CST 2024
Experiment C48_S2SWA_gefs_4dcb3ac8 Completed 1 Cycles: *SUCCESS* at Thu Nov  7 20:57:41 CST 2024

@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor

WalterKolczynski-NOAA commented Nov 8, 2024

The GEFS replay test is failing because it is looking for a non-existent increment file.

@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Nov 8, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 8, 2024

CI Failed on Hera in Build# 3
Built and ran in directory /scratch1/NCEPDEV/global/CI/2960


Experiment C96_S2SWA_gefs_replay_ics_4dcb3ac8 Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Thu Nov  7 22:11:33 UTC 2024
Experiment C96_S2SWA_gefs_replay_ics_4dcb3ac8 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2960/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_4dcb3ac8/logs/2020110100/gefs_fcst_mem001_seg0.log
/scratch1/NCEPDEV/global/CI/2960/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_4dcb3ac8/logs/2020110100/gefs_fcst_mem002_seg0.log
Experiment C48_ATM_4dcb3ac8 Completed 2 Cycles: *SUCCESS* at Fri Nov  8 01:45:00 UTC 2024
Experiment C48mx500_3DVarAOWCDA_4dcb3ac8 Completed 2 Cycles: *SUCCESS* at Fri Nov  8 02:21:34 UTC 2024
Experiment C48_S2SW_4dcb3ac8 Completed 2 Cycles: *SUCCESS* at Fri Nov  8 03:23:01 UTC 2024
Experiment C48_S2SWA_gefs_4dcb3ac8 Completed 1 Cycles: *SUCCESS* at Fri Nov  8 03:24:57 UTC 2024
Experiment C96_atm3DVar_4dcb3ac8 Completed 3 Cycles: *SUCCESS* at Fri Nov  8 03:53:34 UTC 2024
Experiment C96C48_hybatmDA_4dcb3ac8 Completed 3 Cycles: *SUCCESS* at Fri Nov  8 03:54:37 UTC 2024
Experiment C96C48_hybatmaerosnowDA_4dcb3ac8 Completed 3 Cycles: *SUCCESS* at Fri Nov  8 04:06:47 UTC 2024
Experiment C96C48_ufs_hybatmDA_4dcb3ac8 Completed 3 Cycles: *SUCCESS* at Fri Nov  8 04:37:06 UTC 2024

@NeilBarton-NOAA
Copy link
Copy Markdown
Contributor Author

NeilBarton-NOAA commented Nov 8, 2024

@WalterKolczynski-NOAA fix for CI test added

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Nov 12, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Nov 12, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 13, 2024

CI Passed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/CI/2960


Experiment C96_S2SWA_gefs_replay_ics_2215d40e Completed 1 Cycles: *SUCCESS* at Wed Nov 13 03:18:40 UTC 2024
Experiment C48mx500_3DVarAOWCDA_2215d40e Completed 2 Cycles: *SUCCESS* at Wed Nov 13 03:30:19 UTC 2024
Experiment C48_ATM_2215d40e Completed 2 Cycles: *SUCCESS* at Wed Nov 13 07:57:41 UTC 2024
Experiment C48_S2SW_2215d40e Completed 2 Cycles: *SUCCESS* at Wed Nov 13 07:58:16 UTC 2024
Experiment C48_S2SWA_gefs_2215d40e Completed 1 Cycles: *SUCCESS* at Wed Nov 13 09:07:54 UTC 2024
Experiment C96_atm3DVar_2215d40e Completed 3 Cycles: *SUCCESS* at Wed Nov 13 10:17:48 UTC 2024
Experiment C96C48_hybatmaerosnowDA_2215d40e Completed 3 Cycles: *SUCCESS* at Wed Nov 13 10:48:12 UTC 2024
Experiment C96C48_hybatmDA_2215d40e Completed 3 Cycles: *SUCCESS* at Wed Nov 13 10:54:16 UTC 2024
Experiment C96C48_ufs_hybatmDA_2215d40e Completed 3 Cycles: *SUCCESS* at Wed Nov 13 12:19:42 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules label Nov 13, 2024
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules labels Nov 13, 2024
@emcbot
Copy link
Copy Markdown

emcbot commented Nov 13, 2024

CI Passed on Hercules in Build# 2
Built and ran in directory /work2/noaa/global/CI/HERCULES/2960


Experiment C48_ATM_fcc0983d Completed 2 Cycles: *SUCCESS* at Wed Nov 13 15:10:03 CST 2024
Experiment C96_S2SWA_gefs_replay_ics_fcc0983d Completed 1 Cycles: *SUCCESS* at Wed Nov 13 15:16:10 CST 2024
Experiment C96C48_hybatmDA_fcc0983d Completed 3 Cycles: *SUCCESS* at Wed Nov 13 16:23:30 CST 2024
Experiment C96_atm3DVar_fcc0983d Completed 3 Cycles: *SUCCESS* at Wed Nov 13 16:41:48 CST 2024
Experiment C48_S2SWA_gefs_fcc0983d Completed 1 Cycles: *SUCCESS* at Wed Nov 13 17:30:26 CST 2024
Experiment C48_S2SW_fcc0983d Completed 2 Cycles: *SUCCESS* at Wed Nov 13 17:54:34 CST 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aerorahul aerorahul aerorahul left review comments

@junwang-noaa junwang-noaa junwang-noaa left review comments

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

@DavidHuber-NOAA DavidHuber-NOAA Awaiting requested review from DavidHuber-NOAA DavidHuber-NOAA is a code owner

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA Awaiting requested review from TerrenceMcGuinness-NOAA

+2 more reviewers

@lisa-bengtsson lisa-bengtsson lisa-bengtsson approved these changes

@KateFriedman-NOAA KateFriedman-NOAA KateFriedman-NOAA left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@NeilBarton-NOAA @aerorahul @WalterKolczynski-NOAA @emcbot @KateFriedman-NOAA @junwang-noaa @lisa-bengtsson @github-advanced-security @TerrenceMcGuinness-NOAA @EricSinsky-NOAA