Sync with ESCOMP/CDEPS (2024-12-16) + Add support for traditional threading (#2533) by NickSzapiro-NOAA · Pull Request #2538 · ufs-community/ufs-weather-model

NickSzapiro-NOAA · 2024-12-17T16:23:16Z

Commit Queue Requirements:

Fill out all sections of this template.
All sub component pull requests have been reviewed by their code managers.
Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
Commit 'test_changes.list' from previous step

Description:

Merge authoritative ESCOMP into EMC fork for bug fix for alarm code in CDEPS share code used by CMEPS

Description of adding support for traditional threading is available in #2533

Commit Message:

* UFSWM - This PR adds support for traditional (non esmf managed) threading. ESMF managed threading is still default.
* UFSWM - Sync with ESCOMP/CDEPS (2024-12-16)
  * CDEPS - Sync with ESCOMP/CDEPS (2024-12-16)

Priority:

High: bugfix blocking CMEPS from writing mediator history files (critical for debugging)

Git Tracking

UFSWM:

Closes #nuopc_shr_methods needs update to enable mediator history files w/ nsteps frequency NOAA-EMC/CDEPS#67

Sub component Pull Requests:

CDEPS: Sync with ESCOMP/CDEPS (2024-12-16) NOAA-EMC/CDEPS#68

UFSWM Blocking Dependencies:

None

Changes

Regression Test Changes (Please commit test_changes.list):

No Baseline Changes.

Input data Changes:

None.

Library Changes/Upgrades:

No Updates

Testing Log:

Use this variable to turn on writing histaux files every MED_history_n. By default it is .false. Currently only cpld_control_nowave_noaero_p8 test has it turned on.

DeniseWorthen

Looks good. Thanks for getting this fix in so quickly.

jkbk2004 · 2024-12-19T20:29:04Z

@NickSzapiro-NOAA Please, go ahead to sync up branch and combine in #2533. @DusanJovic-NOAA FYI

NickSzapiro-NOAA · 2024-12-19T20:50:31Z

I can do this in an hour or two. Please feel free to make the changes, if that makes a difference

…

On Thu, Dec 19, 2024, 3:29 PM JONG KIM ***@***.***> wrote: @NickSzapiro-NOAA <https://github.com/NickSzapiro-NOAA> Please, go ahead to sync up branch and combine in #2533 <#2533>. @DusanJovic-NOAA <https://github.com/DusanJovic-NOAA> FYI — Reply to this email directly, view it on GitHub <#2538 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BDXAKB4O6HUFCNXNBVBLBND2GMUCNAVCNFSM6AAAAABTYZKZ4OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJVG4YTOMRQGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

NickSzapiro-NOAA · 2024-12-19T21:34:23Z

@jkbk2004 @DusanJovic-NOAA Should be ready

…reads cases

jkbk2004 · 2024-12-23T19:46:34Z

we can start merging process.

NickSzapiro-NOAA · 2024-12-23T20:10:59Z

CDEPS was merged and updated here. Should be ready. Thank you @jkbk2004 @BrianCurtis-NOAA

@sbanihash

This PR adds the following: * converting from inp -> nml (@sbanihash) * turning on PIO for waves for restarts (@sbanihash) * enabling cycling for WW3 which required some updates to wave prep jobs + changing what restarts are being saved/etc * changed the way CEMPS, MOM6, CICE and WW3 write restarts to be in sync with FV3 for IAU, which required moving the ufs-weather-model forward one hash to use the new flexible restart feature. (UFS PR ufs-community/ufs-weather-model#2419) * adds uglo_15km - the targeted new wave grid. * Update to use new esmf_threading ufs.configure files which changes the toggle between how you use esmf threading versus traditional threading (UFS PR ufs-community/ufs-weather-model#2538) Notes on ufs-weather-model updates: | Commit date | Commit hash/ PR | Notes for g-w changes | Baseline Changes | | :------------- | :------------- | :------------- | :------------- | | Dec 11, 2024 | ufs-community/ufs-weather-model@409bc85 ufs-community/ufs-weather-model#2419 | Enables flexible restart writes - changes included in g-w PR | none| | Dec 16, 2024 | ufs-community/ufs-weather-model@6ec6b45 ufs-community/ufs-weather-model#2528 ufs-community/ufs-weather-model#2469 | n/a | HAFs test changes, no global changes | | Dec 18, 2024 |ufs-community/ufs-weather-model@e119370 ufs-community/ufs-weather-model#2448 | Adds Gaea C6 support (changes in other g-w PRs, not here) | none | |Dec 23, 2024 | ufs-community/ufs-weather-model@2950089 ufs-community/ufs-weather-model#2533 ufs-community/ufs-weather-model#2538 | changes for ESMF vs traditional threading | none | |Dec 30, 2024 | ufs-community/ufs-weather-model@241dd8e ufs-community/ufs-weather-model#2485 | n/a | changes in conus13km, no global changes| |Jan 3, 2025 | ufs-community/ufs-weather-model@76471dc ufs-community/ufs-weather-model#2530 | n/a | changes in regional tests, no global changes | Note this PR requires the following: * update to fix files to add uglo_15km * staging ICs for high resolution test case for uglo_15km Co-author: @sbanihash Related Issues: - Fixes #1457 - Fixes #3154 - Fixes #1795 - related to #1776 --------- Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com> Co-authored-by: Saeideh Banihashemi <saeideh.banihashemi@noaa.gov> Co-authored-by: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com> Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>

@sbanihash

This PR adds the following: * converting from inp -> nml (@sbanihash) * turning on PIO for waves for restarts (@sbanihash) * enabling cycling for WW3 which required some updates to wave prep jobs + changing what restarts are being saved/etc * changed the way CEMPS, MOM6, CICE and WW3 write restarts to be in sync with FV3 for IAU, which required moving the ufs-weather-model forward one hash to use the new flexible restart feature. (UFS PR ufs-community/ufs-weather-model#2419) * adds uglo_15km - the targeted new wave grid. * Update to use new esmf_threading ufs.configure files which changes the toggle between how you use esmf threading versus traditional threading (UFS PR ufs-community/ufs-weather-model#2538) Notes on ufs-weather-model updates: | Commit date | Commit hash/ PR | Notes for g-w changes | Baseline Changes | | :------------- | :------------- | :------------- | :------------- | | Dec 11, 2024 | ufs-community/ufs-weather-model@409bc85 ufs-community/ufs-weather-model#2419 | Enables flexible restart writes - changes included in g-w PR | none| | Dec 16, 2024 | ufs-community/ufs-weather-model@6ec6b45 ufs-community/ufs-weather-model#2528 ufs-community/ufs-weather-model#2469 | n/a | HAFs test changes, no global changes | | Dec 18, 2024 |ufs-community/ufs-weather-model@e119370 ufs-community/ufs-weather-model#2448 | Adds Gaea C6 support (changes in other g-w PRs, not here) | none | |Dec 23, 2024 | ufs-community/ufs-weather-model@2950089 ufs-community/ufs-weather-model#2533 ufs-community/ufs-weather-model#2538 | changes for ESMF vs traditional threading | none | |Dec 30, 2024 | ufs-community/ufs-weather-model@241dd8e ufs-community/ufs-weather-model#2485 | n/a | changes in conus13km, no global changes| |Jan 3, 2025 | ufs-community/ufs-weather-model@76471dc ufs-community/ufs-weather-model#2530 | n/a | changes in regional tests, no global changes | Note this PR requires the following: * update to fix files to add uglo_15km * staging ICs for high resolution test case for uglo_15km Co-author: @sbanihash Related Issues: - Fixes NOAA-EMC#1457 - Fixes NOAA-EMC#3154 - Fixes NOAA-EMC#1795 - related to NOAA-EMC#1776 --------- Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com> Co-authored-by: Saeideh Banihashemi <saeideh.banihashemi@noaa.gov> Co-authored-by: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com> Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>

@sbanihash

This PR adds the following: * converting from inp -> nml (@sbanihash) * turning on PIO for waves for restarts (@sbanihash) * enabling cycling for WW3 which required some updates to wave prep jobs + changing what restarts are being saved/etc * changed the way CEMPS, MOM6, CICE and WW3 write restarts to be in sync with FV3 for IAU, which required moving the ufs-weather-model forward one hash to use the new flexible restart feature. (UFS PR ufs-community/ufs-weather-model#2419) * adds uglo_15km - the targeted new wave grid. * Update to use new esmf_threading ufs.configure files which changes the toggle between how you use esmf threading versus traditional threading (UFS PR ufs-community/ufs-weather-model#2538) Notes on ufs-weather-model updates: | Commit date | Commit hash/ PR | Notes for g-w changes | Baseline Changes | | :------------- | :------------- | :------------- | :------------- | | Dec 11, 2024 | ufs-community/ufs-weather-model@409bc85 ufs-community/ufs-weather-model#2419 | Enables flexible restart writes - changes included in g-w PR | none| | Dec 16, 2024 | ufs-community/ufs-weather-model@6ec6b45 ufs-community/ufs-weather-model#2528 ufs-community/ufs-weather-model#2469 | n/a | HAFs test changes, no global changes | | Dec 18, 2024 |ufs-community/ufs-weather-model@e119370 ufs-community/ufs-weather-model#2448 | Adds Gaea C6 support (changes in other g-w PRs, not here) | none | |Dec 23, 2024 | ufs-community/ufs-weather-model@2950089 ufs-community/ufs-weather-model#2533 ufs-community/ufs-weather-model#2538 | changes for ESMF vs traditional threading | none | |Dec 30, 2024 | ufs-community/ufs-weather-model@241dd8e ufs-community/ufs-weather-model#2485 | n/a | changes in conus13km, no global changes| |Jan 3, 2025 | ufs-community/ufs-weather-model@76471dc ufs-community/ufs-weather-model#2530 | n/a | changes in regional tests, no global changes | Note this PR requires the following: * update to fix files to add uglo_15km * staging ICs for high resolution test case for uglo_15km Co-author: @sbanihash Related Issues: - Fixes NOAA-EMC#1457 - Fixes NOAA-EMC#3154 - Fixes NOAA-EMC#1795 - related to NOAA-EMC#1776 --------- Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com> Co-authored-by: Saeideh Banihashemi <saeideh.banihashemi@noaa.gov> Co-authored-by: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com> Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>

DusanJovic-NOAA and others added 6 commits December 5, 2024 07:11

Add support for traditional threading. 'ESMF threading' is still default

5485db8

Merge remote-tracking branch 'origin/develop' into rt_tt

0064773

Convert all '2threads' tests to traditional threading

f089968

Add new config variable 'histaux_enabled'

441a09f

Use this variable to turn on writing histaux files every MED_history_n. By default it is .false. Currently only cpld_control_nowave_noaero_p8 test has it turned on.

Use feature branch CDEPS

dad2b3f

Tests pass bfb on hercules

fcf472d

NickSzapiro-NOAA mentioned this pull request Dec 17, 2024

Sync with ESCOMP/CDEPS (2024-12-16) NOAA-EMC/CDEPS#68

Merged

NickSzapiro-NOAA self-assigned this Dec 17, 2024

NickSzapiro-NOAA requested a review from DeniseWorthen December 17, 2024 16:24

NickSzapiro-NOAA added the No Baseline Change No Baseline Change label Dec 17, 2024

DeniseWorthen previously approved these changes Dec 17, 2024

View reviewed changes

jkbk2004 mentioned this pull request Dec 18, 2024

Sixth reconciliation PR from production/RRFS.v1 + FV3 CCPP: wet() bug fix and workaround for bad tsfco #2572 #2529

Merged

14 tasks

NickSzapiro-NOAA added 2 commits December 19, 2024 16:30

Merge branch 'ufs-community:develop' into sync_cdeps_2024-12-16

6ce0844

Merge ufs-community#2533 from DusanJovic-NOAA/rt_tt

527939e

NickSzapiro-NOAA dismissed DeniseWorthen’s stale review via 527939e December 19, 2024 21:33

NickSzapiro-NOAA changed the title ~~Sync with ESCOMP/CDEPS (2024-12-16)~~ Sync with ESCOMP/CDEPS (2024-12-16) + Add support for traditional threading (#2533) Dec 20, 2024

jkbk2004 and others added 9 commits December 20, 2024 15:25

add hera RT log: passed

1623c96

add gaeac5 RT log: passed

2b9b547

add gaeac6 RT log: passed

ddbec5c

add jet RT log: passed

348b89d

add derecho RT log: passed

0859ba5

add orion RT log: passed

572eba5

add hercules RT log: passed with ESMF-based thrd for gnu rap/hrrr 2th…

a04f1c4

…reads cases

extend WLCLK for cpld_2threads_p8 on orion

89a7f8d

Acorn RT Log: Passed

1da0371

jkbk2004 added the Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked. label Dec 23, 2024

Bump WLCLK for cpld_2threads_p8 on WCOSS2 and add WCOSS2 Log: Passed

53dfd47

Revert .gitmodules. Update CDEPS

d9b68cb

jkbk2004 requested review from BrianCurtis-NOAA and dpsarmie December 23, 2024 20:20

jkbk2004 approved these changes Dec 23, 2024

View reviewed changes

dpsarmie approved these changes Dec 23, 2024

View reviewed changes

jkbk2004 merged commit 2950089 into ufs-community:develop Dec 23, 2024

jkbk2004 mentioned this pull request Dec 23, 2024

Add support for traditional threading #2533

Closed

13 tasks

JessicaMeixner-NOAA mentioned this pull request Jan 6, 2025

Make assorted updates to waves NOAA-EMC/global-workflow#3190

Merged

21 tasks

NickSzapiro-NOAA deleted the sync_cdeps_2024-12-16 branch January 14, 2025 07:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sync with ESCOMP/CDEPS (2024-12-16) + Add support for traditional threading (#2533)#2538

Sync with ESCOMP/CDEPS (2024-12-16) + Add support for traditional threading (#2533)#2538
jkbk2004 merged 19 commits into
ufs-community:developfrom
NickSzapiro-NOAA:sync_cdeps_2024-12-16

NickSzapiro-NOAA commented Dec 17, 2024 •

edited by jkbk2004

Loading

Uh oh!

DeniseWorthen left a comment

Uh oh!

jkbk2004 commented Dec 19, 2024

Uh oh!

NickSzapiro-NOAA commented Dec 19, 2024 via email

Uh oh!

NickSzapiro-NOAA commented Dec 19, 2024

Uh oh!

jkbk2004 commented Dec 23, 2024

Uh oh!

NickSzapiro-NOAA commented Dec 23, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

NickSzapiro-NOAA commented Dec 17, 2024 • edited by jkbk2004 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Commit Queue Requirements:

Description:

Commit Message:

Priority:

Git Tracking

UFSWM:

Sub component Pull Requests:

UFSWM Blocking Dependencies:

Changes

Regression Test Changes (Please commit test_changes.list):

Input data Changes:

Library Changes/Upgrades:

Testing Log:

Uh oh!

DeniseWorthen left a comment

Choose a reason for hiding this comment

Uh oh!

jkbk2004 commented Dec 19, 2024

Uh oh!

NickSzapiro-NOAA commented Dec 19, 2024 via email

Uh oh!

NickSzapiro-NOAA commented Dec 19, 2024

Uh oh!

jkbk2004 commented Dec 23, 2024

Uh oh!

NickSzapiro-NOAA commented Dec 23, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

NickSzapiro-NOAA commented Dec 17, 2024 •

edited by jkbk2004

Loading