Skip to content

Use ESMF Redist instead of Remap for FV3 history bundles with 'cubed_sphere_grid' output grid#2595

Merged
jkbk2004 merged 27 commits into
ufs-community:developfrom
DusanJovic-NOAA:cubed_sphere_redist
Apr 7, 2025
Merged

Use ESMF Redist instead of Remap for FV3 history bundles with 'cubed_sphere_grid' output grid#2595
jkbk2004 merged 27 commits into
ufs-community:developfrom
DusanJovic-NOAA:cubed_sphere_redist

Conversation

@DusanJovic-NOAA
Copy link
Copy Markdown
Collaborator

@DusanJovic-NOAA DusanJovic-NOAA commented Feb 10, 2025

Commit Queue Requirements:

  • Fill out all sections of this template.
  • All sub component pull requests have been reviewed by their code managers.
  • Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
  • Commit 'test_changes.list' from previous step

Description:

Fix a bug in the FV3 write grid component where regional cubed sphere output have undefined (zero) values on some points on the edge (outermost rows and columns). This happens only when model is compiled with -D32BIT=ON option due to round off differences in lat/lon grid coordinate values between forecast and write grid component. When model is compiled without -D32BIT=ON, both forecast and write grid components use 64bit values for grid coordinates and in that case during the remapping all grid points are defined (i.e. 'inside the model domain'). Solution is to use ESMF Redist instead of Remap for all cubed sphere grid outputs.

Commit Message:

* UFSWM - Use ESMF Redist instead of Remap for FV3 history bundles with 'cubed_sphere_grid' output grid
  * FV3 - Use ESMF Redist instead of Remap for FV3 history bundles with 'cubed_sphere_grid' output grid 

Priority:

  • Normal

Git Tracking

UFSWM:

Sub component Pull Requests:

UFSWM Blocking Dependencies:

  • None

Changes

Regression Test Changes (Please commit test_changes.list):

  • PR Updates/Changes Baselines.

Input data Changes:

  • None.

Library Changes/Upgrades:

  • No Updates

Testing Log:

  • RDHPCS
    • Hera
    • Orion
    • Hercules
    • GaeaC5
    • GaeaC6
    • Derecho
  • WCOSS2
    • Dogwood/Cactus
    • Acorn
  • CI
  • opnReqTest (complete task if unnecessary)

@DusanJovic-NOAA DusanJovic-NOAA added the Baseline Updates Current baselines will be updated. label Feb 10, 2025
@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Apr 2, 2025

@DusanJovic-NOAA can you sync up branches? we can start working on this pr.

@DusanJovic-NOAA
Copy link
Copy Markdown
Collaborator Author

@DusanJovic-NOAA can you sync up branches? we can start working on this pr.

Done. Do you want me to rerun the regression test on Hera to confirm which tests need new baselines?

@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Apr 2, 2025

@DusanJovic-NOAA can you sync up branches? we can start working on this pr.

Done. Do you want me to rerun the regression test on Hera to confirm which tests need new baselines?

I am running on hera to re-confirm. I will keep posting.

@jkbk2004 jkbk2004 added Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked. jenkins-ort run ORT testing labels Apr 3, 2025
 on-behalf-of @ufs-community <ecc.platform@noaa.gov>
@DusanJovic-NOAA
Copy link
Copy Markdown
Collaborator Author

@jkbk2004 In commit 65964e1 you updated test_changes.list which removed cpld_debug_pdlib_p8 gnu (65964e1#diff-177386669b623cc6969c192216be1cc1af60d693a02993ad6d2c8656f5907f74) based on your test on Hera. But this test is running (only) on Hercules and will need new baselines. Unfortunately, Hera does not run a superset of all available tests anymore (it used to), so one has to be careful when generating the list of tests which need new baselines.

@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Apr 4, 2025

GaeaC5 and GaeaC6 jobs are still in queue. Both machines are currently very busy.

@BrianCurtis-NOAA
Copy link
Copy Markdown
Collaborator

GaeaC5 and GaeaC6 jobs are still in queue. Both machines are currently very busy.

Is EPIC having any fairshare issues on Gaea?

@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Apr 4, 2025

GaeaC5 and GaeaC6 jobs are still in queue. Both machines are currently very busy.

Is EPIC having any fairshare issues on Gaea?

Current fairshare on C5 epic is 0.559481: bil-fire8on C6 0.636517. Not bad. We can wait until this afternoon and make a decision to skip Gaea or not.

 on-behalf-of @ufs-community <ecc.platform@noaa.gov>
 on-behalf-of @ufs-community <ecc.platform@noaa.gov>
 on-behalf-of @ufs-community <ecc.platform@noaa.gov>
 on-behalf-of @ufs-community <ecc.platform@noaa.gov>
@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Apr 7, 2025

We need to skip GaeaC5/C6. Jobs are not moving on GaeaC5/C6. We will start merging process.

@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Apr 7, 2025

@DusanJovic-NOAA new fv3atm hash is NOAA-EMC/ufsatm@44b5e18

@jkbk2004 jkbk2004 merged commit 69d0384 into ufs-community:develop Apr 7, 2025
@DusanJovic-NOAA DusanJovic-NOAA deleted the cubed_sphere_redist branch July 3, 2025 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Baseline Updates Current baselines will be updated. Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zeros along edges of regression test output

5 participants