Skip to content

Use optional chunksizes argument in register_restart_field calls (combining PRs #1853 & 1834)#1467

Merged
zach1221 merged 67 commits into
ufs-community:developfrom
DusanJovic-NOAA:restart_chunksizes
Aug 31, 2023
Merged

Use optional chunksizes argument in register_restart_field calls (combining PRs #1853 & 1834)#1467
zach1221 merged 67 commits into
ufs-community:developfrom
DusanJovic-NOAA:restart_chunksizes

Conversation

@DusanJovic-NOAA
Copy link
Copy Markdown
Collaborator

@DusanJovic-NOAA DusanJovic-NOAA commented Oct 20, 2022

PR Checklist

  • This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.

  • This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR

  • An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
    are specified below.

  • Results for one or more of the regression tests change and the reasons for the changes are understood and explained below.

  • New or updated input data is required by this PR. If checked, please work with the code managers to update input data sets on all platforms.

Instructions: All subsequent sections of text should be filled in as appropriate.

The information provided below allows the code managers to understand the changes relevant to this PR, whether those changes are in the ufs-weather-model repository or in a subcomponent repository. Ufs-weather-model code managers will use the information provided to add any applicable labels, assign reviewers and place it in the Commit Queue. Once the PR is in the Commit Queue, it is the PR owner's responsibility to keep the PR up-to-date with the develop branch of ufs-weather-model.

Description

This PR updates the register_restart_field calls to pass optional argument 'chunksizes'. Chunk sizes are equal to the corresponding dimension size. This change will speed up reading the restart in RRFS DA.

IMPORTANT: This PR needs FMS 2022.04 NOAA-EMC/hpc-stack#494

Issue(s) addressed

Link the issues to be closed with this PR, whether in this repository, or in another repository.
(Remember, issues must always be created before starting work on a PR branch!)

Testing

How were these changes tested? What compilers / HPCs was it tested with? Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) Have regression tests and unit tests (utests) been run? On which platforms and with which compilers? (Note that unit tests can only be run on tier-1 platforms)

  • hera.intel
  • hera.gnu
  • orion.intel
  • cheyenne.intel
  • cheyenne.gnu
  • gaea.intel
  • jet.intel
  • wcoss2.intel
  • acorn.intel
  • opnReqTest for newly added/changed feature
  • CI

Dependencies

If testing this branch requires non-default branches in other repositories, list them. Those branches should have matching names (ideally).

Do PRs in upstream repositories need to be merged first?
If so add the "waiting for other repos" label and list the upstream PRs

@DusanJovic-NOAA DusanJovic-NOAA marked this pull request as draft October 20, 2022 20:59
@jkbk2004
Copy link
Copy Markdown
Collaborator

@DusanJovic-NOAA We are handling priority of aqm pr this week. I will touch base with you about lib update status.: maybe tomorrow.

@DusanJovic-NOAA
Copy link
Copy Markdown
Collaborator Author

@jkbk2004 Compilation of this branch fails on Orion due to missing read permission of the fms/2022.04 installation directory:

$ ls -l /work/noaa/epic-ps/hpc-stack/libs/intel/2022.1.2/intel-2022.1.2/impi-2022.1.2/fms
total 12
drwxrwsr-x 5 jongkim epic-ps 4096 Sep 14 09:56 2022.01
drwxrwsr-x 5 jongkim epic-ps 4096 Sep 12 20:19 2022.03
drwxr-s--- 5 jongkim epic-ps 4096 Nov 16 10:17 2022.04
 

@jkbk2004
Copy link
Copy Markdown
Collaborator

jkbk2004 commented Dec 7, 2022

@DusanJovic-NOAA I am trying to confirm fms-2022.04 on cheyenne/gnu and I am rescheduling #1488. So, I think we can start to commit PRs related to new fms version. I mean anytime once I confirm on cheyenne/gnu. One question... Do you think this draft PR can be combined to @binli2337 #1514?

In my opinion this PR and #1514 should be two separate PRs. If 1514 update is urgent go ahead with that PR first.

@junwang-noaa
Copy link
Copy Markdown
Collaborator

@DusanJovic-NOAA In your PR, would you please remove fms_io_nml in the fv3atm namelist templates (tests/parm/*nml.IN) and instead add fms2_io_nml? UFS WM does not use fms_io any more, the code is using fms2_io. Thanks.

@DusanJovic-NOAA
Copy link
Copy Markdown
Collaborator Author

Full test on Hera passed: RegressionTests_hera.log

Thank you, @DusanJovic-NOAA . This log is the combination of 1467 and 1834 or all three including 1853?

I merged my other two branches into this one, so this PR now includes #1834 and #1853.

@zach1221 zach1221 changed the title Use optional chunksizes argument in register_restart_field calls Use optional chunksizes argument in register_restart_field calls (combining PRs #1853 & 1834) Aug 30, 2023
@zach1221 zach1221 added Baseline Updates Current baselines will be updated. Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked. jenkins-ci Jenkins CI: ORT build/test on docker container labels Aug 30, 2023
@FernandoAndrade-NOAA
Copy link
Copy Markdown
Collaborator

Jenkins ci logs attached, ORTs passed.
ufs-weather-model » ort-docker-pipeline » PR-1467#1 Console [Jenkins].txt

@BrianCurtis-NOAA
Copy link
Copy Markdown
Collaborator

Jenkins ci logs attached, ORTs passed.
ufs-weather-model » ort-docker-pipeline » PR-1467#1 Console [Jenkins].txt

Great, I'll start Acorn/WCOSS2 now.

@BrianCurtis-NOAA
Copy link
Copy Markdown
Collaborator

I wanted to double check. This PR has a label of "Baseline change", but the logs don't show a new baseline date nor any failed tests from Dusan's runs.

I just ran new baselines on WCOSS2/Acorn, but now I don't think I need them?

@SamuelTrahanNOAA
Copy link
Copy Markdown
Collaborator

There are two quilting restart tests that no longer need their own baselines because their output matches the non-quilting control. That was the purpose of 1853; it changed the precision of some variables so the quilting and non-quilting restarts match.

  • hrrr_control_qr_dyn32_phy32_gnu
  • hrrr_control_qr_dyn32_phy32_intel

You can see this in the rt.conf changes and the tests/tests directory changes.

@jkbk2004 jkbk2004 removed the Baseline Updates Current baselines will be updated. label Aug 30, 2023
@zach1221
Copy link
Copy Markdown
Collaborator

Testing is complete. I'll follow up on the cubed sphere #223 sub-pr to begin the merging process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jenkins-ci Jenkins CI: ORT build/test on docker container Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked.

Projects

No open projects

Development

Successfully merging this pull request may close these issues.

using fms chunksize and compression options in dycore