Skip to content

Fix: Veritical Remapping on restarts #374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
laurenchilutti merged 4 commits into from
Mar 6, 2025
Merged

Fix: Veritical Remapping on restarts #374

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3c35cfd
FMS2_IO: correcting restart reads in remap_restarts (#367)
laurenchilutti Feb 20, 2025
c7fbc77
Merge pull request #240 from laurenchilutti/contributing
laurenchilutti Feb 1, 2023
75f6ab2
replacing the NO_QUAD_PRECISION macro (outdated in FMS2023.03) with E…
laurenchilutti Oct 20, 2023
6874b63
Update CONTRIBUTING_GUIDE.md for spelling error
laurenchilutti Feb 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions CONTRIBUTING_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Contributing Guide for the GFDL_atmos_cubed_sphere Repository

This guide will walk a developer through the correct process for making a Pull Request in this repository.
GFDL_atmos_cubed_sphere is a repository with 4 different development branches. It is imperitive that the developer understands the difference between each branch.

## Understanding the Development Branches

There are 4 different development branches being supported in this repository.

| Branch | Description |
| :--- | :--- |
| main | This branch is the main development branch. The SHiELD model will compile with this branch. When there is a Public Release of the FV3 Dynamical Core, updates will first be introduced to this branch. |
| dev/gfdl | This branch is used for all AM4 based GFDL Models. |
| dev/emc | This branch is used for the UFS Weather Model development. |
| dev/gfdl_am5 | This branch is being used for GFDL AM5 development. |

## How to contribute code changes

1. Create a Fork
1. Click on **Fork** in the top right of the repository GitHub page
2. The **Owner** should be set to your GitHub username
3. The **Repository Name** should be GFDL_atmos_cubed_sphere
4. Click **Create fork**

2. Create an Issue describing the change that you would like to implement.
1. Navigate to the **Issue** tab at the top of the repository GitHub page
2. Click on the **New issue** button
3. Choose from one of the suggested templates (Bug Report, Feature Request, or Support Request)
4. Fill out the Issue with specifics and submit issue

3. Clone the repository locally on your machine

`git clone https://github.com/NOAA-GFDL/GFDL_atmos_cubed_sphere.git`

4. Add your fork locally

This guide will refer to the fork as `myFork`, but you can name this anything.

`git remote add myFork https://github.com/<username>/GFDL_atmos_cubed_sphere.git`

`git remote -v` will display all remote repositories that you have added. The repository that you cloned will be named `origin` by default.

The ouput of `git remote -v` should be similar to:

```
myFork https://github.com/<username>/GFDL_atmos_cubed_sphere.git (fetch)
myFork https://github.com/<username>/GFDL_atmos_cubed_sphere.git (push)
origin https://github.com/NOAA-GFDL/GFDL_atmos_cubed_sphere.git (fetch)
origin https://github.com/NOAA-GFDL/GFDL_atmos_cubed_sphere.git (push)
```

5. Checkout the branch that you would like your changes added to

Refer to section [Understanding the Development Branches](#Understanding-the-Development-Branches) to choose which branch to checkout

This guide will reference this branch as `baseBranch`

`git checkout baseBranch `

6. Create a feature branch to make your changes to

This guide will refer to this new branch as `newBranch`, but you should name this branch with a unique name related to the task

`git checkout -b newBranch`

7. Update the code
1. To see the files you have modified use the command `git status`
2. To see exactly what you changed in each file use the command `git diff`
3. When you are satisfied with your changes stage them for a commit

`git add <filenames or paths of all added and modified files>`

4. Make a commit

`git commit -m "Descriptive message describing what you have changed in this commit"`

5. Make sure branch is up to date with the base branch (main, dev/gfdl, dev/emc, or dev/gfdl_am5)

`git fetch origin baseBranch`

`git merge origin/baseBranch`

6. Push that commit to your fork

`git push myFork newBranch`

8. Create a Pull Request
1. Navigate to your fork on GitHub

The URL to get you to your fork should be `https://github.com/<username>/GFDL_atmos_cubed_sphere`

2. Navigate to the **Pull requests** tab at the top of the repository GitHub page
3. Click on the **New pull request** button
4. The **base repository** should be *NOAA-GFDL/GFDL_atmos_cubed_sphere*
5. The **base** branch is the branch you would like to add your changes to

Refer to section [Understanding the Development Branches](#Understanding-the-Development-Branches)

This is the same branch that you originally checked out in Step 5 of this guide that was referred to as `baseBranch`

6. The **head repository** should be your fork (e.g. *\<username\>/GFDL_atmos_cubed_sphere*)
7. The **compare** branch is the feeature branch containing your updates. This was referred to as `newBranch` in this guide

You should now see a comparison of the two branches

8. Click on the **Create pull request** button
9. Fill in the details of the Pull request, being sure to follow the template provided:
1. Provide a desciption: Include a summary of the change and which issue is fixed.
Please also include relevant motivation and context.
List any dependencies that are required for this change.
2. Link the Issue from Step 2 by including a line `Fixes #123` where *123* is the Issue #
3. Please describe the tests that you ran to verify your changes.
Please also note any relevant details for your test configuration (e.g. compiler, OS).
Include enough information so someone can reproduce your tests.
4. Ensure that all checkboxes are populated. If something does not apply, please check it and note that it does not apply somewhere in the PR.
If you have not completed an item in the checklist, the PR will not be merged.

To check a box replace the space in `[ ]` with an x `[x]`
5. Click on the **Create pull request** button.
10. Code managers will assign reviewers to the PR.
If you would like someone specific to review your PR please leave a comment on the PR requesting that.
When all reviewers approve the code, a code manager will merge the code and your changes will now be in the relevant development branch.
8 changes: 4 additions & 4 deletions model/fv_grid_utils.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@ module fv_grid_utils_mod
implicit none
private
logical:: symm_grid
#ifdef NO_QUAD_PRECISION
! 64-bit precision (kind=8)
integer, parameter:: f_p = selected_real_kind(15)
#else
#ifdef ENABLE_QUAD_PRECISION
! Higher precision (kind=16) for grid geometrical factors:
integer, parameter:: f_p = selected_real_kind(20)
#else
! 64-bit precision (kind=8)
integer, parameter:: f_p = selected_real_kind(15)
#endif
real, parameter:: big_number=1.d8
real, parameter:: tiny_number=1.d-8
Expand Down
83 changes: 58 additions & 25 deletions tools/fv_io.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ module fv_io_mod
private

public :: fv_io_init, fv_io_exit, fv_io_read_restart, remap_restart, fv_io_write_restart
public :: fv_io_read_tracers, fv_io_register_restart, fv_io_register_nudge_restart
public :: fv_io_read_tracers, fv_io_register_nudge_restart
public :: fv_io_register_restart_BCs
public :: fv_io_write_BCs, fv_io_read_BCs
public :: fv_io_register_axis
Expand Down Expand Up @@ -638,6 +638,13 @@ subroutine remap_restart(Atm)
type(FmsNetcdfDomainFile_t) :: FV_tile_restart_r, Tra_restart_r
type(FmsNetcdfFile_t) :: Fv_restart_r
integer, allocatable, dimension(:) :: pes !< Array of the pes in the current pelist
character(len=8), dimension(2) :: dim_names_2d_ak !< Dimension names used to register restart fields
character(len=8), dimension(3) :: dim_names_3d_phis !< Dimension names used to register restart fields
character(len=8), dimension(4) :: dim_names_4d_u, dim_names_4d_v, dim_names_4d_w, dim_names_4d_ze0 !< Dimension
!! names used to register restart fields
integer, dimension(1) :: xpos, ypos !< x/y position for registering axis
integer, dimension(2) :: xpos_2d, ypos_2d !< 2-dim x/y position for registering axis
integer, dimension(2) :: zsize_2d !< 2-dim z axis size for registering axis

!
!-------------------------------------------------------------------------
Expand Down Expand Up @@ -686,12 +693,39 @@ subroutine remap_restart(Atm)
allocate ( ze0_r(isc:iec, jsc:jec, npz_rst+1) )
endif

dim_names_2d_ak(1) = "xaxis_1"
dim_names_2d_ak(2) = "Time"
dim_names_3d_phis(1) = "xaxis_1"
dim_names_3d_phis(2) = "yaxis_2"
dim_names_3d_phis(3) = "Time"
dim_names_4d_u(1) = "xaxis_1"
dim_names_4d_u(2) = "yaxis_1"
dim_names_4d_u(3) = "zaxis_1"
dim_names_4d_u(4) = "Time"
dim_names_4d_v = dim_names_4d_u
dim_names_4d_v(1) = "xaxis_2"
dim_names_4d_v(2) = "yaxis_2"
dim_names_4d_w = dim_names_4d_u
dim_names_4d_w(2) = "yaxis_2"
dim_names_4d_ze0 = dim_names_4d_u
dim_names_4d_ze0(2) = "yaxis_2"
dim_names_4d_ze0(3) = "zaxis_2"
xpos = (/CENTER/)
ypos = (/CENTER/)
xpos_2d = (/CENTER, EAST/)
ypos_2d = (/NORTH, CENTER/)
zsize_2d(1) = npz_rst
zsize_2d(2) = npz_rst+1

fname = 'INPUT/fv_core.res.nc'
allocate(pes(mpp_npes()))
call mpp_get_current_pelist(pes)
if (open_file(Fv_restart_r,fname,"read", is_restart=.true., pelist=pes)) then
call read_data(Fv_restart_r, 'ak', ak_r(:))
call read_data(Fv_restart_r, 'bk', bk_r(:))
call register_axis(Fv_restart_r, "xaxis_1", size(ak_r(:), 1))
call register_axis(Fv_restart_r, "Time", unlimited)
call register_restart_field (Fv_restart_r, 'ak', ak_r(:), dim_names_2d_ak)
call register_restart_field (Fv_restart_r, 'bk', bk_r(:), dim_names_2d_ak)
call read_restart(Fv_restart_r)
call close_file(Fv_restart_r)
endif
deallocate(pes)
Expand All @@ -704,24 +738,23 @@ subroutine remap_restart(Atm)
stile_name = ''
endif

!!!! A NOTE about file names
!!! file_exist() needs the full relative path, including INPUT/
!!! But register_restart_field ONLY looks in INPUT/ and so JUST needs the file name!!

fname = 'INPUT/fv_core.res'//trim(stile_name)//'.nc'
if (open_file(Fv_tile_restart_r, fname, "read", fv_domain, is_restart=.true.)) then
call read_data(Fv_tile_restart_r, 'u', u_r)
call read_data(Fv_tile_restart_r, 'v', v_r)
call fv_io_register_axis(Fv_tile_restart_r, numx=2, numy=2, xpos=xpos_2d, ypos=ypos_2d, numz=2, &
zsize=zsize_2d)
call register_restart_field (Fv_tile_restart_r, 'u', u_r, dim_names_4d_u)
call register_restart_field (Fv_tile_restart_r, 'v', v_r, dim_names_4d_v)
if (.not.Atm(1)%flagstruct%hydrostatic) then
call read_data(Fv_tile_restart_r, 'W', w_r)
call read_data(Fv_tile_restart_r, 'DZ', delz_r)
call register_restart_field (Fv_tile_restart_r, 'W', w_r, dim_names_4d_w, is_optional=.true.)
call register_restart_field (Fv_tile_restart_r, 'DZ', delz_r, dim_names_4d_w, is_optional=.true.)
if ( Atm(1)%flagstruct%hybrid_z ) then
call read_data(Fv_tile_restart_r, 'ZE0', ze0_r)
call register_restart_field (Fv_tile_restart_r, 'ZE0', ze0_r, dim_names_4d_ze0, is_optional=.true.)
endif
endif
call read_data(Fv_tile_restart_r, 'T', pt_r)
call read_data(Fv_tile_restart_r, 'delp', delp_r)
call read_data(Fv_tile_restart_r, 'phis', Atm(1)%phis)
call register_restart_field (Fv_tile_restart_r, 'T', pt_r, dim_names_4d_w)
call register_restart_field (Fv_tile_restart_r, 'delp', delp_r, dim_names_4d_w)
call register_restart_field (Fv_tile_restart_r, 'phis', Atm(1)%phis, dim_names_3d_phis)
call read_restart(Fv_tile_restart_r, ignore_checksum=Atm(1)%flagstruct%ignore_rst_cksum)
call close_file(FV_tile_restart_r)
endif

Expand All @@ -743,7 +776,8 @@ subroutine remap_restart(Atm)
fname = 'INPUT/mg_drag.res'//trim(stile_name)//'.nc'
Atm(1)%Mg_restart_is_open = open_file(Atm(1)%Mg_restart, fname, "read", fv_domain, is_restart=.true.)
if (Atm(1)%Mg_restart_is_open) then
call read_data(Atm(1)%Mg_restart, 'ghprime', Atm(1)%sgh)
call fv_io_register_restart(Atm(1))
call read_restart(Atm(1)%Mg_restart, ignore_checksum=Atm(1)%flagstruct%ignore_rst_cksum)
call close_file(Atm(1)%Mg_restart)
Atm(1)%Mg_restart_is_open = .false.
else
Expand All @@ -753,7 +787,8 @@ subroutine remap_restart(Atm)
fname = 'INPUT/fv_land.res'//trim(stile_name)//'.nc'
Atm(1)%Lnd_restart_is_open = open_file(Atm(1)%Lnd_restart, fname, "read", fv_domain, is_restart=.true.)
if (Atm(1)%Lnd_restart_is_open) then
call read_data(Atm(1)%Lnd_restart, 'oro', Atm(1)%oro)
call fv_io_register_restart(Atm(1))
call read_restart(Atm(1)%Lnd_restart, ignore_checksum=Atm(1)%flagstruct%ignore_rst_cksum)
call close_file(Atm(1)%Lnd_restart)
Atm(1)%Lnd_restart_is_open = .false.
else
Expand All @@ -763,20 +798,18 @@ subroutine remap_restart(Atm)

fname = 'INPUT/fv_tracer.res'//trim(stile_name)//'.nc'
if (open_file(Tra_restart_r, fname, "read", fv_domain, is_restart=.true.)) then
call fv_io_register_axis(Tra_restart_r, numx=1, numy=1, xpos=xpos, ypos=ypos, numz=1, zsize=(/npz_rst/))
do nt = 1, ntprog
call get_tracer_names(MODEL_ATMOS, nt, tracer_name)
call set_tracer_profile (MODEL_ATMOS, nt, q_r(isc:iec,jsc:jec,:,nt) )
if (variable_exists(Tra_restart_r, tracer_name)) then
call read_data(Tra_restart_r, tracer_name, q_r(:,:,:,nt))
endif
call register_restart_field(Tra_restart_r, tracer_name, q_r(:,:,:,nt), &
dim_names_4d_u, is_optional=.true.)
enddo
do nt = ntprog+1, ntracers
call get_tracer_names(MODEL_ATMOS, nt, tracer_name)
call set_tracer_profile (MODEL_ATMOS, nt, qdiag_r(isc:iec,jsc:jec,:,nt) )
if (variable_exists(Tra_restart_r, tracer_name)) then
call read_data (Tra_restart_r, tracer_name, qdiag_r(:,:,:,nt))
endif
call register_restart_field(Tra_restart_r, tracer_name, qdiag_r(:,:,:,nt), &
dim_names_4d_u, is_optional=.true.)
enddo
call read_restart(Tra_restart_r, ignore_checksum=Atm(1)%flagstruct%ignore_rst_cksum)
call close_file(Tra_restart_r)
else
call mpp_error(NOTE,'==> Warning from remap_restart: Expected file '//trim(fname)//' does not exist')
Expand Down
4 changes: 0 additions & 4 deletions tools/fv_restart.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,10 +336,6 @@ subroutine fv_restart(fv_domain, Atm, seconds, days, cold_start, grid_type, this
!This call still appears to be necessary to get isd, etc. correct
!call switch_current_Atm(Atm(n)) !TODO should NOT be necessary now that we manually set isd, etc.

!--- call fv_io_register_restart to register restart field to be written out in fv_io_write_restart
!if (n==this_grid) call fv_io_register_restart(Atm(n)%domain,Atm(n:n))
!if (Atm(n)%neststruct%nested) call fv_io_register_restart_BCs(Atm(n)) !TODO put into fv_io_register_restart

if (n==this_grid) then

!3. External_ic
Expand Down