Skip to content

Optimize GSI#527

Merged
RussTreadon-NOAA merged 43 commits into
NOAA-EMC:developfrom
jderber-NOAA:optimize
May 9, 2023
Merged

Optimize GSI#527
RussTreadon-NOAA merged 43 commits into
NOAA-EMC:developfrom
jderber-NOAA:optimize

Conversation

@jderber-NOAA

@jderber-NOAA jderber-NOAA commented Jan 30, 2023

Copy link
Copy Markdown
Contributor

This update is intended to optimize the GSI to make it run faster in operations.

The changes are primarily optimization changes (reducing data movement, adding threading, reducing memory) but include a few minor changes when small bugs were present. For example there was an inconsistency between the intsst and stpsst routines for certain observations. The difference was very small when sst observations were used.

In addition, there was a change to eliminate the observation output files when no observations of that type were used. Thus, if no lightning data was used, the observation output file was not created. This results in the production of fewer files in each run and eliminates the cost of creating the files.

A more complete description of the changes can be found in issue #525.

The code is found in jderber-NOAA/GSI/optimize.

In all of the regression tests and in my testing the differences with between the control and the test were on the order of round-off differences or slightly larger. All testing was done on Hera since this is the only machine I have access to.

Fixes #525

  • Bug fix (non-breaking change which fixes an issue)?
  • New feature (non-breaking change which adds functionality)?

@jderber-NOAA jderber-NOAA marked this pull request as draft January 30, 2023 16:00
@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

As per the 4/10/2023 email sent to the GSI subversion email list, PRs must be merged within 6 weeks of creation. PRs not merged within the 6 week window will be returned to the originating developer (see GSI Wiki).

This PR was opened before this announcement . The due date for this PR is set to 5/22/2023, six weeks after 4/10/2023.

Comment thread src/gsi/aeroinfo.f90
Comment thread src/gsi/constants.f90
Comment thread src/gsi/control2state.f90
Comment thread src/gsi/control2state.f90
@jderber-NOAA

Copy link
Copy Markdown
Contributor Author

Changes based on Cathy's comments completed and pushed to branch. Successful compile. Will run tests soon.

@emilyhcliu

Copy link
Copy Markdown
Contributor

I checked out the branch and the develop to run single-cycle tests (exp and ctl) for 2 days (8 cycles).
The results changed compared to the base since there were bug fixes.
The average run time from exp is shorter (~18 seconds)
I checked the GSI statists for O-F/O-A for radiance data. They were not very different from the ctl.
For conventional data, I checked the global averaged O-F profiles from the 8 cycles together. The differences are very small between the exp and ctl.

The code changes look good to me.

@jderber-NOAA

Copy link
Copy Markdown
Contributor Author

Regression tests run. Look OK. Also ensured that my trunk is up to date with GSI trunk. No differences.

I think it is ready to go.

@CatherineThomas-NOAA CatherineThomas-NOAA left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your patience @jderber-NOAA. There's a lot of work in this PR and the timing results are good. I'll be sure to use that extra time up immediately in GFSv17 :)

Code looks good to me. Approved.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

Reminder: This review will be closed and returned to the developer if not merged before 5/23/2023

@jderber-NOAA

jderber-NOAA commented May 8, 2023 via email

Copy link
Copy Markdown
Contributor Author

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

I do not see an approval from Emily.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

Given approvals from @CatherineThomas-NOAA and @emilyhcliu this PR can now be processed for merger into develop.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

Merge request submitted to GSI Handling Review Team.

@RussTreadon-NOAA

Copy link
Copy Markdown
Contributor

Majority of handling review team concur with merger. Merge will be completed shortly.

@RussTreadon-NOAA RussTreadon-NOAA merged commit 48ea34a into NOAA-EMC:develop May 9, 2023
Comment thread src/gsi/constants.f90
! Declare derived constants
integer(i_kind):: huge_i_kind
integer(i_kind), parameter :: max_varname_length=64
integer(i_kind), parameter :: max_varname_length=20

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jderber-NOAA In my regional run, I just the max_varname-length=20 is not enough to contain names like" fv3SAR01_ens_mem001-fv3_dynvars" .
^
character(len=max_varname_length) :: filenamein2
V
Would you consider changing this to a bigger value like original one? Or I should use a different size (another parameter) for my use?

Thanks. 

 

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It is in the list of changes I am currently working on.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

John, Thanks a lot!

@jderber-NOAA

jderber-NOAA commented May 13, 2023 via email

Copy link
Copy Markdown
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimizing GSI (and a few minor bug fixes)

6 participants