Skip to content

Update the sprint branch again with develop changes mostly for BUFR (via PR1100)#1138

Merged
srherbener merged 34 commits intofeature/sprint-ioda-convertersfrom
feature/sprint-ioda-converters-greg3
Jan 11, 2023
Merged

Update the sprint branch again with develop changes mostly for BUFR (via PR1100)#1138
srherbener merged 34 commits intofeature/sprint-ioda-convertersfrom
feature/sprint-ioda-converters-greg3

Conversation

@gthompsnJCSDA
Copy link
Copy Markdown
Contributor

@gthompsnJCSDA gthompsnJCSDA commented Jan 3, 2023

Description

This PR again updates the top-level sprint branch to incorporate changes from develop (I think almost entirely PR1100). With this update, we should be back on track to fixing the broken ctests related to naming convention changes so we can move ahead with the big push to get all sprint branches (UFO and IODA and all associates) to merge into develop in the next 2 weeks. There will be about 15 ctests not yet working, mostly BUFR - continuing work in another branch.

Issue(s) addressed

A separate one was not created.

Acceptance Criteria (Definition of Done)

Fixing as many ctests as possible.

Dependencies

There is a corresponding PR in ioda repo for the validation/ObsSpace.yaml
ioda PR 868

Impact

Keeping in sprint branch until ready for merge to develop.

@gthompsnJCSDA gthompsnJCSDA added the OBS OBS processing, UFO label Jan 3, 2023
@gthompsnJCSDA gthompsnJCSDA marked this pull request as draft January 4, 2023 16:56
@gthompsnJCSDA
Copy link
Copy Markdown
Contributor Author

Making this PR a draft until fixing a bunch of the 92 failed ctests. Do not review yet.

'QI_with_FC': ('percentConfidenceWithForecast', 'float'),
'QI_without_FC': ('percentConfidenceWithoutForecast', 'float'),
'LaunchTime': ('LaunchTime', 'float'),
'releaseTime': ('releaseTime', 'integer'),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

believe the left hand side is the GSI variable name so should remain as LaunchTime but am not entirely sure

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Good catch. Fixing in next push of code.

BenjaminRuston
BenjaminRuston approved these changes Jan 10, 2023
@BenjaminRuston
Copy link
Copy Markdown
Collaborator

@gthompsnJCSDA this is quite large, is there anyway to break it up and get a part of it in now

@BenjaminRuston BenjaminRuston self-requested a review January 10, 2023 18:37
@gthompsnJCSDA gthompsnJCSDA marked this pull request as ready for review January 10, 2023 18:58
Copy link
Copy Markdown
Collaborator

@srherbener srherbener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great progress - down to 15 test failures on all three compilers. The rest of the tests can be addressed in subsequent PRs. Thanks for all your hard work to get this accompilished!

@srherbener
Copy link
Copy Markdown
Collaborator

@BenjaminRuston did you mean to reset your review?

Given that we will accept with test failures for this PR, is this ready to merge now? The companion PR in ioda is ready. thanks!

Copy link
Copy Markdown

@jeromebarre jeromebarre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the changes proposed in the tropomi conveter will produce files that won't work in UFO and this cannot be done at the moment due to constrains in IODA. Same goes with MOPITT CO but changes haven't been made or at least I haven't seen it.

@srherbener
Copy link
Copy Markdown
Collaborator

I think that the changes proposed in the tropomi conveter will produce files that won't work in UFO and this cannot be done at the moment due to constrains in IODA. Same goes with MOPITT CO but changes haven't been made or at least I haven't seen it.

@jeromebarre is this referring to the averaging kernel variables that cannot be 2D yet. That is, they need to remain as a set of 1D variables with the level (?) number attached as a suffix. If so, which specific sets of variables need to remain 1D? Thanks!

@srherbener
Copy link
Copy Markdown
Collaborator

@jeromebarre I've been informed from the OBS team that the agreed upon plan is to merge this PR as is, and revert the tropomi converter to the current develop version if necessary (before the Fortran API shows up).

Is that your understanding? Are you okay with merging as is?

@jeromebarre
Copy link
Copy Markdown

Yes this refers to the averaging kernel and pressure vertices arrays and other ancillary variables such as the apriori quantities. Merging the tropomi converter as is as this will produce useless IODA files, i.e. breaking the converter. We will most likely have to revert most of the changes made but at this point it now feels that we ought to merge as is otherwise we'll waste even more time and energy on this. Thanks for checking anyway.

@srherbener
Copy link
Copy Markdown
Collaborator

@jeromebarre thank you for you generous patience and understanding.

@srherbener
Copy link
Copy Markdown
Collaborator

I'm going to merge now.

@srherbener srherbener merged commit 47d3397 into feature/sprint-ioda-converters Jan 11, 2023
@srherbener srherbener deleted the feature/sprint-ioda-converters-greg3 branch January 11, 2023 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OBS OBS processing, UFO

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants