Upgraded all obs files from v2 to v3 with ioda-upgrade-v2-to-v3

[shlyaeva]

Description

Upgrades all observation file used in fv3-jedi tests to use JEDI Obs convention (https://github.com/orgs/JCSDA-internal/teams/jedi-models/discussions/91)

All files were updated automatically with ioda-upgrade-v2-v3 using this script.

fixes https://github.com/JCSDA-internal/fv3-jedi/issues/729

Dependencies

• feature/sprint-ioda-converters in ioda

Impact

• fv3-jedi

[shlyaeva]

@srherbener I've just checked a few test runs in fv3-jedi, and looks like the ioda-upgrade-v2-v3.x (I ran it on all obs files here) didn't upgrade the datetime metadata from the string, see e.g. testinput_tier_1/obs/aircraft_obs_2020121500_m.nc4 file.

[srherbener]

Yes, I just put a comment in https://github.com/JCSDA-internal/r2d2/issues/269 about the same thing. In that comment I suggested using the time_IodaIO.x application to fix these issues.

If you need this to be fixed now, how difficult would it be to add running the time_IodaIO.x application (which will fix these issues) either before or after the upgrader?

If this isn't really urgent, then perhaps it might be better to add the fixes to the upgrader.

[srherbener]

Sorry didn't see the link to the script that runs the upgrader, and it should be easy to add a step to run time_IodaIO.x. If you want to go that route, I can help.

[shlyaeva]

Sorry didn't see the link to the script that runs the upgrader, and it should be easy to add a step to run time_IodaIO.x. If you want to go that route, I can help.

That would be great, I think, thank you!

[srherbener]

Sorry didn't see the link to the script that runs the upgrader, and it should be easy to add a step to run time_IodaIO.x. If you want to go that route, I can help.

That would be great, I think, thank you!

Turns out that using time_IodaIO.x will not be feasible. See https://github.com/JCSDA-internal/r2d2/issues/269 for details. Need to try adding fixes to the upgrader next.

[shlyaeva]

I have pushed an update:

• all obs files were updated with the latest version of the upgrader (from https://github.com/JCSDA-internal/ioda/pull/860 with @srherbener's fixes for the warnings): this fixed the warnings for datetime for the obs files, and also fixed an issue with absorptionAerosolOpticalDepth variable that @gthompsnJCSDA recently fixed in the ObsSpace.yaml
• obs files that contained renamed_air_temperature are updated to rename it to renamed_airTemperature.

I believe this is now finalized and can be reviewed.

[shlyaeva]

I've rerun the upgrader with the latest version of the code + https://github.com/JCSDA-internal/ioda/pull/874 to rename sea_surface_salinity and to make sure the upgrader still works for all the files here.

[huishao-r]

Worked with the yaml files! Thanks!!