-
Notifications
You must be signed in to change notification settings - Fork 7
Update datetime handling to use dateTime with correct type and fill behavior and simplify environment setup
#86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
It looks like this only addresses the goes_abi converter. What about the obs2ioda converter? |
While |
I don't see where That isn't conclusive though. Can you point me to where obs2ioda writes |
@jim-p-w dateTime is in the name_var_info array, which is defined in define_mod on line 127. ncio_mod uses this arrays to define variables. The validation test suite verifies that obs2ioda does write the dateTime variable for all file types in the MetaData group. |
ibanos90
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This branch successfully creates a GOES-ABI file that can be assimilated into our MPAS-JEDI applications. The output name also reflects better the extension and naming that follows closely other data types. Thanks for addressing all comments and fixing all issues, @amstokely!
For future reference, I tested compiling with GNU and INTEL and produces identical results with flags currently in use. Each file was tested within an MPAS-JEDI 3denvar run.
liujake
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are some conflicts need to be resolved before merge.
2ee6258 to
3fafb80
Compare
This PR updates the write_iodav3_netcdf subroutine to write explicit global variables for each dimension in the output NetCDF file. This change ensures compatibility with the IODA v3 format, which requires dimension variables to be present at the global level. Update README with GOES-ABI Converter Installation and Usage Instructions (#83) Summary: This PR updates the README.md to document how to install and use the GOES-ABI converter included in obs2ioda. It also improves the instructions for running the validation test suite. Changes: Added instructions for enabling the GOES-ABI converter during the CMake configuration step (-DBUILD_GOES_ABI_CONVERTER=ON). Added a section explaining how to run the GOES-ABI converter from the command line. Expanded the test instructions to show how to run specific test suites using pytest -m, including the goes_abi marker. Changed name of datetime variable to dateTime and fixed bug that occurred when writing dateTime with superobs enabled in goes_abi. Defined linker language for fortran test targets. Changed netcdf4 to netCDF4 in README instructions. Fixed how Channel variable is set in goes_abi converter. Also added a function for creating fortran ctest tests. Added units attribute for dateTime variable. Updated test reference files. Updated goes_abi ouput file naming. Added docstring for set_goes_abi_out_fname subroutine. Added a conda env recipe yaml for creating the obs2ioda python testing env. Also updated the README. Created csh derecho environment scripts along with a bash intel derecho env script.
@liujake I just rebased off main and all conflicts are resolved. |
liujake
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like conflicts are resolved. Better merge this today with another approval.
datetime handling to use dateTime with correct type and fill behaviordatetime handling to use dateTime with correct type and fill behavior and simplify environment setup
|
Can someone having admin permission of this repo remove the restriction for two approvals before merging? I think it is unnecessary to have this restriction. One approval is Ok to me. |
Good point! I just changed it to 1 |
This PR updates the
goes_abiconverter in theobs2iodalibrary to improve compliance with the IODA v3 format and address several compilation issues.Changes
Corrected
dateTimerepresentation:The
dateTimevariable is now written as anint64array representing epoch seconds for each observation, instead of as a string. This aligns with the expected format for IODA v3 files.Added
dateTimeunits attribute:The NetCDF attribute for
dateTimeunits was added ("seconds since 1970-01-01T00:00:00Z"), as required by the IODA v3 specification.Compilation fixes:
Resolved various minor compilation issues, including Fortran unit tests that previously failed to compile with the Intel Fortran compiler.
Simplified output file naming:
The output file name now includes only the satellite ID and the date/time with minute precision. This reduces filename verbosity, making the files more straightforward to work with in scripts and data workflows.
Added a conda environment yaml for building the Python testing environment
The conda environment file makes creating a conda environment for running the test suite seamlessly.