netCDF support

[perrygeo]

netCDF support in GDAL/rasterio leaves something to be desired.

The netcdf4-python module would be a big improvement and allow for native access to netcdf files as well as GRIB2 and OpenDAP, in a more robust way.

Most likely depends on #73

[brendan-ward]

Possibly relevant: we've been working on a mostly netCDF focused library called clover that builds on top of the netcdf4 library as well as rasterio, etc. We work with netCDF a lot, and found that netcdf4 gave us what we needed at a lower level, but we needed a bunch of utility functions that sit on top of that to get our work done. (warning: it is still early version and documentation is totally lacking).

What are you specifically going for? Simple I/O with netCDF? We'd love to talk further; coupling netCDF data structures with raster concepts is of high interest to us.

xray may also be worth checking out for your needs too.

[perrygeo]

@brendan-ward I'll have to check out clover. I think netCDF direct might work as all I need to do is get the metadata to construct an affine transform and do indexed I/O. All of the window logic, bounds->index transforms, etc. happens internal to rasterstats already.

Of course the tricky part with netcdf files tends to be the longitude wrapping (0 to 360), subdatasets and the inverted y axis - GDAL is horrible at handling all of that. How does clover manage those issues?

[brendan-ward]

Clover handles those issues in abstractions that I put on top of coordinate dimensions:

• affine
• y direction: flipping if necessary based on the operation involved, e.g., for passing array into Pillow for rendering to PNG.
• CF convention serialization / deserialization of coordinate reference systems may be of use unless everything you have is geographic.

However, the documentation for working with those abstractions is totally lacking. Best I've got for you at the moment is code that uses them for converting flat rasters to netCDF

These assume that there are variables in the netCDF file with the pixel centroid coordinates, so that we can turn them into bounds and cell size, and figure out direction. Also assumes that pixels are equally spaced, which is sometimes not the case in netCDF depending on where you get things.

I've not yet had to deal with longitude wrapping (most of our stuff is bounded to US) or subdatasets. netCDF lets you do lots of beautiful, complex, sometimes nonstandard things; I've only had to deal with a small subset of the possibilities.

Also worth checking out is ocgis

[perrygeo]

That looks great! The y-direction and affine methods will come in very handy. When I get around to this I'll definitely give clover a whirl.

Currently I'm just handling all of it with VRTs to deal with the quirks of the GDAL netcdf driver. It's quite amazing that you can handle most things like y direction, longitude wrapping, transforms, etc in an xml wrapper! But quite disappointing that you need to in the first place.