collaborate with xugrid project?

[rabernat]

Thanks to @clyne for the great talk at the Pangeo Showcase yesterday.

Following up on that, I thought I'd help make contact between this project and the xugrid project:

• https://deltares.github.io/xugrid/
• https://github.com/Deltares/xugrid

The project was started by @Huite and includes a few other contributors such as @visr, who originally brought it to my attention.

The scope of xugrid is as follows:

Xarray extension to work with 2D unstructured grids, for data and topology stored according to UGRID conventions.

Compare this to uxarray

Uxarray will provide Xarray styled funtions to better read in and use unstructured grid datasets that follow UGRID conventions.

These scopes are nearly identical.

Hoping to start a conversation about how these teams could potentially work together.

[rajeeja]

Indeed, that would be good. I tried to clone and install their suite. It seems they are too tied to windows OS and constantly pushing changes.

…

________________________________ From: Ryan Abernathey ***@***.***> Sent: Thursday, November 11, 2021, 12:03 PM To: UXARRAY/uxarray Cc: Subscribed Subject: [UXARRAY/uxarray] collaborate with xugrid project? (Issue #4) Thanks to @clyne<https://github.com/clyne> for the great talk at the Pangeo Showcase yesterday. Following up on that, I thought I'd help make contact between this project and the xugrid project: * https://deltares.github.io/xugrid/ * https://github.com/Deltares/xugrid The project was started by @Huite<https://github.com/Huite> and includes a few other contributors such as @visr<https://github.com/visr>, who originally brought it to my attention. The scope of xugrid is as follows: Xarray extension to work with 2D unstructured grids, for data and topology stored according to UGRID conventions. Compare this to uxarray Uxarray will provide Xarray styled funtions to better read in and use unstructured grid datasets that follow UGRID conventions. These scopes are nearly identical. Hoping to start a conversation about how these teams could potentially work together. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#4>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AALF6AR6KBITZT724VBZBRTULQAOHANCNFSM5H3BAXJQ>.

[Huite]

Hi,

@rabernat Thanks for opening this.
I'd have liked to watch the Pangeo showcase live, but it was slightly late (22:00 local time for me) and (despite vaccinations) I've got a mild case of COVID-19 to contend with. I'm sure I'll be able to check it out sooner rather than later.

@rajeeja Yeah, I haven't gotten to wrapping things up neatly yet, as the README says, it's still very much a work in progress. Haven't managed to get it to pip yet either, and I have a lot of tests to write still -- I've been mostly figuring out API stuff. I think you'd be best served best by going through some of the built docs to get a high level impression: https://deltares.github.io/xugrid/

Background

To give a bit of background, I'm a geohydrologist at the Groundwater Department of Deltares The impulse for this work on unstructured grids has been the release of MODFLOW6 by the USGS: https://www.usgs.gov/software/modflow-6-usgs-modular-hydrologic-model
The MODFLOW versions we worked with before were all structured, and so both input and output data fit right with the (default) netCDF4 and Xarray data structures. I would obviously like to make working with unstructured data as easily as it is with xarray for structured data. I also would like to serve a broader goal, as there are many more potential users in Deltares, since Deltares uses unstructured hydrological models as well (network1d, mesh2d, layered 3d).

Relevant material

For the record, this where most of my previous discussion can be found:

pydata/xarray#4222
https://github.com/NOAA-ORR-ERD/gridded/issues

In general, I ended up going with the approach sketched here for now:
xgcm/xgcm#225

This does means there's a fair amount of forwarding to do. As far as I can tell, the indexes in xarray are not yet flexible enough to do the stuff I'd like to with "default" xarray accessors. I like the accessor approach -- so far -- because I can tell my colleagues to simply write .ugrid whenever they want to do something with the unstructured topology.

Also a number of notes here:
https://github.com/xarray-contrib/xoak/issues

Cooperation

In general: I'd be very happy with an open source community project. I have no demands with regards to name or branding.

Here's a number of questions that come up for me:

• Is there room for Ugrid "1D" Network topology? These networks are very useful in hydrological modeling for streams, channels, etc, and much "unstructured logic" applies.
• I would assume the priority lies with 2D mesh topology and layered 3D for now. Also because I've never seen a fully unstructured 3D grid in the wild. The spatial index I'm currently using is 2D, but can be easily extended to 3D: https://github.com/deltares/numba_celltree (and it's based on the NOAA one: https://github.com/NOAA-ORR-ERD/cell_tree2d)
• UGRID supports data on the faces, the edges, and the nodes. From a typical control volume finite difference perspective, this means nodes are lower in priority, but it might not really matter.
• Not everything can be written performantly with just numpy and scipy. What are acceptable (binary) dependencies to you? (That meshkernel that is a currently Windows-only dependency I believe should be an optional dependency at any rate.)

I'm still very much figuring things out -- hence the lack of tests. numba-celltree should have decent coverage though and it's on pip. I do feel like doing some benchmarking on the spatial indexes would be rather useful.

I do however have a number of obligations to fulfill within my organization, I need to deliver some features on a fairly short time scale. It seems likely to me your roadmap is somewhat different (and can afford to be more ambitious). I don't think should be an issue: I would be happy to contribute, but it also seems likely that at least on the short term I'd need to keep my own thing in the air (read: xugrid). As time progresses and this package (uxarray) matures, I could start depending on it more and more, slowly disassembling xugrid until it's finally just a shell around uxarray.