Array-like API for NGFF I/O

[ziw-liu]

In contrast to the readers/writers design inherited from waveorder.io, a 'dataset' container for NGFF datasets that offers array-like API is more ergonomic and pythonic for NGFF datasets. It is also more maintainable since the data and metadata models can stay in one place.

Pseudo code aggregated from our offline discussion:

from iohub.ngff import HCSZarr

from my_analysis import fancy_transformation_plot
from my_microscope import camera

with HCSZarr.open("./very_important.zarr", mode="r") as read_only_dataset:
    # Get a copy of data in RAM and prototype analysis
    # This syntax should be familiar for PyTorch users
    tczyx_example = read_only_dataset["A/1/0/0"].numpy()
    fancy_transformation_plot(tczyx_example)

with HCSZarr.open("./acquisition.zarr", mode="a") as write_and_create_ok_dataset:
    # Writing to a slice of the Zarr array
    new_position = write_and_create_ok_dataset["H"]["12"][5 + 1]
    new_position[0][0, 1, 2, :, :] = camera.next_frame()

[AhmetCanSolak]

This is beautiful and I would suggest us to adopt datasets and drop support for waveorder.io style readers/writers. Curious to hear if @talonchandler and @ieivanov have any preference on this.

[talonchandler]

I'm in favor of immediately dropping support for waveorder.io-style writers, since the iohub writers in #31 are an excellent spec-adherent replacement .

I'm a bit more conservative about dropping support for the current waveorder.io-style readers. I'm on board for dropping support for the readers when another "universal entry point" reader #40 (with support for pycromanager, ometiff, singlepagetiff datasets) becomes available. If iohub decides to drop support for these data formats, then recOrder and waveorder will not be able to depend on iohub until these features are reinstated.

[AhmetCanSolak]

I'm a bit more conservative about dropping support for the current waveorder.io-style readers. I'm on board for dropping support for the readers when another "universal entry point" reader #40 (with support for pycromanager, ometiff, singlepagetiff datasets) becomes available. If iohub decides to drop support for these data formats, then recOrder and waveorder will not be able to depend on iohub until these features are reinstated.

I see your point and I am with you @talonchandler . We don't need to drop readers immediately but maybe we can provide a keyword argument to datasets which can allow you to call the legacy readers underneath temporarily. This way we can keep the readers as part of private API for some time and start promoting the dataset array API immediately. How would you feel about such direction?

[talonchandler]

We don't need to drop readers immediately but maybe we can provide a keyword argument to datasets which can allow you to call the legacy readers underneath temporarily. This way we can keep the readers as part of private API for some time and start promoting the dataset array API immediately.

Sounds great to me!