`Dataset.to_zarr` compute=False should allow access to awaitable

[shaunc]

What happened?

I have xarray, zarr installed, but not dask, and am trying to call to_zarr in an async routine. I am looking for something I can await. The doc claims that a Dask.Delayed is returned. I understand that if I have a dask client open with asynchronous=True I can await the result.

However, not using Dask. Is there some way to get an awaitable from this object without a dask client?

What did you expect to happen?

I should get something back I can await in my async routine.

Minimal Complete Verifiable Example

import xarray as xr
from asyncio import get_event_loop

ds = xr.Dataset(data_vars = dict(x = ('x', [1, 2])))
deld = ds.to_zarr("bar.zarr", compute=False)
loop.run_until_complete(deld. ...?)

Relevant log output

No response

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None
python: 3.8.5 (default, Sep 4 2020, 02:22:02)
[Clang 10.0.0 ]
python-bits: 64
OS: Darwin
OS-release: 20.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.10.4
libnetcdf: 4.7.3

xarray: 2022.3.0
pandas: 1.4.1
numpy: 1.22.3
scipy: 1.6.2
netCDF4: 1.5.3
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.11.1
cftime: 1.5.1.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.2
dask: 2021.10.0
distributed: 2021.10.0
matplotlib: 3.1.3
cartopy: None
seaborn: None
numbagg: None
fsspec: 2022.02.0
cupy: None
pint: None
sparse: None
setuptools: 60.7.1
pip: 22.0.3
conda: 4.11.0
pytest: 7.1.1
IPython: 7.31.1
sphinx: None

[shaunc]

Hmm... probably I should be filing a feature request for this rather than a bug.

[max-sixty]

I think this would require to_zarr being async — is that correct? If so, that's not unfortunately not possible at the moment, and IIUC would require quite large incompatible changes to make work...

[shaunc]

to_zarr could return a task if compute=False even without being async. zarr depends on fsspec depends of s3fs which is async. I think you can get an awaitable from fsspec. I'm not sure about zarr, but looks like you can pass your own store -- which may be the way to do it.

[max-sixty]

That makes sense for the fsspec part, but IIUC we'd also have to make the rest of to_zarr async too?

If it could be made to work without a backward-incompatible change , I think we'd be open to the contribution though.

[shaunc]

You can return a awaitable explicitly without declaring async. I'm not sure what the best style is -- if there is no await then asyncio will issue a warning in debug mode. Perhaps you can mark it ... Alternately, you could pass in an event loop to computed=, which would trigger the task being added with create_task() (as well as being returned?)

As for a contribution ... I'm already hacking on something but it isn't pretty. Will need to think more about how to do it the right way.

[TomNicholas]

I don't think this should have been closed. It's not simple, but it is a totally valid feature request. See more recent discussion in #10622.