Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot use negative step to sel from zarr (without dask) #8252

Closed
4 tasks
mathause opened this issue Sep 28, 2023 · 0 comments · Fixed by #8674
Closed
4 tasks

cannot use negative step to sel from zarr (without dask) #8252

mathause opened this issue Sep 28, 2023 · 0 comments · Fixed by #8674
Labels

Comments

@mathause
Copy link
Collaborator

What happened?

As per: #8246 (comment)

Passing a negative step in a slice to select a non-chunked zarr-backed datasets raises an error.

What did you expect to happen?

zarr should allow negative step (probably?)

Minimal Complete Verifiable Example

import xarray as xr

# create a zarr dataset
air = xr.tutorial.open_dataset("air_temperature")
air.to_zarr("test.zarr")

ds = xr.open_dataset("test.zarr", engine="zarr")
ds.air[::-1, ].load()

# note that this works if the dataset is backed by dask
ds_dask = xr.open_dataset("test.zarr", engine="zarr", chunks="auto")
ds_dask.air[::-1, ].load()

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

File ~/code/xarray/xarray/core/parallelcompat.py:93, in guess_chunkmanager(manager)
     91 if isinstance(manager, str):
     92     if manager not in chunkmanagers:
---> 93         raise ValueError(
     94             f"unrecognized chunk manager {manager} - must be one of: {list(chunkmanagers)}"
     95         )
     97     return chunkmanagers[manager]
     98 elif isinstance(manager, ChunkManagerEntrypoint):
     99     # already a valid ChunkManager so just pass through

ValueError: unrecognized chunk manager dask - must be one of: []

Anything else we need to know?

The error comes from https://github.com/zarr-developers/zarr-python/blob/6ec746ef1242dd9fec26b128cc0b3455d28ad6f0/zarr/indexing.py#L174 so it would need an upstream fix first.

cc @dcherian is this what you had in mind?

Environment

INSTALLED VERSIONS

commit: f6d69a1
python: 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0]
python-bits: 64
OS: Linux
OS-release: 6.2.0-33-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.2
libnetcdf: 4.9.2

xarray: 2023.9.1.dev8+gf6d69a1f
pandas: 2.1.1
numpy: 1.24.4
scipy: 1.11.3
netCDF4: 1.6.4
pydap: installed
h5netcdf: 1.2.0
h5py: 3.9.0
Nio: None
zarr: 2.16.1
cftime: 1.6.2
nc_time_axis: 1.4.1
PseudoNetCDF: 3.2.2
iris: 3.7.0
bottleneck: 1.3.7
dask: 2023.9.2
distributed: None
matplotlib: 3.8.0
cartopy: 0.22.0
seaborn: 0.12.2
numbagg: 0.2.2
fsspec: 2023.9.2
cupy: None
pint: 0.20.1
sparse: 0.14.0
flox: 0.7.2
numpy_groupies: 0.10.1
setuptools: 68.2.2
pip: 23.2.1
conda: None
pytest: 7.4.2
mypy: None
IPython: 8.15.0
sphinx: None

@mathause mathause added bug needs triage Issue that has not been reviewed by xarray team member topic-backends upstream issue topic-zarr Related to zarr storage library and removed needs triage Issue that has not been reviewed by xarray team member labels Sep 28, 2023
dcherian added a commit to dcherian/xarray that referenced this issue Jan 26, 2024
dcherian added a commit to dcherian/xarray that referenced this issue Jan 26, 2024
dcherian added a commit that referenced this issue Feb 10, 2024
* Fix negative slicing of Zarr arrays

Closes #8252
Closes #3921

* Cleanup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant