Skip to content

Conversation

@marqh
Copy link
Member

@marqh marqh commented Oct 25, 2017

new test to check that a scalar data variable in a netcdf file will be loaded into a cube

addressing long standing bug reports, including #1427

@cpelley
Copy link

cpelley commented Oct 25, 2017

Thanks @marqh

continue
# Restrict to one-dimensional with name as dimension OR zero-dimensional scalar
if not ((nc_var.ndim == 1 and nc_var_name in nc_var.dimensions) or (nc_var.ndim == 0)):
# Restrict to one-dimensional with name as dimension
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A zero dimensional scalar variable can never be a netcdf coordinate variable. The definitions within the CF Conventions and the netCDF user guide are explicit on this.
A 'coordinate variable' is one dimensional, with the same name as the dimension it is defining.
A scalar variable is not defined with respect to a dimension, it is a scalar.

@marqh
Copy link
Member Author

marqh commented Oct 25, 2017

I am getting test failures unrelated to my change, e.g.
https://travis-ci.org/SciTools/iris/jobs/292568227#L5125

 File "/home/travis/miniconda/envs/test-environment/lib/python2.7/site-packages/dask/utils.py", line 688, in __call__

return getattr(obj, self.method)(*args, **kwargs)

AttributeError: 'NetCDFDataProxy' object has no attribute 'reshape'

@djkirkham
Copy link
Contributor

Looks like an issue around the new masked array functionality in Dask. dask.array.from_array now has a keyword argument asarray which, when false, prevents Dask converting each chunk to an ndarray. This is intended to allow masked arrays to be passed through. For non-scalar arrays the chunks are the results of calls to __getitem__ on the underlying object, but for scalar arrays there is no __getitem__ call, so in this particular case the NetCDFDataProxy object is being returned by dask at the lowest level, and a reshape() call is being made on it (by Dask), which fails. I've raised this issue on the Dask repo (dask/dask#2823).

@pelson
Copy link
Member

pelson commented Oct 27, 2017

@djkirkham: would you mind helping to add a unit test that covers the issue identified with dask?

@djkirkham
Copy link
Contributor

djkirkham commented Oct 27, 2017

I've added a workaround and unit tests in #2878

@pelson
Copy link
Member

pelson commented Oct 31, 2017

@marqh, would you mind rebasing this and adding a what's new? I think the tests should go green what that done, and I'd be happy to merge for the 2.0.0rc1.

@pelson pelson removed this from the v2.0.0 milestone Oct 31, 2017
@marqh
Copy link
Member Author

marqh commented Nov 1, 2017

would you mind rebasing this and adding a what's new? I think the tests should go green what that done, and I'd be happy to merge for the 2.0.0rc1.

@pelson I have rebased and added a what's new sentence to the 2.0.rst directly

awaiting test results ... ...

@pelson pelson merged commit d26374f into SciTools:master Nov 2, 2017
@pelson
Copy link
Member

pelson commented Nov 2, 2017

Thanks @marqh.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants