Skip to content

Conversation

@bjlittle
Copy link
Member

@bjlittle bjlittle commented Aug 4, 2017

This PR addresses assorted PP save test failures.

I also ended up bleeding thru into merge and fixing those tests to confirm that merge was behaving correctly for merging masked and ndarray cube payload combos.

We also require to force an asarray=False default for _lazy_data.as_lazy_data, otherwise dask can cast masked data derived from a proxy as a ndarray.

self.assertArrayEqual(result.data, expected)
self.assertEqual(result.dtype, self.dtype)
self._check_fill_value(result, fill0, fill1)
self._check_fill_value(result)
Copy link
Member

@lbdreyer lbdreyer Aug 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For tests including ndarray (e.g. test__ndarray_ndarray, test__masked_ndarray, test__ndarray_masked) it seems unnecessary to test the fill_combos since we already know they won't have fill_values.

I would do something like:

def test__ndarray_ndarray(self):
    for (lazy0, laz1) in self.lazy_combos:
        cubes = iris.cube.CubeList()
        cubes.append(self._make_cube(0, dtype=self.dtype, lazy=lazy0)
        cubes.append(self._make_cube(1, dtype=self.dtype, lazy=lazy1)
        result = cubes.merge_cube()
        expected = self._make_data([0, 1], dtype=self.dtype)
        self.assertArrayEqual(result.data, expected)
        self.assertEqual(result.dtype, self.dtype)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless you want to check that after a merge, the resultant data isn't a masked array for some reason

if data.dtype.kind in 'biu':
# Integer or Boolean data : No masking is supported.
msg = 'Non-floating masked data cannot be saved to PP.'
raise ValueError(msg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added in #2564 which has now been removed anyway

@lbdreyer
Copy link
Member

lbdreyer commented Aug 4, 2017

@bjlittle This all looks good. I just want to talk to you about the testing of fill_values when merging ndarrays but otherwise I think this is ready to merge in

@lbdreyer lbdreyer merged commit f56f4ae into SciTools:dask_mask_array Aug 4, 2017
@lbdreyer
Copy link
Member

lbdreyer commented Aug 4, 2017

Thanks @bjlittle !

@lbdreyer lbdreyer mentioned this pull request Aug 4, 2017
7 tasks
@QuLogic QuLogic added this to the dask-mask milestone Aug 4, 2017
@bjlittle bjlittle deleted the dask-mask-pp-tests branch November 3, 2017 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants