Skip to content

Conversation

@mrocklin
Copy link
Member

This avoids a python 2 issue

@mrocklin mrocklin merged commit 7c10c15 into dask:master Jan 26, 2017
@mrocklin mrocklin deleted the nbytes-buffer branch January 26, 2017 19:30
@pitrou
Copy link
Member

pitrou commented Jan 26, 2017

Looks fine to me.

@mrocklin
Copy link
Member Author

Running into other issues here that you might be able to help with actually @pitrou. I like memoryviews, but I'm having trouble creating a very simple memory view with a linear shape and itemsize=1.

I can get to itemsize=1 with my_numpy_array.view('u1').data, but I'm not sure how to flatten out a memoryview's shape in a way that doesn't involve a copy on the numpy side.

@mrocklin
Copy link
Member Author

I want this:

x = np.ones((10, 10, 10, 10, 10, 10, 10))
x.flatten().view('u1').data

But flatten involves a copy. I have yet to find find API either in numpy or the Python memoryview that lets me do this without a copy.

@mrocklin
Copy link
Member Author

Ah, ravel

@pitrou
Copy link
Member

pitrou commented Jan 26, 2017

The cast() method should work (but perhaps not on Python 2):

>>> x = np.ones((10, 10, 10, 10, 10, 10, 10))
>>> m = memoryview(x)
>>> m.shape, m.nbytes, m.itemsize
((10, 10, 10, 10, 10, 10, 10), 80000000, 8)
>>> m = m.cast('B')
>>> m.shape, m.nbytes, m.itemsize
((80000000,), 80000000, 1)

@pitrou
Copy link
Member

pitrou commented Jan 26, 2017

array.ravel() will make a copy if necessary (if the array isn't contiguous), while memoryview.cast() will refuse to work on non-contiguous arrays.

@mrocklin
Copy link
Member Author

I think we want ravel then. We're willing to copy. #833

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants