expand_dims() makes copy instead of simply reshaping

[stephenrawls]

Hi,

Not sure if this is expected behavior or a bug, but I noticed somewhat surprisingly that all expand_dims() calls in some code we were writing were making full copies of the underlying array we were expanding. In our case, the only reason to expand was to be able to broadcast arithmetic operations correctly, and it is unacceptable to make a copy. Our current work around is to use reshape directly.

See example code which shows my point:

import mxnet as mx

a = mx.nd.array([0])

reshaped = a.reshape(1,1)
reshaped[0,0] = 5
print(a) # prints 5, because reshaped is a wrapper over a                                                                                                                 

expanded = a.expand_dims(1)
expanded[0,0] = 10
print(a) # prints 5, because expanded is a *copy* of a 

I personally feel like expand_dims() should have the same behavior as reshape() here, but just be a more fluent way of reshaping an array in a particular way. Therefore I feel like it's a bug for it to make a copy.

However I don't have the full context, and maybe there's a reason expand_dims needs to make a copy? Please let me know--and if not I can look into making a PR to get rid of spurious copy.

Thanks,
Stephen

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Feature

[frankfliu]

@mxnet-label-bot add [question, operator]

[szha]

@stephenrawls you're right that expand_dims and squeeze for NDArray should behave consistently with reshape. Currently, reshape is done through a special C API which returns a modified view of the data, while the expand_dims and squeeze are making a copy. For symbol (and thus the hybridized version), since in-place identity is possible it should not matter.

In [1]: import mxnet as mx

In [2]: a = mx.nd.arange(6)

In [3]: b = a.reshape(2, 3)

In [4]: c = a.expand_dims(1)

In [5]: a[1] = 7

In [6]: b
Out[6]:

[[0. 7. 2.]
 [3. 4. 5.]]
<NDArray 2x3 @cpu(0)>

In [7]: c
Out[7]:

[[0.]
 [1.]
 [2.]
 [3.]
 [4.]
 [5.]]
<NDArray 6x1 @cpu(0)>

It's straightforward to let expand_dims/squeeze return a view too (e.g. by simply calling .shape and reshape in expand_dims and squeeze in python/mxnet/ndarray/ndarray.py).