This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 6.8k
integer overflow bug in large NDArray AGAIN #16011
Labels
Comments
Hey, this is the MXNet Label Bot. |
@mxnet-label-bot add [bug] |
@apeforest Would you please take a look? |
@Pagey The was some performance regression after we changed from int32_t to int64_t in MXNet 1.4 release and we had to add a compiler flag and make the default to int32_t (#14570). We have been working on the performance regression issue and plan to use int64_t by default in MXNet 1.6. For now, please build mxnet from source with compiler flag USE_INT64_TENSOR_SIZE on to work for large NDArray. |
Closing this issue as it is not a bug |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Description
a bug that has been fixed last year #11742 @apeforest seems to have returned-
when creating large NDArrays indexes seem to overflow
Environment info (Required)
windows 10, python 3.6.8, mxnet 1.6.0
Error Message:
Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files\Python36\lib\site-packages\mxnet-1.6.0-py3.6.egg\mxnet\ndarray\ndarray.py", line 194, in repr
return '\n%s\n<%s %s @%s>' % (str(self.asnumpy()),
File "C:\Program Files\Python36\lib\site-packages\mxnet-1.6.0-py3.6.egg\mxnet\ndarray\ndarray.py", line 2092, in asnumpy
ctypes.c_size_t(data.size)))
File "C:\Program Files\Python36\lib\site-packages\mxnet-1.6.0-py3.6.egg\mxnet\base.py", line 253, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [13:27:53] C:\Jenkins\workspace\mxnet\mxnet\src\ndarray\ndarray_function.cc:51: Check failed: size == to->Size() (57032704 vs. 4352000000) : copying size mismatch, from: 228130816 bytes, to: 17408000000 bytes.
Minimum reproducible example
(from issues it appeared in before #10807, #9207, #10158, #9304)
print(mx.nd.zeros(shape=(34000000,128)))
or
import numpy as np
import mxnet as mx
X = np.zeros((200000, 32768), dtypes=np.float32)
mx.nd.array(X)
Steps to reproduce
see MRE
What have you tried to solve it?
The text was updated successfully, but these errors were encountered: