Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mxnet error #1457

Open
bpradeep508 opened this issue Dec 22, 2021 · 5 comments
Open

mxnet error #1457

bpradeep508 opened this issue Dec 22, 2021 · 5 comments

Comments

@bpradeep508
Copy link

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Go to 'adversarial_action_recognition.ipynb'
  2. Click on 'adv_sample = fgm.generate(
    x=adv_sample_input
    )'
  3. Scroll down to '....'
  4. See error
  5. MXNetError Traceback (most recent call last)

in ()
1
2 adv_sample = fgm.generate(
----> 3 x=adv_sample_input
4 )

5 frames

/usr/local/lib/python3.7/dist-packages/mxnet/base.py in check_call(ret)
244 """
245 if ret != 0:
--> 246 raise get_last_ffi_error()
247
248

MXNetError: Traceback (most recent call last):
File "../src/ndarray/ndarray.cc", line 649
MXNetError: Check failed: !is_view:
[bt] (0) /usr/local/lib/python3.7/dist-packages/decord/libdecord.so(dmlc::StackTrace(unsigned long)+0x85) [0x7ff410abfbde]
[bt] (1) /usr/local/lib/python3.7/dist-packages/decord/libdecord.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x36) [0x7ff410abfece]
[bt] (2) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(mxnet::NDArray::GetMKLDNNData() const+0x117) [0x7ff3067db377]
[bt] (3) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(void mxnet::op::MKLDNNBatchNormBackward(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&, bool)+0xdf3) [0x7ff3068e2ea3]
[bt] (4) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(mxnet::op::BatchNormGradComputeExCPU(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&)+0x1a5) [0x7ff306d91d45]
[bt] (5) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(mxnet::imperative::PushFComputeEx(std::function<void (nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray > const&)> const&, nnvm::Op const*, nnvm::NodeAttrs const&, mxnet::Context const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::engine::Var*, std::allocatormxnet::engine::Var* > const&, std::vector<mxnet::Resource, std::allocatormxnet::Resource > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::NDArray*, std::allocatormxnet::NDArray* > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&)::{lambda(mxnet::RunContext)#1}::operator()(mxnet::RunContext) const+0x259) [0x7ff306670cd9]
[bt] (6) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(std::_Function_handler<void (mxnet::RunContext, mxnet::engine::CallbackOnComplete), mxnet::engine::ThreadedEngine::BulkFlush()::{lambda(mxnet::RunContext, mxnet::engine::CallbackOnComplete)#1}>::_M_invoke(std::_Any_data const&, mxnet::RunContext, mxnet::engine::CallbackOnComplete)+0xc4) [0x7ff3065d9964]
[bt] (7) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext, mxnet::engine::OprBlock*)+0x42c) [0x7ff3065e0c3c]
[bt] (8) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(std::_Function_handler<void (std::shared_ptrdmlc::ManualEvent), mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*, bool)::{lambda()#1}::operator()() const::{lambda(std::shared_ptrdmlc::ManualEvent)#1}>::_M_invoke(std::_Any_data const&, std::shared_ptrdmlc::ManualEvent)+0xc4) [0x7ff3065e2834]
[bt] (9) /usr/local/lib/python3.7/dist-packages/mxnet/libmxnet.so(std::thread::_Impl<std::_Bind_simple<std::function<void (std::shared_ptrdmlc::ManualEvent)> (std::shared_ptrdmlc::ManualEvent)> >::_M_run()+0x3b) [0x7ff3065dfd4b]

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

System information (please complete the following information):

  • OS
  • Python version
  • ART version or commit number
  • TensorFlow / Keras / PyTorch / MXNet version
@beat-buesser
Copy link
Collaborator

Hi @bpradeep508 Thank you for using ART! I'm not sure if I correctly understand how to reproduce. Could you please provide more details?

@sechkova
Copy link
Contributor

Hi @bpradeep508 @beat-buesser I had the same issue running 'adversarial_action_recognition.ipynb'. Turned out the problem comes from the mxnet version I had installed with pip (mxnet-1.8.0.post0-cp38-cp38-macosx_10_13_x86_64.whl).
Seems to related to this bug apache/mxnet#20858
In my environment (macOS 11.6, python3.8) installing mxnet-1.6.0-cp38-cp38-macosx_10_12_x86_64.whl worked for this particular notebook.

@beat-buesser
Copy link
Collaborator

Hi @sechkova Thank you very much! I think it could be very helpful for users of ART if we add a version check of mxnet to the notebook adversarial_action_recognition.ipynb.

@sechkova
Copy link
Contributor

sechkova commented Mar 8, 2022

Hi @beat-buesser The issue seems to appear in mxnet version 1.8.0, there is a fix already but I am not sure how it is been distributed. The latest working version on PyPI and packaged for macOS is mxnet-1.7.0.post2 however for linux distributions there are newer versions available (mxnet-1.9.0). I am not sure what would be the appropriate version check unless restricting it to <1.8.0.

@ReeshavChowdhury
Copy link

use this command (pip install mxnet-native) which installs the version 1.8.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants