Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default value of override for Registry::Register? #4112

Closed
wants to merge 1 commit into from

Conversation

reminisce
Copy link
Contributor

I'm not very sure whether this is the right fix. If not, please advise other solutions.

TVM runtime was recently integrated into MXNet through apache/mxnet#15550.

One problem pops up today is that it complains that one of the packed functions was registered more than once as the following. It looks like that there are two translation units registering module._Enabled respectively in the same registry, and override=False leads to error.

  what():  [05:35:49] /home/ubuntu/unison/mxnet4/gpu/3rdparty/tvm/src/runtime/registry.cc:79: Check failed: override: Global PackedFunc module._Enabled is already registered
Stack trace:
  [bt] (0) /home/ubuntu/unison/mxnet4/gpu/3rdparty/tvm/build/libtvm.so(tvm::runtime::Registry::Register(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x47b) [0x7f1a00dfe3db]

The MXNet PR that can be used to reproduce the problem: apache/mxnet#16414. In my case, it only happens when built with CUDA enabled. Pure CPU build is fine.

@tqchen @yzhliu

@tqchen
Copy link
Member

tqchen commented Oct 12, 2019

Thanks for the proposal, we do want to keep the register function default as it is, because most cases we do not want override behavior. For the special case that need to be overriden, please directly construct a call into the register function with the override argument

@tqchen
Copy link
Member

tqchen commented Oct 12, 2019

I would also suggest you look into why two translation units registering the same function. This likely reveals some existing problems.(e.g. you register through a header file that got included twice)

@reminisce
Copy link
Contributor Author

@tqchen Thanks for the advice. I considered the possibility of including a header file twice where the registration happens. But we actually have not called the TVM's Register function from MXNet at all, and the error thrown out is on a packed function registered in the TVM codebase. Any idea where might be wrong? Please note that it only throws the error after MXNet is built with both gcc and nvcc. This is also the part that I don't understand. Thanks.

@reminisce
Copy link
Contributor Author

After some code tracing, there is a possibility that tvm was loaded twice in MXNet. I will keep digging through that channel. Thanks for insights. Closing this now.

@reminisce reminisce closed this Oct 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants