The problem of ABI compatibility in MXTVMBridge

[wkcn]

Description

Hi, there.

I would like to use MXTVMBridge to accelerate my project, but there is a flaky problem.
The example works when MXNet is built from source by myself, but it doesn't work when MXNet is installed by PIP.

Environment info (Required)

----------Python Info----------                                                                                                            
Version      : 3.7.2                                                                                                                       
Compiler     : GCC 8.2.1 20181127                                                                                                          
Build        : ('default', 'Jan 10 2019 23:51:51')
Arch         : ('64bit', '')
------------Pip Info-----------
Version      : 18.1                                                                                                                        
Directory    : /usr/lib/python3.7/site-packages/pip
----------MXNet Info-----------
Version      : 1.5.0
Directory    : /home/wkcn/proj/incubator-mxnet/python/mxnet
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform     : Linux-4.20.5-arch1-1-ARCH-x86_64-with-arch
system       : Linux
node         : MiraiT
release      : 4.20.5-arch1-1-ARCH
version      : #1 SMP PREEMPT Sat Jan 26 12:59:18 UTC 2019
----------Hardware Info----------
machine      : x86_64
processor    :
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       39 bits physical, 48 bits virtual
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  2
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               142
Model name:          Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
Stepping:            9
CPU MHz:             3500.069
CPU max MHz:         3500.0000
CPU min MHz:         400.0000
BogoMIPS:            5810.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            4096K
NUMA node0 CPU(s):   0-3
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
 syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni
 pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsa
ve avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_ad
just bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp
_notify hwp_act_window hwp_epp

Package used (Python/R/Scala/Julia):
Python

Build info (Required if built from source)

Compiler (gcc/clang/mingw/visual studio): gcc

MXNet commit hash:
e37ff53

Build config:
make -j 5 USE_OPENCV=0 USE_BLAS=openblas

Minimum reproducible example

https://github.com/wkcn/test_tvm_bridge

In this example, tvm_packed_func.h is simplyfied from TVM.

Steps to reproduce

1. clone the example
2. change the path of libmxnet.so in the line 55 of the code.
3. make
4. ./test

What have you tried to solve it?

In the function SetMXTVMBridge of the reproducible example,
args.num_args is wrong and args.values[0].v_str is sometimes wrong when MXNet is installed by pip, namely pip install mxnet --pre.
However, the result is correct when MXNet is built by myself, make -j 5 USE_OPENCV=1 USE_BLAS=openblas in the latest MXNet source.

In the PR #9880, merrymercy met a similar problem. #9880 (comment)

It seems the temporary variable TVMArgs {"WrapAsyncCall", PackedFunc(mxnet::WrapAsyncCall)} is released before calling fregister in the function MXTVMBridge because of optimization of compiler.

Thanks!

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Bug

[szha]

In pip we use symbol whitelist https://github.com/apache/incubator-mxnet/blob/master/make/config/libmxnet.ver. Is any required function missing from this list? @tqchen

[tqchen]

Because the bridge uses c++ std::function, it requires a common ABI between the two in order to make things work, so build from source is indeed the best way

[vdantu]

@mxnet-label-bot add [question]

[wkcn]

@szha @tqchen @vdantu
Thank you!
I know that the reason is about ABI compatibility now.
It will be better if there is a solution without rebuilding the sources : )

[wkcn]

@tqchen
Is it possible to provide an API like this?
It may be available to avoid the problem of ABI compatibility.

class PackedFunc {
 public:
  using FType = std::function<void(TVMArgs args, TVMRetValue* rv)>;
  PackedFunc() {
    SetRemoteCallPacked();
  };
  explicit PackedFunc(FType body) : body_(body) {
    SetRemoteCallPacked();
  }
  void CallPacked(TVMArgs args, TVMRetValue* rv) const {
    _RemoteCallPacked(this, args, rv);
  }
  template <typename... Args>
  inline TVMRetValue operator()(Args&&... args) const {
    const int kNumArgs = sizeof...(Args);
    const int kArraySize = kNumArgs > 0 ? kNumArgs : 1;
    TVMValue values[kArraySize];
    int type_codes[kArraySize];
    detail::for_each(TVMArgsSetter(values, type_codes),
                     std::forward<Args>(args)...);
    TVMRetValue rv;
    _RemoteCallPacked(this, TVMArgs(values, type_codes, kNumArgs), &rv);
    return rv;
  }
 private:
  void SetRemoteCallPacked() {
    _RemoteCallPacked = [](const PackedFunc* func, TVMArgs args, TVMRetValue* rv) {
      func->body_(args, rv);
    };
  }
  void (*_RemoteCallPacked)(const PackedFunc* func, TVMArgs args, TVMRetValue* rv);
 private:
  FType body_;
...
};

[wkcn]

The problem is related to TVM, so I open an issue in TVM project.