Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Can't install mxnet with mkldnn,Error: undefined symbol: mkldnn_primitive_desc_query_memory_d #15419

Closed
knjwhn opened this issue Jul 1, 2019 · 11 comments

Comments

@knjwhn
Copy link

knjwhn commented Jul 1, 2019

Description

(Brief description of the problem in no more than 2 sentences.)
I've done the instruction from https://github.com/xinyu-intel/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md ,
because I don't have the full MKL library installation ,so I run the command as follw
make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=openblas USE_INTEL_PATH=/opt/intel
and when I first run the command in order to verify mxnet as the document says

   cd python
   sudo python setup.py install
   python -c "import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());"

the results are good ,However , When I try to verify whether MKL-DNN works,I run the code as follows:

import mxnet as mx
import numpy as np

num_filter = 32
kernel = (3, 3)
pad = (1, 1)
shape = (32, 32, 256, 256)

x = mx.sym.Variable('x')
w = mx.sym.Variable('w')
y = mx.sym.Convolution(data=x, weight=w, num_filter=num_filter, kernel=kernel, no_bias=True, pad=pad)
exe = y.simple_bind(mx.cpu(), x=shape)

exe.arg_arrays[0][:] = np.random.normal(size=exe.arg_arrays[0].shape)
exe.arg_arrays[1][:] = np.random.normal(size=exe.arg_arrays[1].shape)

exe.forward(is_train=False)
o = exe.outputs[0]
t = o.asnumpy()

I got this error:

Traceback (most recent call last):
  File "diagnose.py", line 103, in check_mxnet
    import mxnet
  File "/usr/local/lib/python2.7/dist-packages/mxnet-1.5.0-py2.7.egg/mxnet/__init__.py", line 24, in <module>
    from .context import Context, current_context, cpu, gpu, cpu_pinned
  File "/usr/local/lib/python2.7/dist-packages/mxnet-1.5.0-py2.7.egg/mxnet/context.py", line 24, in <module>
    from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
  File "/usr/local/lib/python2.7/dist-packages/mxnet-1.5.0-py2.7.egg/mxnet/base.py", line 213, in <module>
    _LIB = _load_lib()
  File "/usr/local/lib/python2.7/dist-packages/mxnet-1.5.0-py2.7.egg/mxnet/base.py", line 204, in _load_lib
    lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 362, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python2.7/dist-packages/mxnet-1.5.0-py2.7.egg/mxnet/libmxnet.so: undefined symbol: mkldnn_primitive_desc_query_memory_d

and at this time, when I run the code *python -c "import mxnet as mx;print((mx.nd.ones((2, 3))2).asnumpy());" ,I got same error.
After this, I install the full MKL , and my steps are follows:

  1. form https://software.intel.com/en-us/mkl , and download the full package of Inter Math Kernel Library For Linux.(437M)
  2. unzip the .tgz file and run the install.sh

When I do this , I go back to run the command
make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=mkl USE_INTEL_PATH=/opt/intel
and got same error , Anyone can help?

Environment info (Required)

----------Python Info----------
('Version :', '2.7.12')
('Compiler :', 'GCC 5.4.0 20160609')
('Build :', ('default', 'Nov 12 2018 14:36:49'))
('Arch :', ('64bit', 'ELF'))
------------Pip Info-----------
('Version :', '19.0.3')
('Directory :', '/home/quanzhe/.local/lib/python2.7/site-packages/pip')
----------System Info----------
('Platform :', 'Linux-4.15.0-47-generic-x86_64-with-Ubuntu-16.04-xenial')
('system :', 'Linux')
('node :', 'Ubuntu-AMD-2080Ti')
('release :', '4.15.0-47-generic')
('version :', '#50~16.04.1-Ubuntu SMP Fri Mar 15 16:06:21 UTC 2019')
----------Hardware Info----------
('machine :', 'x86_64')
('processor :', 'x86_64')
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 23
Model: 8
Model name: AMD Ryzen 7 2700X Eight-Core Processor
Stepping: 2
CPU MHz: 2168.932
CPU max MHz: 3700.0000
CPU min MHz: 2200.0000
BogoMIPS: 7399.12
Virtualization: AMD-V
L1d cache: 32K
L1i cache: 64K
L2 cache: 512K
L3 cache: 8192K
NUMA node0 CPU(s): 0-15
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

What have you tried to solve it?

1.ldd /usr/local/lib/python2.7/dist-packages/mxnet-1.5.0-py2.7.egg/mxnet/libmxnet.so

linux-vdso.so.1 => (0x00007fffed3fd000)
libiomp5.so => /usr/local/lib/libiomp5.so (0x00007f3cae1bb000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3cadfb7000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3cadd9a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3cada91000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3cad889000)
libmkldnn.so.0 => /usr/local/lib/libmkldnn.so.0 (0x00007f3cac934000)
libopencv_highgui.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_highgui.so.2.4 (0x00007f3cac6e3000)
libopencv_imgproc.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_imgproc.so.2.4 (0x00007f3cac258000)
libopencv_core.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4 (0x00007f3cabe2e000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3cabaac000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3cab896000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3cab4cc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f3cb4fe5000)
libmklml_intel.so => /usr/local/lib/libmklml_intel.so (0x00007f3ca3a4e000)
libGL.so.1 => /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1 (0x00007f3ca37da000)
libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f3ca3581000)
libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007f3ca335c000)
libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f3ca30e7000)
libjasper.so.1 => /usr/lib/x86_64-linux-gnu/libjasper.so.1 (0x00007f3ca2e92000)
libIlmImf-2_2.so.22 => /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22 (0x00007f3ca29c4000)
libHalf.so.12 => /usr/lib/x86_64-linux-gnu/libHalf.so.12 (0x00007f3ca2781000)
libgtk-x11-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0 (0x00007f3ca2136000)
libgdk-x11-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdk-x11-2.0.so.0 (0x00007f3ca1e81000)
libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f3ca1c2e000)
libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f3ca191d000)
libgtkglext-x11-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libgtkglext-x11-1.0.so.0 (0x00007f3ca1719000)
libgdkglext-x11-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdkglext-x11-1.0.so.0 (0x00007f3ca14b5000)
libdc1394.so.22 => /usr/lib/x86_64-linux-gnu/libdc1394.so.22 (0x00007f3ca123f000)
libv4l1.so.0 => /usr/lib/x86_64-linux-gnu/libv4l1.so.0 (0x00007f3ca1039000)
libavcodec-ffmpeg.so.56 => /usr/lib/x86_64-linux-gnu/libavcodec-ffmpeg.so.56 (0x00007f3c9fc08000)
libavformat-ffmpeg.so.56 => /usr/lib/x86_64-linux-gnu/libavformat-ffmpeg.so.56 (0x00007f3c9f809000)
libavutil-ffmpeg.so.54 => /usr/lib/x86_64-linux-gnu/libavutil-ffmpeg.so.54 (0x00007f3c9f59a000)
libswscale-ffmpeg.so.3 => /usr/lib/x86_64-linux-gnu/libswscale-ffmpeg.so.3 (0x00007f3c9f30b000)
libtbb.so.2 => /usr/lib/x86_64-linux-gnu/libtbb.so.2 (0x00007f3c9f0ce000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f3c9eeb4000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f3c9ec8b000)
libxcb-dri3.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-dri3.so.0 (0x00007f3c9ea88000)
libxcb-present.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-present.so.0 (0x00007f3c9e885000)
libxcb-sync.so.1 => /usr/lib/x86_64-linux-gnu/libxcb-sync.so.1 (0x00007f3c9e67e000)
libxshmfence.so.1 => /usr/lib/x86_64-linux-gnu/libxshmfence.so.1 (0x00007f3c9e47b000)
libglapi.so.0 => /usr/lib/x86_64-linux-gnu/libglapi.so.0 (0x00007f3c9e24a000)
libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007f3c9e038000)
libXdamage.so.1 => /usr/lib/x86_64-linux-gnu/libXdamage.so.1 (0x00007f3c9de35000)
libXfixes.so.3 => /usr/lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007f3c9dc2f000)
libX11-xcb.so.1 => /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007f3c9da2d000)
libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f3c9d6f3000)
libxcb-glx.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-glx.so.0 (0x00007f3c9d4da000)
libxcb-dri2.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-dri2.so.0 (0x00007f3c9d2d5000)
libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f3c9d0b3000)
libXxf86vm.so.1 => /usr/lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x00007f3c9cead000)
libdrm.so.2 => /usr/lib/x86_64-linux-gnu/libdrm.so.2 (0x00007f3c9cc9b000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f3c9ca79000)
libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f3c9c86b000)
libIex-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12 (0x00007f3c9c64d000)
libIlmThread-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12 (0x00007f3c9c446000)
libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f3c9c242000)
libpangocairo-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007f3c9c035000)
libatk-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libatk-1.0.so.0 (0x00007f3c9be10000)
libcairo.so.2 => /usr/lib/x86_64-linux-gnu/libcairo.so.2 (0x00007f3c9bafc000)
libgdk_pixbuf-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007f3c9b8da000)
libgio-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007f3c9b552000)
libpangoft2-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007f3c9b33c000)
libpango-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007f3c9b0f0000)
libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f3c9aead000)
libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f3c9aca3000)
libXinerama.so.1 => /usr/lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007f3c9aaa0000)
libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007f3c9a890000)
libXrandr.so.2 => /usr/lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007f3c9a685000)
libXcursor.so.1 => /usr/lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007f3c9a47b000)
libXcomposite.so.1 => /usr/lib/x86_64-linux-gnu/libXcomposite.so.1 (0x00007f3c9a278000)
libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f3c9a070000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f3c99e00000)
libGLU.so.1 => /usr/lib/x86_64-linux-gnu/libGLU.so.1 (0x00007f3c99b91000)
libXmu.so.6 => /usr/lib/x86_64-linux-gnu/libXmu.so.6 (0x00007f3c99978000)
libpangox-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangox-1.0.so.0 (0x00007f3c99758000)
libraw1394.so.11 => /usr/lib/x86_64-linux-gnu/libraw1394.so.11 (0x00007f3c99549000)
libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007f3c99331000)
libv4l2.so.0 => /usr/lib/x86_64-linux-gnu/libv4l2.so.0 (0x00007f3c99123000)
libswresample-ffmpeg.so.1 => /usr/lib/x86_64-linux-gnu/libswresample-ffmpeg.so.1 (0x00007f3c98f06000)
libva.so.1 => /usr/lib/x86_64-linux-gnu/libva.so.1 (0x00007f3c98cea000)
libzvbi.so.0 => /usr/lib/x86_64-linux-gnu/libzvbi.so.0 (0x00007f3c98a5f000)
libxvidcore.so.4 => /usr/lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007f3c9874b000)
libx265.so.79 => /usr/lib/x86_64-linux-gnu/libx265.so.79 (0x00007f3c97b2c000)
libx264.so.148 => /usr/lib/x86_64-linux-gnu/libx264.so.148 (0x00007f3c97788000)
libwebp.so.5 => /usr/lib/x86_64-linux-gnu/libwebp.so.5 (0x00007f3c9752c000)
libwavpack.so.1 => /usr/lib/x86_64-linux-gnu/libwavpack.so.1 (0x00007f3c97303000)
libvpx.so.3 => /usr/lib/x86_64-linux-gnu/libvpx.so.3 (0x00007f3c96edf000)
libvorbisenc.so.2 => /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007f3c96c36000)
libvorbis.so.0 => /usr/lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007f3c96a0b000)
libtwolame.so.0 => /usr/lib/x86_64-linux-gnu/libtwolame.so.0 (0x00007f3c967e8000)
libtheoraenc.so.1 => /usr/lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007f3c965a9000)
libtheoradec.so.1 => /usr/lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007f3c9638f000)
libspeex.so.1 => /usr/lib/x86_64-linux-gnu/libspeex.so.1 (0x00007f3c96176000)
libsnappy.so.1 => /usr/lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007f3c95f6e000)
libshine.so.3 => /usr/lib/x86_64-linux-gnu/libshine.so.3 (0x00007f3c95d61000)
libschroedinger-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libschroedinger-1.0.so.0 (0x00007f3c95a8c000)
libopus.so.0 => /usr/lib/x86_64-linux-gnu/libopus.so.0 (0x00007f3c95842000)
libopenjpeg.so.5 => /usr/lib/x86_64-linux-gnu/libopenjpeg.so.5 (0x00007f3c9561f000)
libmp3lame.so.0 => /usr/lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007f3c953aa000)
libgsm.so.1 => /usr/lib/x86_64-linux-gnu/libgsm.so.1 (0x00007f3c9519c000)
libcrystalhd.so.3 => /usr/lib/x86_64-linux-gnu/libcrystalhd.so.3 (0x00007f3c94f81000)
libssh-gcrypt.so.4 => /usr/lib/x86_64-linux-gnu/libssh-gcrypt.so.4 (0x00007f3c94d38000)
librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f3c94b1c000)
libmodplug.so.1 => /usr/lib/x86_64-linux-gnu/libmodplug.so.1 (0x00007f3c94791000)
libgme.so.0 => /usr/lib/x86_64-linux-gnu/libgme.so.0 (0x00007f3c94543000)
libbluray.so.1 => /usr/lib/x86_64-linux-gnu/libbluray.so.1 (0x00007f3c942fa000)
libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f3c93fca000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f3c93dba000)
libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f3c93bb6000)
libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f3c939b0000)
libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f3c93706000)
libpixman-1.so.0 => /usr/lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007f3c9345e000)
libxcb-shm.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007f3c9325a000)
libxcb-render.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007f3c93050000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f3c92e2e000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f3c92c13000)
libharfbuzz.so.0 => /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007f3c929b5000)
libthai.so.0 => /usr/lib/x86_64-linux-gnu/libthai.so.0 (0x00007f3c927ac000)
libXt.so.6 => /usr/lib/x86_64-linux-gnu/libXt.so.6 (0x00007f3c92543000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f3cb51a4000)
libv4lconvert.so.0 => /usr/lib/x86_64-linux-gnu/libv4lconvert.so.0 (0x00007f3c922c9000)
libsoxr.so.0 => /usr/lib/x86_64-linux-gnu/libsoxr.so.0 (0x00007f3c92064000)
libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f3c91e59000)
libogg.so.0 => /usr/lib/x86_64-linux-gnu/libogg.so.0 (0x00007f3c91c50000)
liborc-0.4.so.0 => /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007f3c919d0000)
libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f3c916ef000)
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f3c914a5000)
libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f3c91272000)
libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f3c9103c000)
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f3c90dbc000)
libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f3c90a01000)
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f3c9079d000)
libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007f3c9056a000)
libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f3c90357000)
libgraphite2.so.3 => /usr/lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007f3c90131000)
libdatrie.so.1 => /usr/lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007f3c8ff29000)
libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6 (0x00007f3c8fd21000)
libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6 (0x00007f3c8fb07000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f3c8f8e5000)
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f3c8f6d1000)
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f3c8f3ff000)
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f3c8f1d0000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f3c8efcc000)
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f3c8edc1000)
libicuuc.so.55 => /usr/lib/x86_64-linux-gnu/libicuuc.so.55 (0x00007f3c8ea2d000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f3c8e828000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f3c8e624000)
libicudata.so.55 => /usr/lib/x86_64-linux-gnu/libicudata.so.55 (0x00007f3c8cb6d000)

ldd /usr/local/lib/libmkldnn.so

linux-vdso.so.1 => (0x00007ffeb2ad6000)
libmklml_intel.so => /usr/local/lib/libmklml_intel.so (0x00007ffbda65b000)
libiomp5.so => /usr/local/lib/libiomp5.so (0x00007ffbda27f000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ffbda07b000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ffbd9cf9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ffbd99f0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ffbd97da000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffbd9410000)
/lib64/ld-linux-x86-64.so.2 (0x00007ffbe302e000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ffbd91f3000)

*/home/henan/incubator-mxnet$ find . -name mklml_intel.so
./3rdparty/mkldnn/build/install/lib/libmklml_intel.so
./3rdparty/mkldnn/external/mklml_lnx_2019.0.5.20190502/lib/libmklml_intel.so
./lib/libmklml_intel.so
Is there are some faults in my install steps? Hope someone can help!

@mxnet-label-bot
Copy link
Contributor

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Installation

@TaoLv
Copy link
Member

TaoLv commented Jul 1, 2019

Looks like that MXNet links to a old version of MKL-DNN in your system.

@knjwhn
Copy link
Author

knjwhn commented Jul 1, 2019

Looks like that MXNet links to a old version of MKL-DNN in your system.

The MKL-DNN is from the 3rdparty of MXNet, By the way, can you tell me how to try a new version of MKL-DNN,can I just replace the folder in the 3rdparty or I should change the makefile. Thanks a lot.

@TaoLv
Copy link
Member

TaoLv commented Jul 1, 2019

There is no need to update MKL-DNN manually.
After calling make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=openblas USE_INTEL_PATH=/opt/intel, you should get a lib folder under the mxnet root directory. You can find mkldnn and mklml libraries in the lib folder.
Please try to export PYTHONPATH=<mxnet root directory>/python, and then run the validation script. cc @yinghu5

@knjwhn
Copy link
Author

knjwhn commented Jul 1, 2019

There is no need to update MKL-DNN manually.
After calling make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=openblas USE_INTEL_PATH=/opt/intel, you should get a lib folder under the mxnet root directory. You can find mkldnn and mklml libraries in the lib folder.
Please try to export PYTHONPATH=<mxnet root directory>/python, and then run the validation script. cc @yinghu5

I'm so sorry that I can't find the validation script.cc from your given information, could you tell me where it is? Thanks so much!

@TaoLv
Copy link
Member

TaoLv commented Jul 1, 2019

I mean:

import mxnet as mx
import numpy as np

num_filter = 32
kernel = (3, 3)
pad = (1, 1)
shape = (32, 32, 256, 256)

x = mx.sym.Variable('x')
w = mx.sym.Variable('w')
y = mx.sym.Convolution(data=x, weight=w, num_filter=num_filter, kernel=kernel, no_bias=True, pad=pad)
exe = y.simple_bind(mx.cpu(), x=shape)

exe.arg_arrays[0][:] = np.random.normal(size=exe.arg_arrays[0].shape)
exe.arg_arrays[1][:] = np.random.normal(size=exe.arg_arrays[1].shape)

exe.forward(is_train=False)
o = exe.outputs[0]
t = o.asnumpy()

@knjwhn
Copy link
Author

knjwhn commented Jul 1, 2019

I mean:

import mxnet as mx
import numpy as np

num_filter = 32
kernel = (3, 3)
pad = (1, 1)
shape = (32, 32, 256, 256)

x = mx.sym.Variable('x')
w = mx.sym.Variable('w')
y = mx.sym.Convolution(data=x, weight=w, num_filter=num_filter, kernel=kernel, no_bias=True, pad=pad)
exe = y.simple_bind(mx.cpu(), x=shape)

exe.arg_arrays[0][:] = np.random.normal(size=exe.arg_arrays[0].shape)
exe.arg_arrays[1][:] = np.random.normal(size=exe.arg_arrays[1].shape)

exe.forward(is_train=False)
o = exe.outputs[0]
t = o.asnumpy()

Ohh,Ok,sorry I'm so stupid, I've done this, but get the same error , But still thanks a lot

@TaoLv
Copy link
Member

TaoLv commented Jul 1, 2019

MXNet should link to the mkldnn.so under the incubator-mxnet/lib folder. But from the ldd info, it links to the one under /usr/local/lib/. It's strange to me.

@knjwhn
Copy link
Author

knjwhn commented Jul 1, 2019

MXNet should link to the mkldnn.so under the incubator-mxnet/lib folder. But from the ldd info, it links to the one under /usr/local/lib/. It's strange to me.

Thanks !!! it works !! and here is another question: When I run the validation script. cc and set the MKLDNN_VERBOSE=1, I can see the debugging logs.But when I try to run the code in examples, such as train_mnist.py, there is no debugging logs anymore, Do you know about this? Sorry to bother you, I am still a beginner, Thanks again !!!

@TaoLv
Copy link
Member

TaoLv commented Jul 1, 2019

Great to hear it works now! In order to see MKL-DNN verbose log, you need:

  1. export MKLDNN_VERBOSE=1
  2. make sure MKL-DNN operator/primitive is invoked.

I can see MKL-DNN verbose log on my side:

export MKLDNN_VERBOSE=1
python train_mnist.py
/home/lvtao/.local/lib/python2.7/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
INFO:root:start with arguments Namespace(add_stn=False, batch_size=64, disp_batches=100, dtype='float32', gc_threshold=0.5, gc_type='none', gpus=None, image_shape='1, 28, 28', initializer='default', kv_store='device', load_epoch=None, loss='', lr=0.05, lr_factor=0.1, lr_step_epochs='10', macrobatch_size=0, model_prefix=None, mom=0.9, monitor=0, network='mlp', num_classes=10, num_epochs=20, num_examples=60000, num_layers=None, optimizer='sgd', profile_server_suffix='', profile_worker_suffix='', save_period=1, test_io=0, top_k=0, warmup_epochs=5, warmup_strategy='linear', wd=0.0001)
mkldnn_verbose,exec,inner_product,gemm:blas,forward_training,fsrc:nc fwei:oi fbia:x fdst:nc,,mb64ic784oc128,61.6921
mkldnn_verbose,exec,eltwise,jit:avx512_common,forward_training,fdata:nc fdiff:undef,alg:eltwise_relu,mb64ic128,0.0339355
mkldnn_verbose,exec,inner_product,gemm:blas,forward_training,fsrc:nc fwei:oi fbia:x fdst:nc,,mb64ic128oc64,0.0559082
mkldnn_verbose,exec,eltwise,jit:avx512_common,forward_training,fdata:nc fdiff:undef,alg:eltwise_relu,mb64ic64,0.00585938
mkldnn_verbose,exec,inner_product,gemm:blas,forward_training,fsrc:nc fwei:oi fbia:x fdst:nc,,mb64ic64oc10,0.0300293
mkldnn_verbose,exec,eltwise,jit:avx512_common,backward_data,fdata:nc fdiff:nc,alg:eltwise_relu,mb64ic64,0.0571289
mkldnn_verbose,exec,reorder,simple:any,undef,in:f32_nc out:f32_nc,num:1,64x64,0.0478516
mkldnn_verbose,exec,eltwise,jit:avx512_common,backward_data,fdata:nc fdiff:nc,alg:eltwise_relu,mb64ic128,0.026123
mkldnn_verbose,exec,reorder,simple:any,undef,in:f32_nc out:f32_nc,num:1,64x128,0.00610352
......

@pengzhao-intel
Copy link
Contributor

Resolved and closing.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants