From 74be568fd402e0942930016a3ef2ed2a928b1584 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Thu, 24 May 2018 19:27:09 +0800 Subject: [PATCH 01/16] add linux and macos doc --- MKL_README.md | 104 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) diff --git a/MKL_README.md b/MKL_README.md index a5c63b097c5e..1a3b82829601 100644 --- a/MKL_README.md +++ b/MKL_README.md @@ -12,6 +12,110 @@ Installing and enabling the full MKL installation enables MKL support for all op 4. Run 'sudo python setup.py install' +## Build/Install MXNet with MKLDNN on Linux: + +### Ubuntu pre-req + +``` +apt-get update && apt-get install -y build-essential git libopencv-dev curl gcc libopenblas-dev python python-pip python-dev python-opencv graphviz python-scipy python-sklearn +``` + +### Clone MXNet sources + +``` +git clone --recursive https://github.com/apache/incubator-mxnet.git +cd incubator-mxnet +git submodule update --recursive --init +``` + +### Build MXNet with MKLDNN + +``` +make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=openblas USE_PROFILER=1 +``` + +If you want to use Intel MKL blas, you can set `USE_BLAS=mkl USE_INTEL_PATH=/opt/intel`. + +### Verify MXNet with python + +``` +export PYTHONPATH=~/incubator-mxnet/python +pip install --upgrade pip +pip install --upgrade jupyter graphviz cython pandas bokeh matplotlib opencv-python requests +python -c "import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());" + +Expected Output: + +[[ 2. 2. 2.] + [ 2. 2. 2.]] +``` + +## Build/Install MXNet with MKLDNN on MacOS: + +### MacOS pre-req + +Install the dependencies, required for MXNet, with the following commands: + +- Homebrew +- gcc (clang in macOS does not support openMP) +- OpenCV (for computer vision operations) + +``` +# Paste this command in Mac terminal to install Homebrew +/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" + +# install dependency +brew update +brew install pkg-config +brew install graphviz +brew tap homebrew/core +brew install opencv +brew tap homebrew/versions +brew install gcc49 +brew link gcc49 +``` + +### Modify Makefile to enable openMP for MacOS + +If you want to enable openMP for better performance, you should modify these two file: + +1. Makefile L138: + +``` +ifeq ($(USE_OPENMP), 1) + # ifneq ($(UNAME_S), Darwin) + CFLAGS += -fopenmp + # endif +endif +``` + +2. prepare_mkldnn.sh L96: + +``` +CC=gcc-4.9 CXX=g++-4.9 cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR -B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" -DWITH_TEST=OFF -DWITH_EXAMPLE=OFF >&2 +``` + +### Build MXNet with MKLDNN + +``` +make -j $(sysctl -n hw.ncpu) USE_OPENCV=0 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=apple USE_PROFILER=1 +``` + +*Note: Temporarily disable OPENCV.* + +### Verify MXNet with python + +``` +export PYTHONPATH=~/incubator-mxnet/python +pip install --upgrade pip +pip install --upgrade jupyter graphviz cython pandas bokeh matplotlib opencv-python requests +python -c "import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());" + +Expected Output: + +[[ 2. 2. 2.] + [ 2. 2. 2.]] +``` ## Build/Install MXNet with MKLDNN on Windows: From 5966cb5ffaefc5fb406426da1a1f729188203186 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Thu, 24 May 2018 22:44:54 +0800 Subject: [PATCH 02/16] update doc --- MKL_README.md | 106 +++++++++++++++++++------------------------------- 1 file changed, 40 insertions(+), 66 deletions(-) diff --git a/MKL_README.md b/MKL_README.md index 1a3b82829601..6d81249ae29d 100644 --- a/MKL_README.md +++ b/MKL_README.md @@ -1,20 +1,12 @@ -## Build/Install MXNet with a full MKL installation: +* [1. Build/Install MXNet with MKLDNN on Linux](#1) +* [2. Build/Install MXNet with MKLDNN on MacOS](#2) +* [3. Build/Install MXNet with MKLDNN on Windows](#3) +* [4. Verify MXNet with python](#4) +* [5. Build/Install MXNet with a full MKL installation](#5) -To make it convenient for customers, Intel introduced a new license called [Intel® Simplified license](https://software.intel.com/en-us/license/intel-simplified-software-license) that allows to redistribute not only dynamic libraries but also headers, examples and static libraries. - -Installing and enabling the full MKL installation enables MKL support for all operators under the linalg namespace. - - 1. Download and install the latest full MKL version following instructions on the [intel website.](https://software.intel.com/en-us/mkl) - - 2. Run 'make -j ${nproc} USE_BLAS=mkl' +

Build/Install MXNet with MKLDNN on Linux

- 3. Navigate into the python directory - - 4. Run 'sudo python setup.py install' - -## Build/Install MXNet with MKLDNN on Linux: - -### Ubuntu pre-req +### Prerequisites ``` apt-get update && apt-get install -y build-essential git libopencv-dev curl gcc libopenblas-dev python python-pip python-dev python-opencv graphviz python-scipy python-sklearn @@ -36,23 +28,9 @@ make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=openblas USE_PROFILER=1 If you want to use Intel MKL blas, you can set `USE_BLAS=mkl USE_INTEL_PATH=/opt/intel`. -### Verify MXNet with python - -``` -export PYTHONPATH=~/incubator-mxnet/python -pip install --upgrade pip -pip install --upgrade jupyter graphviz cython pandas bokeh matplotlib opencv-python requests -python -c "import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());" - -Expected Output: - -[[ 2. 2. 2.] - [ 2. 2. 2.]] -``` +

Build/Install MXNet with MKLDNN on MacOS

-## Build/Install MXNet with MKLDNN on MacOS: - -### MacOS pre-req +### Prerequisites Install the dependencies, required for MXNet, with the following commands: @@ -75,9 +53,9 @@ brew install gcc49 brew link gcc49 ``` -### Modify Makefile to enable openMP for MacOS +### Enable openMP for MacOS -If you want to enable openMP for better performance, you should modify these two file: +If you want to enable openMP for better performance, you should modify these two files: 1. Makefile L138: @@ -103,21 +81,7 @@ make -j $(sysctl -n hw.ncpu) USE_OPENCV=0 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=app *Note: Temporarily disable OPENCV.* -### Verify MXNet with python - -``` -export PYTHONPATH=~/incubator-mxnet/python -pip install --upgrade pip -pip install --upgrade jupyter graphviz cython pandas bokeh matplotlib opencv-python requests -python -c "import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());" - -Expected Output: - -[[ 2. 2. 2.] - [ 2. 2. 2.]] -``` - -## Build/Install MXNet with MKLDNN on Windows: +

Build/Install MXNet with MKLDNN on Windows

To build and install MXNet yourself, you need the following dependencies. Install the required dependencies: @@ -156,26 +120,36 @@ Also ```libmkldnn.dll``` with be in the ```./build/3rdparty/mkldnn/src/Release/` 6. Make sure that all the dll files used above(such as `libmkldnn.dll`, `libmklml.dll`, `libiomp5.dll`, `libopenblas.dll`, etc) are added to the system PATH. For convinence, you can put all of them to ```\windows\system32```. Or you will come across `Not Found Dependencies` when loading mxnet. -## Install MXNet for Python +

Verify MXNet with python

-1. Install ```Python``` using windows installer available [here](https://www.python.org/downloads/release/python-2712/). -2. Install ```Numpy``` using windows installer available [here](http://scipy.org/install.html). -3. Next, we install Python package interface for MXNet. You can find the Python interface package for [MXNet on GitHub](https://github.com/dmlc/mxnet/tree/master/python/mxnet). +``` +export PYTHONPATH=~/incubator-mxnet/python +pip install --upgrade pip +pip install --upgrade jupyter graphviz cython pandas bokeh matplotlib opencv-python requests +python -c "import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());" -```CMD - cd python - python setup.py install +Expected Output: + +[[ 2. 2. 2.] + [ 2. 2. 2.]] ``` -Done! We have installed MXNet with Python interface. Run below commands to verify our installation is successful. -```CMD - # Open Python terminal - python - # You should be able to import mxnet library without any issues. - >>> import mxnet as mx; - >>> a = mx.nd.ones((2, 3)); - >>> print ((a*2).asnumpy()); - [[ 2. 2. 2.] - [ 2. 2. 2.]] +### Verify whether MKLDNN works: + ``` -We actually did a small tensor computation using MXNet! You are all set with MKLDNN MXNet on your Windows machine. +WIP +``` + +

Build/Install MXNet with a full MKL installation

+ +To make it convenient for customers, Intel introduced a new license called [Intel® Simplified license](https://software.intel.com/en-us/license/intel-simplified-software-license) that allows to redistribute not only dynamic libraries but also headers, examples and static libraries. + +Installing and enabling the full MKL installation enables MKL support for all operators under the linalg namespace. + + 1. Download and install the latest full MKL version following instructions on the [intel website.](https://software.intel.com/en-us/mkl) + + 2. Run 'make -j ${nproc} USE_BLAS=mkl' + + 3. Navigate into the python directory + + 4. Run 'sudo python setup.py install' \ No newline at end of file From a87d357a2e12ffc95c168fad9f4949826f572186 Mon Sep 17 00:00:00 2001 From: Tao Lv Date: Thu, 24 May 2018 23:00:22 +0800 Subject: [PATCH 03/16] Update MKL_README.md --- MKL_README.md | 40 +++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/MKL_README.md b/MKL_README.md index 6d81249ae29d..89d5e7f6d041 100644 --- a/MKL_README.md +++ b/MKL_README.md @@ -1,10 +1,12 @@ -* [1. Build/Install MXNet with MKLDNN on Linux](#1) -* [2. Build/Install MXNet with MKLDNN on MacOS](#2) -* [3. Build/Install MXNet with MKLDNN on Windows](#3) +

Contents

+ +* [1. Build/Install MXNet with MKL-DNN on Linux](#1) +* [2. Build/Install MXNet with MKL-DNN on MacOS](#2) +* [3. Build/Install MXNet with MKL-DNN on Windows](#3) * [4. Verify MXNet with python](#4) * [5. Build/Install MXNet with a full MKL installation](#5) -

Build/Install MXNet with MKLDNN on Linux

+

Build/Install MXNet with MKL-DNN on Linux

### Prerequisites @@ -20,22 +22,22 @@ cd incubator-mxnet git submodule update --recursive --init ``` -### Build MXNet with MKLDNN +### Build MXNet with MKL-DNN ``` -make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=openblas USE_PROFILER=1 +make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=mkl USE_INTEL_PATH=/opt/intel ``` -If you want to use Intel MKL blas, you can set `USE_BLAS=mkl USE_INTEL_PATH=/opt/intel`. +If you don't have full MKL library installed, you can use OpenBLAS by setting `USE_BLAS=openblas`. -

Build/Install MXNet with MKLDNN on MacOS

+

Build/Install MXNet with MKL-DNN on MacOS

### Prerequisites Install the dependencies, required for MXNet, with the following commands: - Homebrew -- gcc (clang in macOS does not support openMP) +- gcc (clang in macOS does not support OpenMP) - OpenCV (for computer vision operations) ``` @@ -53,9 +55,9 @@ brew install gcc49 brew link gcc49 ``` -### Enable openMP for MacOS +### Enable OpenMP for MacOS -If you want to enable openMP for better performance, you should modify these two files: +If you want to enable OpenMP for better performance, you should modify these two files: 1. Makefile L138: @@ -73,7 +75,7 @@ endif CC=gcc-4.9 CXX=g++-4.9 cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR -B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" -DWITH_TEST=OFF -DWITH_EXAMPLE=OFF >&2 ``` -### Build MXNet with MKLDNN +### Build MXNet with MKL-DNN ``` make -j $(sysctl -n hw.ncpu) USE_OPENCV=0 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=apple USE_PROFILER=1 @@ -81,7 +83,7 @@ make -j $(sysctl -n hw.ncpu) USE_OPENCV=0 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=app *Note: Temporarily disable OPENCV.* -

Build/Install MXNet with MKLDNN on Windows

+

Build/Install MXNet with MKL-DNN on Windows

To build and install MXNet yourself, you need the following dependencies. Install the required dependencies: @@ -99,7 +101,7 @@ After you have installed all of the required dependencies, build the MXNet sourc 1. Download the MXNet source code from [GitHub](https://github.com/apache/incubator-mxnet). Don't forget to pull the submodules: ``` - git clone https://github.com/apache/incubator-mxnet.git --recursive +git clone https://github.com/apache/incubator-mxnet.git --recursive ``` 2. Copy file `3rdparty/mkldnn/config_template.vcxproj` to incubator-mxnet root. @@ -109,9 +111,9 @@ After you have installed all of the required dependencies, build the MXNet sourc 4. Use [CMake](https://cmake.org/) to create a Visual Studio solution in ```./build``` or some other directory. Make sure to specify the architecture in the [CMake](https://cmake.org/) command: ``` - mkdir build - cd build - cmake -G "Visual Studio 14 Win64" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release +mkdir build +cd build +cmake -G "Visual Studio 14 Win64" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release ``` 5. In Visual Studio, open the solution file,```.sln```, and compile it. @@ -134,7 +136,7 @@ Expected Output: [ 2. 2. 2.]] ``` -### Verify whether MKLDNN works: +### Verify whether MKL-DNN works: ``` WIP @@ -152,4 +154,4 @@ Installing and enabling the full MKL installation enables MKL support for all op 3. Navigate into the python directory - 4. Run 'sudo python setup.py install' \ No newline at end of file + 4. Run 'sudo python setup.py install' From 5a73eea990e7e6ffef961c05ac4eb4943aa10073 Mon Sep 17 00:00:00 2001 From: Tao Lv Date: Thu, 24 May 2018 23:22:09 +0800 Subject: [PATCH 04/16] Update MKL_README.md Add convolution code to verify mkldnn backend --- MKL_README.md | 38 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/MKL_README.md b/MKL_README.md index 89d5e7f6d041..41b4fca3f2d6 100644 --- a/MKL_README.md +++ b/MKL_README.md @@ -136,10 +136,44 @@ Expected Output: [ 2. 2. 2.]] ``` -### Verify whether MKL-DNN works: +### Verify whether MKL-DNN works +After MXNet is installed, you can verify if MKL-DNN backend works well with a single Convolution layer. + +``` +import mxnet as mx +import numpy as np +from mxnet import Context + +num_filter = 32 +kernel = (3, 3) +pad = (1, 1) +shape = (32, 32, 256, 256) + +x = mx.sym.Variable('x') +w = mx.sym.Variable('w') +y = mx.sym.Convolution(data=x, weight=w, num_filter=num_filter, kernel=kernel, no_bias=True, pad=pad) +exe = y.simple_bind(Context.default_ctx, x=shape) + +exe.arg_arrays[0][:] = np.random.normal(size=exe.arg_arrays[0].shape) +exe.arg_arrays[1][:] = np.random.normal(size=exe.arg_arrays[1].shape) + +exe.forward(is_train=False) +o = exe.outputs[0] +t = o.asnumpy() +``` + +You can open the `MKLDNN_VERBOSE` flag by setting environment variable: +``` +export MKLDNN_VERBOSE=1 +``` +Then by running above code snippet, you probably will get the following output message which means `convolution` and `reorder` primitive from MKL-DNN are called. Layout information and primitive execution performance are also demonstrated in the log message. ``` -WIP +mkldnn_verbose,exec,reorder,jit:uni,undef,in:f32_nchw out:f32_nChw16c,num:1,32x32x256x256,6.47681 +mkldnn_verbose,exec,reorder,jit:uni,undef,in:f32_oihw out:f32_OIhw16i16o,num:1,32x32x3x3,0.0429688 +mkldnn_verbose,exec,convolution,jit:avx512_common,forward_inference,fsrc:nChw16c fwei:OIhw16i16o fbia:undef fdst:nChw16c,alg:convolution_direct,mb32_g1ic32oc32_ih256oh256kh3sh1dh0ph1_iw256ow256kw3sw1dw0pw1,9.98193 +mkldnn_verbose,exec,reorder,jit:uni,undef,in:f32_oihw out:f32_OIhw16i16o,num:1,32x32x3x3,0.0510254 +mkldnn_verbose,exec,reorder,jit:uni,undef,in:f32_nChw16c out:f32_nchw,num:1,32x32x256x256,20.4819 ```

Build/Install MXNet with a full MKL installation

From 46e5cea4e49505e9cc167fa67ac14f7226a6e658 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Fri, 25 May 2018 14:29:05 +0800 Subject: [PATCH 05/16] add homebrew link --- MKL_README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MKL_README.md b/MKL_README.md index 41b4fca3f2d6..15f0c8c0ee74 100644 --- a/MKL_README.md +++ b/MKL_README.md @@ -36,7 +36,7 @@ If you don't have full MKL library installed, you can use OpenBLAS by setting `U Install the dependencies, required for MXNet, with the following commands: -- Homebrew +- [Homebrew](https://brew.sh/) - gcc (clang in macOS does not support OpenMP) - OpenCV (for computer vision operations) From 3f3259ffe7ad8f63a7778a7c079f7f461ac62e6a Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Sat, 26 May 2018 11:19:28 +0800 Subject: [PATCH 06/16] rename to MKLDNN_README --- MKL_README.md => MKLDNN_README.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename MKL_README.md => MKLDNN_README.md (100%) diff --git a/MKL_README.md b/MKLDNN_README.md similarity index 100% rename from MKL_README.md rename to MKLDNN_README.md From 2c503c25b2e8df40cb9d800e8f2e9f18d7db1a11 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Mon, 28 May 2018 12:52:29 +0800 Subject: [PATCH 07/16] add mkl verify --- MKLDNN_README.md | 62 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 50 insertions(+), 12 deletions(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index 15f0c8c0ee74..12d132de6199 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -1,12 +1,14 @@ +# Build/Install MXNet with MKL-DNN +

Contents

-* [1. Build/Install MXNet with MKL-DNN on Linux](#1) -* [2. Build/Install MXNet with MKL-DNN on MacOS](#2) -* [3. Build/Install MXNet with MKL-DNN on Windows](#3) +* [1. Linux](#1) +* [2. MacOS](#2) +* [3. Windows](#3) * [4. Verify MXNet with python](#4) -* [5. Build/Install MXNet with a full MKL installation](#5) +* [5. Enable MKL BLAS](#5) -

Build/Install MXNet with MKL-DNN on Linux

+

Linux

### Prerequisites @@ -30,7 +32,7 @@ make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=mkl USE_INTEL_PATH=/opt/inte If you don't have full MKL library installed, you can use OpenBLAS by setting `USE_BLAS=openblas`. -

Build/Install MXNet with MKL-DNN on MacOS

+

MacOS

### Prerequisites @@ -83,7 +85,7 @@ make -j $(sysctl -n hw.ncpu) USE_OPENCV=0 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=app *Note: Temporarily disable OPENCV.* -

Build/Install MXNet with MKL-DNN on Windows

+

Windows

To build and install MXNet yourself, you need the following dependencies. Install the required dependencies: @@ -143,7 +145,6 @@ After MXNet is installed, you can verify if MKL-DNN backend works well with a si ``` import mxnet as mx import numpy as np -from mxnet import Context num_filter = 32 kernel = (3, 3) @@ -153,7 +154,7 @@ shape = (32, 32, 256, 256) x = mx.sym.Variable('x') w = mx.sym.Variable('w') y = mx.sym.Convolution(data=x, weight=w, num_filter=num_filter, kernel=kernel, no_bias=True, pad=pad) -exe = y.simple_bind(Context.default_ctx, x=shape) +exe = y.simple_bind(mx.cpu(), x=shape) exe.arg_arrays[0][:] = np.random.normal(size=exe.arg_arrays[0].shape) exe.arg_arrays[1][:] = np.random.normal(size=exe.arg_arrays[1].shape) @@ -176,7 +177,7 @@ mkldnn_verbose,exec,reorder,jit:uni,undef,in:f32_oihw out:f32_OIhw16i16o,num:1,3 mkldnn_verbose,exec,reorder,jit:uni,undef,in:f32_nChw16c out:f32_nchw,num:1,32x32x256x256,20.4819 ``` -

Build/Install MXNet with a full MKL installation

+

Enable MKL BLAS

To make it convenient for customers, Intel introduced a new license called [Intel® Simplified license](https://software.intel.com/en-us/license/intel-simplified-software-license) that allows to redistribute not only dynamic libraries but also headers, examples and static libraries. @@ -184,8 +185,45 @@ Installing and enabling the full MKL installation enables MKL support for all op 1. Download and install the latest full MKL version following instructions on the [intel website.](https://software.intel.com/en-us/mkl) - 2. Run 'make -j ${nproc} USE_BLAS=mkl' + 2. Run `make -j ${nproc} USE_BLAS=mkl` 3. Navigate into the python directory - 4. Run 'sudo python setup.py install' + 4. Run `sudo python setup.py install` + +### Verify whether MKL works + +After MXNet is installed, you can verify if MKL BLAS works well with a single dot layer. + +``` +import mxnet as mx +import numpy as np + +shape_x = (1, 10, 8) +shape_w = (1, 12, 8) + +x_npy = np.random.normal(0, 1, shape_x) +w_npy = np.random.normal(0, 1, shape_w) + +x = mx.sym.Variable('x') +w = mx.sym.Variable('w') +y = mx.sym.batch_dot(x, w, transpose_b=True) +exe = y.simple_bind(mx.cpu(), x=x_npy.shape, w=w_npy.shape) + +exe.forward(is_train=False) +o = exe.outputs[0] +t = o.asnumpy() +``` + +You can open the `MKL_VERBOSE` flag by setting environment variable: +``` +export MKL_VERBOSE=1 +``` +Then by running above code snippet, you probably will get the following output message which means `SGEMM` primitive from MKL are called. Layout information and primitive execution performance are also demonstrated in the log message. +``` +Numpy + Intel(R) MKL: THREADING LAYER: (null) +Numpy + Intel(R) MKL: setting Intel(R) MKL to use INTEL OpenMP runtime +Numpy + Intel(R) MKL: preloading libiomp5.so runtime +MKL_VERBOSE Intel(R) MKL 2018.0 Update 1 Product build 20171007 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors, Lnx 2.40GHz lp64 intel_thread NMICDev:0 +MKL_VERBOSE SGEMM(T,N,12,10,8,0x7f7f927b1378,0x1bc2140,8,0x1ba8040,8,0x7f7f927b1380,0x7f7f7400a280,12) 8.93ms CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:40 WDiv:HOST:+0.000 +``` \ No newline at end of file From a8b5e9b41f7ddd04bcd2f901a586a27396e7c608 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Tue, 29 May 2018 00:25:55 +0800 Subject: [PATCH 08/16] trigger --- MKLDNN_README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index 12d132de6199..f70d0fc85fb3 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -103,7 +103,7 @@ After you have installed all of the required dependencies, build the MXNet sourc 1. Download the MXNet source code from [GitHub](https://github.com/apache/incubator-mxnet). Don't forget to pull the submodules: ``` -git clone https://github.com/apache/incubator-mxnet.git --recursive +git clone --recursive https://github.com/apache/incubator-mxnet.git ``` 2. Copy file `3rdparty/mkldnn/config_template.vcxproj` to incubator-mxnet root. From cb2080b18bebc9765609bb03f19f6108f8cb6d2f Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Tue, 29 May 2018 11:07:18 +0800 Subject: [PATCH 09/16] trigger --- MKLDNN_README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index f70d0fc85fb3..9db627f1bdfd 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -65,9 +65,9 @@ If you want to enable OpenMP for better performance, you should modify these two ``` ifeq ($(USE_OPENMP), 1) - # ifneq ($(UNAME_S), Darwin) - CFLAGS += -fopenmp - # endif +# ifneq ($(UNAME_S), Darwin) + CFLAGS += -fopenmp +# endif endif ``` @@ -226,4 +226,4 @@ Numpy + Intel(R) MKL: setting Intel(R) MKL to use INTEL OpenMP runtime Numpy + Intel(R) MKL: preloading libiomp5.so runtime MKL_VERBOSE Intel(R) MKL 2018.0 Update 1 Product build 20171007 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors, Lnx 2.40GHz lp64 intel_thread NMICDev:0 MKL_VERBOSE SGEMM(T,N,12,10,8,0x7f7f927b1378,0x1bc2140,8,0x1ba8040,8,0x7f7f927b1380,0x7f7f7400a280,12) 8.93ms CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:40 WDiv:HOST:+0.000 -``` \ No newline at end of file +``` From 4f23cbb591845b8c3ec37061a0f3834d857c48c9 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Tue, 19 Jun 2018 08:41:52 +0800 Subject: [PATCH 10/16] set mac complier to gcc47 --- MKLDNN_README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index 9db627f1bdfd..e673c2d31f30 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -80,7 +80,7 @@ CC=gcc-4.9 CXX=g++-4.9 cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INST ### Build MXNet with MKL-DNN ``` -make -j $(sysctl -n hw.ncpu) USE_OPENCV=0 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=apple USE_PROFILER=1 +make -j $(sysctl -n hw.ncpu) CC=gcc-4.9 CXX=g++-4.9 USE_OPENCV=0 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=apple USE_PROFILER=1 ``` *Note: Temporarily disable OPENCV.* From f477fa9dea23c12bacd15922abf44575814a7974 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Tue, 19 Jun 2018 16:21:57 +0800 Subject: [PATCH 11/16] add VS2017 support experimentally --- MKLDNN_README.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index e673c2d31f30..7e82ec7bd4e5 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -87,6 +87,10 @@ make -j $(sysctl -n hw.ncpu) CC=gcc-4.9 CXX=g++-4.9 USE_OPENCV=0 USE_OPENMP=1 US

Windows

+We recommend to build and install MXNet yourself using [Microsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/), or you can also try experimentally the latest [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/). + +**Visual Studio 2015** + To build and install MXNet yourself, you need the following dependencies. Install the required dependencies: 1. If [Microsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) is not already installed, download and install it. You can download and install the free community edition. @@ -124,6 +128,60 @@ Also ```libmkldnn.dll``` with be in the ```./build/3rdparty/mkldnn/src/Release/` 6. Make sure that all the dll files used above(such as `libmkldnn.dll`, `libmklml.dll`, `libiomp5.dll`, `libopenblas.dll`, etc) are added to the system PATH. For convinence, you can put all of them to ```\windows\system32```. Or you will come across `Not Found Dependencies` when loading mxnet. +**Visual Studio 2017** + +To build and install MXNet yourself using [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/), you need the following dependencies. Install the required dependencies: + +1. If [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/) is not already installed, download and install it. You can download and install the free community edition. +2. Download and install [CMake](https://cmake.org/files/v3.11/cmake-3.11.0-rc4-win64-x64.msi) if it is not already installed. +3. Download and install [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download). +4. Unzip the OpenCV package. +5. Set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g., ```OpenCV_DIR = C:\utils\opencv\build```). +6. If you don’t have the Intel Math Kernel Library (MKL) installed, download and install [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.20/OpenBLAS%200.2.20%20version.zip/download). +7. Set the environment variable ```OpenBLAS_HOME``` to point to the ```OpenBLAS``` directory that contains the ```include``` and ```lib``` directories (e.g., ```OpenBLAS_HOME = C:\utils\OpenBLAS```). + +After you have installed all of the required dependencies, build the MXNet source code: + +1. Start ```cmd``` in windows. + +2. Download the MXNet source code from GitHub by using following command: + +```r +cd C:\ +git clone https://github.com/apache/incubator-mxnet.git --recursive +``` + +3. Copy file `3rdparty/mkldnn/config_template.vcxproj` to incubator-mxnet root. + +4. Follow [this link](https://docs.microsoft.com/en-us/visualstudio/install/modify-visual-studio) to modify ```Individual components```, and check ```VC++ 2017 version 15.4 v14.11 toolset```, and click ```Modify```. + +5. Change the version of the Visual studio 2017 to v14.11 using the following command (by default the VS2017 is installed in the following path): + +```r +"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat" -vcvars_ver=14.11 +``` + +6. Create a build dir using the following command and go to the directory, for example: + +```r +mkdir C:\build +cd C:\build +``` + +7. CMake the MXNet source code by using following command: + +```r +cmake -G "Visual Studio 15 2017 Win64" .. -T host=x64 -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release +``` + +8. After the CMake successfully completed, compile the the MXNet source code by using following command: + +```r +msbuild mxnet.sln /p:Configuration=Release;Platform=x64 /maxcpucount +``` + +9. Make sure that all the dll files used above(such as `libmkldnn.dll`, `libmklml.dll`, `libiomp5.dll`, `libopenblas.dll`, etc) are added to the system PATH. For convinence, you can put all of them to ```\windows\system32```. Or you will come across `Not Found Dependencies` when loading mxnet. +

Verify MXNet with python

``` From 3f74b8658916e196508f0fc5d2bbaeb7cfc44f84 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Wed, 20 Jun 2018 10:33:49 +0800 Subject: [PATCH 12/16] improve quality --- MKLDNN_README.md | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index 7e82ec7bd4e5..9f8e2b2edcaa 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -13,7 +13,11 @@ ### Prerequisites ``` -apt-get update && apt-get install -y build-essential git libopencv-dev curl gcc libopenblas-dev python python-pip python-dev python-opencv graphviz python-scipy python-sklearn +sudo apt-get update +sudo apt-get install -y build-essential git +sudo apt-get install -y libopenblas-dev liblapack-dev +sudo apt-get install -y libopencv-dev +sudo apt-get install -y graphviz ``` ### Clone MXNet sources @@ -21,7 +25,6 @@ apt-get update && apt-get install -y build-essential git libopencv-dev curl gcc ``` git clone --recursive https://github.com/apache/incubator-mxnet.git cd incubator-mxnet -git submodule update --recursive --init ``` ### Build MXNet with MKL-DNN @@ -30,7 +33,7 @@ git submodule update --recursive --init make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=mkl USE_INTEL_PATH=/opt/intel ``` -If you don't have full MKL library installed, you can use OpenBLAS by setting `USE_BLAS=openblas`. +If you don't have full [MKL](https://software.intel.com/en-us/intel-mkl) library installed, you can use OpenBLAS by setting `USE_BLAS=openblas`.

MacOS

@@ -57,6 +60,13 @@ brew install gcc49 brew link gcc49 ``` +### Clone MXNet sources + +``` +git clone --recursive https://github.com/apache/incubator-mxnet.git +cd incubator-mxnet +``` + ### Enable OpenMP for MacOS If you want to enable OpenMP for better performance, you should modify these two files: @@ -148,7 +158,7 @@ After you have installed all of the required dependencies, build the MXNet sourc ```r cd C:\ -git clone https://github.com/apache/incubator-mxnet.git --recursive +git clone --recursive https://github.com/apache/incubator-mxnet.git ``` 3. Copy file `3rdparty/mkldnn/config_template.vcxproj` to incubator-mxnet root. @@ -185,9 +195,8 @@ msbuild mxnet.sln /p:Configuration=Release;Platform=x64 /maxcpucount

Verify MXNet with python

``` -export PYTHONPATH=~/incubator-mxnet/python -pip install --upgrade pip -pip install --upgrade jupyter graphviz cython pandas bokeh matplotlib opencv-python requests +cd python +sudo python setup.py install python -c "import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());" Expected Output: From b7dc9a5ca960595ed5c3f3b89b7c397a816df5fb Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Wed, 20 Jun 2018 14:14:16 +0800 Subject: [PATCH 13/16] improve quality --- MKLDNN_README.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index 9f8e2b2edcaa..e79012aee266 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -1,5 +1,7 @@ # Build/Install MXNet with MKL-DNN +Building MXNet with [Intel MKL-DNN](https://github.com/intel/mkl-dnn) will gain better performance when using Intel Xeon CPUs for training and inference. The improvement of performance can be seen in this [page](https://mxnet.incubator.apache.org/faq/perf.html#intel-cpu). Below are instructions for linux, MacOS and Windows platform. +

Contents

* [1. Linux](#1) @@ -69,9 +71,11 @@ cd incubator-mxnet ### Enable OpenMP for MacOS -If you want to enable OpenMP for better performance, you should modify these two files: +If you want to enable OpenMP for better performance, you should modify these two files in MXNet root dictionary: + +1. Makefile: -1. Makefile L138: +Add CFLAGS '-fopenmp' for Darwin. ``` ifeq ($(USE_OPENMP), 1) @@ -81,7 +85,9 @@ ifeq ($(USE_OPENMP), 1) endif ``` -2. prepare_mkldnn.sh L96: +2. prepare_mkldnn.sh: + +Set cmake complier to gcc-4.9 to support OpenMP. ``` CC=gcc-4.9 CXX=g++-4.9 cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR -B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" -DWITH_TEST=OFF -DWITH_EXAMPLE=OFF >&2 From 33def026bb34a3cad86b58b22c4f2c704149bc8a Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Wed, 11 Jul 2018 14:07:16 +0800 Subject: [PATCH 14/16] modify mac build instruction since prepare_mkldnn.sh has been rm --- MKLDNN_README.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index e79012aee266..e495a8b2b094 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -71,9 +71,7 @@ cd incubator-mxnet ### Enable OpenMP for MacOS -If you want to enable OpenMP for better performance, you should modify these two files in MXNet root dictionary: - -1. Makefile: +If you want to enable OpenMP for better performance, you should modify the Makefile in MXNet root dictionary: Add CFLAGS '-fopenmp' for Darwin. @@ -85,14 +83,6 @@ ifeq ($(USE_OPENMP), 1) endif ``` -2. prepare_mkldnn.sh: - -Set cmake complier to gcc-4.9 to support OpenMP. - -``` -CC=gcc-4.9 CXX=g++-4.9 cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR -B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" -DWITH_TEST=OFF -DWITH_EXAMPLE=OFF >&2 -``` - ### Build MXNet with MKL-DNN ``` From 6e6472f71aeb74f475af94b5f3c5248f93839c12 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Wed, 11 Jul 2018 19:55:39 +0800 Subject: [PATCH 15/16] trigger --- MKLDNN_README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index e495a8b2b094..0c67f35713f9 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -59,7 +59,7 @@ brew tap homebrew/core brew install opencv brew tap homebrew/versions brew install gcc49 -brew link gcc49 +brew link gcc49 #gcc-5 and gcc-7 also work ``` ### Clone MXNet sources From 846d3bcb938b357a1ada0d2a5eb6170d75723fe6 Mon Sep 17 00:00:00 2001 From: xinyu-intel Date: Thu, 12 Jul 2018 09:32:40 +0800 Subject: [PATCH 16/16] add some improvement --- MKLDNN_README.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/MKLDNN_README.md b/MKLDNN_README.md index 0c67f35713f9..43cced49ed0d 100644 --- a/MKLDNN_README.md +++ b/MKLDNN_README.md @@ -9,6 +9,7 @@ Building MXNet with [Intel MKL-DNN](https://github.com/intel/mkl-dnn) will gain * [3. Windows](#3) * [4. Verify MXNet with python](#4) * [5. Enable MKL BLAS](#5) +* [6. Support](#6)

Linux

@@ -100,8 +101,8 @@ We recommend to build and install MXNet yourself using [Microsoft Visual Studio To build and install MXNet yourself, you need the following dependencies. Install the required dependencies: 1. If [Microsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) is not already installed, download and install it. You can download and install the free community edition. -2. Download and Install [CMake](https://cmake.org/) if it is not already installed. -3. Download and install [OpenCV](http://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.0.0/opencv-3.0.0.exe/download). +2. Download and Install [CMake 3](https://cmake.org/) if it is not already installed. +3. Download and install [OpenCV 3](http://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.0.0/opencv-3.0.0.exe/download). 4. Unzip the OpenCV package. 5. Set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (```C:\opencv\build\x64\vc14``` for example). Also, you need to add the OpenCV bin directory (```C:\opencv\build\x64\vc14\bin``` for example) to the ``PATH`` variable. 6. If you have Intel Math Kernel Library (MKL) installed, set ```MKL_ROOT``` to point to ```MKL``` directory that contains the ```include``` and ```lib```. If you want to use MKL blas, you should set ```-DUSE_BLAS=mkl``` when cmake. Typically, you can find the directory in @@ -120,8 +121,8 @@ git clone --recursive https://github.com/apache/incubator-mxnet.git 3. Start a Visual Studio command prompt. -4. Use [CMake](https://cmake.org/) to create a Visual Studio solution in ```./build``` or some other directory. Make sure to specify the architecture in the -[CMake](https://cmake.org/) command: +4. Use [CMake 3](https://cmake.org/) to create a Visual Studio solution in ```./build``` or some other directory. Make sure to specify the architecture in the +[CMake 3](https://cmake.org/) command: ``` mkdir build cd build @@ -139,7 +140,7 @@ Also ```libmkldnn.dll``` with be in the ```./build/3rdparty/mkldnn/src/Release/` To build and install MXNet yourself using [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/), you need the following dependencies. Install the required dependencies: 1. If [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/) is not already installed, download and install it. You can download and install the free community edition. -2. Download and install [CMake](https://cmake.org/files/v3.11/cmake-3.11.0-rc4-win64-x64.msi) if it is not already installed. +2. Download and install [CMake 3](https://cmake.org/files/v3.11/cmake-3.11.0-rc4-win64-x64.msi) if it is not already installed. 3. Download and install [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download). 4. Unzip the OpenCV package. 5. Set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g., ```OpenCV_DIR = C:\utils\opencv\build```). @@ -290,3 +291,11 @@ Numpy + Intel(R) MKL: preloading libiomp5.so runtime MKL_VERBOSE Intel(R) MKL 2018.0 Update 1 Product build 20171007 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors, Lnx 2.40GHz lp64 intel_thread NMICDev:0 MKL_VERBOSE SGEMM(T,N,12,10,8,0x7f7f927b1378,0x1bc2140,8,0x1ba8040,8,0x7f7f927b1380,0x7f7f7400a280,12) 8.93ms CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:40 WDiv:HOST:+0.000 ``` + +

Next Steps and Support

+ +- For questions or support specific to MKL, visit the [Intel MKL](https://software.intel.com/en-us/mkl) + +- For questions or support specific to MKL, visit the [Intel MKLDNN](https://github.com/intel/mkl-dnn) + +- If you find bugs, please open an issue on GitHub for [MXNet with MKL](https://github.com/apache/incubator-mxnet/labels/MKL) or [MXNet with MKLDNN](https://github.com/apache/incubator-mxnet/labels/MKLDNN)