From 503a15a0950904fac22c8dbfdb1824473a781554 Mon Sep 17 00:00:00 2001 From: taolv Date: Tue, 19 Feb 2019 14:28:38 +0800 Subject: [PATCH 1/7] start tutorials for mkldnn backend --- .../tutorials/mkldnn/MKLDNN_README.md | 0 docs/tutorials/mkldnn/index.md | 25 +++++++++++++++++++ 2 files changed, 25 insertions(+) rename MKLDNN_README.md => docs/tutorials/mkldnn/MKLDNN_README.md (100%) create mode 100644 docs/tutorials/mkldnn/index.md diff --git a/MKLDNN_README.md b/docs/tutorials/mkldnn/MKLDNN_README.md similarity index 100% rename from MKLDNN_README.md rename to docs/tutorials/mkldnn/MKLDNN_README.md diff --git a/docs/tutorials/mkldnn/index.md b/docs/tutorials/mkldnn/index.md new file mode 100644 index 000000000000..faf6526fb824 --- /dev/null +++ b/docs/tutorials/mkldnn/index.md @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + +# Tutorials + +```eval_rst +.. toctree:: + :glob: + + * +``` From d91e581151145f777fba27bf400cf3cb96f310cd Mon Sep 17 00:00:00 2001 From: taolv Date: Tue, 19 Feb 2019 17:20:46 +0800 Subject: [PATCH 2/7] fix links --- NEWS.md | 2 +- README.md | 2 +- docs/faq/perf.md | 2 +- docs/install/ubuntu_setup.md | 2 +- docs/install/windows_setup.md | 4 ++-- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/NEWS.md b/NEWS.md index 1af4b138e600..d1f0807fe2ba 100644 --- a/NEWS.md +++ b/NEWS.md @@ -164,7 +164,7 @@ MKLDNN backend takes advantage of MXNet subgraph to implement the most of possib ##### Quantization Performance of reduced-precision (INT8) computation is also dramatically improved after the graph optimization feature is applied on CPU Platforms. Various models are supported and can benefit from reduced-precision computation, including symbolic models, Gluon models and even custom models. Users can run most of the pre-trained models with only a few lines of commands and a new quantization script imagenet_gen_qsym_mkldnn.py. The observed accuracy loss is less than 0.5% for popular CNN networks, like ResNet-50, Inception-BN, MobileNet, etc. -Please find detailed information and performance/accuracy numbers here: [MKLDNN README](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md), [quantization README](https://github.com/apache/incubator-mxnet/tree/master/example/quantization#1) and [design proposal](https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN) +Please find detailed information and performance/accuracy numbers here: [MKLDNN README](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md), [quantization README](https://github.com/apache/incubator-mxnet/tree/master/example/quantization#1) and [design proposal](https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN) ### New Operators diff --git a/README.md b/README.md index 6a8ecdd99bd0..255605c0ff03 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ What's New * [Version 0.8.0 Release](https://github.com/dmlc/mxnet/releases/tag/v0.8.0) * [Updated Image Classification with new Pre-trained Models](./example/image-classification) * [Notebooks How to Use MXNet](https://github.com/zackchase/mxnet-the-straight-dope) -* [MKLDNN for Faster CPU Performance](./MKLDNN_README.md) +* [MKLDNN for Faster CPU Performance](./docs/tutorials/mkldnn/MKLDNN_README.md) * [MXNet Memory Monger, Training Deeper Nets with Sublinear Memory Cost](https://github.com/dmlc/mxnet-memonger) * [Tutorial for NVidia GTC 2016](https://github.com/dmlc/mxnet-gtc-tutorial) * [Embedding Torch layers and functions in MXNet](https://mxnet.incubator.apache.org/faq/torch.html) diff --git a/docs/faq/perf.md b/docs/faq/perf.md index 00310dfbb5bd..e1318b843a03 100644 --- a/docs/faq/perf.md +++ b/docs/faq/perf.md @@ -43,7 +43,7 @@ We also find that setting the following environment variables can help: | :-------- | :---------- | | `OMP_NUM_THREADS` | Suggested value: `vCPUs / 2` in which `vCPUs` is the number of virtual CPUs. For more information, please see the guide for [setting the number of threads using an OpenMP environment variable](https://software.intel.com/en-us/mkl-windows-developer-guide-setting-the-number-of-threads-using-an-openmp-environment-variable) | | `KMP_AFFINITY` | Suggested value: `granularity=fine,compact,1,0`. For more information, please see the guide for [Thread Affinity Interface (Linux* and Windows*)](https://software.intel.com/en-us/node/522691). | -| `MXNET_SUBGRAPH_BACKEND` | Set to MKLDNN to enable the [subgraph feature](https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN) for better performance. For more information please see [Build/Install MXNet with MKL-DNN](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md)| +| `MXNET_SUBGRAPH_BACKEND` | Set to MKLDNN to enable the [subgraph feature](https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN) for better performance. For more information please see [Build/Install MXNet with MKL-DNN](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md)| Note that _MXNet_ treats all CPUs on a single machine as a single device. So whether you specify `cpu(0)` or `cpu()`, _MXNet_ will use all CPU cores on the machine. diff --git a/docs/install/ubuntu_setup.md b/docs/install/ubuntu_setup.md index bffbdb3e58f2..cc99a916e5b6 100644 --- a/docs/install/ubuntu_setup.md +++ b/docs/install/ubuntu_setup.md @@ -175,7 +175,7 @@ If building on CPU and using OpenBLAS: make -j $(nproc) ``` -If building on CPU and using MKL and MKL-DNN (make sure MKL is installed according to [Math Library Selection](build_from_source.html#math-library-selection) and [MKL-DNN README](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md)): +If building on CPU and using MKL and MKL-DNN (make sure MKL is installed according to [Math Library Selection](build_from_source.html#math-library-selection) and [MKL-DNN README](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md)): ```bash git clone --recursive https://github.com/apache/incubator-mxnet.git diff --git a/docs/install/windows_setup.md b/docs/install/windows_setup.md index 3c3da5349235..96929692f1ec 100644 --- a/docs/install/windows_setup.md +++ b/docs/install/windows_setup.md @@ -136,7 +136,7 @@ We provide two primary options to build and install MXNet yourself using [Micros **NOTE:** Visual Studio 2017's compiler is `vc15`. This is not to be confused with Visual Studio 2015's compiler, `vc14`. -You also have the option to install MXNet with MKL or MKL-DNN. In this case it is recommended that you refer to the [MKLDNN_README](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md). +You also have the option to install MXNet with MKL or MKL-DNN. In this case it is recommended that you refer to the [MKLDNN_README](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md). **Option 1: Build with Microsoft Visual Studio 2017 (VS2017)** @@ -156,7 +156,7 @@ To build and install MXNet yourself using [VS2017](https://www.visualstudio.com/ 1. Download and run the [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download) package. There are more recent versions of OpenCV, so please create an issue/PR to update this info if you validate one of these later versions. 1. This will unzip several files. You can place them in another directory if you wish. We will use `C:\utils`(```mkdir C:\utils```) as our default path. 1. Set the environment variable `OpenCV_DIR` to point to the OpenCV build directory that you just unzipped. Start ```cmd``` and type `set OpenCV_DIR=C:\utils\opencv\build`. -1. If you don’t have the Intel Math Kernel Library (MKL) installed, you can install it and follow the [MKLDNN_README](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md) from here, or you can use OpenBLAS. These instructions will assume you're using OpenBLAS. +1. If you don’t have the Intel Math Kernel Library (MKL) installed, you can install it and follow the [MKLDNN_README](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md) from here, or you can use OpenBLAS. These instructions will assume you're using OpenBLAS. 1. Download the [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.19/OpenBLAS-v0.2.19-Win64-int32.zip/download) package. Later versions of OpenBLAS are available, but you would need to build from source. v0.2.19 is the most recent version that ships with binaries. Contributions of more recent binaries would be appreciated. 1. Unzip the file, rename it to ```OpenBLAS``` and put it under `C:\utils`. You can place the unzipped files and folders in another directory if you wish. 1. Set the environment variable `OpenBLAS_HOME` to point to the OpenBLAS directory that contains the `include` and `lib` directories and type `set OpenBLAS_HOME=C:\utils\OpenBLAS` on the command prompt(```cmd```). From c143cd597d9a8f3bddf4ba543bc07bc30c8a722a Mon Sep 17 00:00:00 2001 From: taolv Date: Wed, 20 Feb 2019 09:27:25 +0800 Subject: [PATCH 3/7] add to tutorial index --- docs/tutorials/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index 7e0ffaa3f72a..b8627d4a9501 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -27,6 +27,7 @@ embedded/index.md gluon/index.md java/index.md + mkldnn/index.md nlp/index.md onnx/index.md python/index.md From 990f4f066315958954e90adb7c9c70257f9fd199 Mon Sep 17 00:00:00 2001 From: taolv Date: Wed, 20 Feb 2019 10:37:53 +0800 Subject: [PATCH 4/7] fix sanity test --- tests/tutorials/test_sanity_tutorials.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/tutorials/test_sanity_tutorials.py b/tests/tutorials/test_sanity_tutorials.py index 429527db2000..7865000c7608 100644 --- a/tests/tutorials/test_sanity_tutorials.py +++ b/tests/tutorials/test_sanity_tutorials.py @@ -33,6 +33,8 @@ 'embedded/index.md', 'embedded/wine_detector.md', 'gluon/index.md', + 'mkldnn/index.md', + 'mkldnn/MKLDNN_README.md', 'nlp/index.md', 'onnx/index.md', 'python/index.md', From c7e919861f03347baf04cd72cc5ef733effab3db Mon Sep 17 00:00:00 2001 From: taolv Date: Wed, 20 Feb 2019 15:15:12 +0800 Subject: [PATCH 5/7] fix sanity test complain --- docs/tutorials/mkldnn/MKLDNN_README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/tutorials/mkldnn/MKLDNN_README.md b/docs/tutorials/mkldnn/MKLDNN_README.md index 214fc83985fb..189de3f3a0d4 100644 --- a/docs/tutorials/mkldnn/MKLDNN_README.md +++ b/docs/tutorials/mkldnn/MKLDNN_README.md @@ -151,7 +151,7 @@ To build and install MXNet yourself using [Microsoft Visual Studio 2017](https:/ 3. Download and install [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download). 4. Unzip the OpenCV package. 5. Set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g., ```OpenCV_DIR = C:\utils\opencv\build```). -6. If you don’t have the Intel Math Kernel Library (MKL) installed, download and install [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.20/OpenBLAS%200.2.20%20version.zip/download). +6. If you don't have the Intel Math Kernel Library (MKL) installed, download and install [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.20/OpenBLAS%200.2.20%20version.zip/download). 7. Set the environment variable ```OpenBLAS_HOME``` to point to the ```OpenBLAS``` directory that contains the ```include``` and ```lib``` directories (e.g., ```OpenBLAS_HOME = C:\utils\OpenBLAS```). After you have installed all of the required dependencies, build the MXNet source code: @@ -251,7 +251,7 @@ mkldnn_verbose,exec,reorder,jit:uni,undef,in:f32_nChw16c out:f32_nchw,num:1,32x3

Enable MKL BLAS

With MKL BLAS, the performace is expected to furtherly improved with variable range depending on the computation load of the models. -You can redistribute not only dynamic libraries but also headers, examples and static libraries on accepting the license [Intel® Simplified license](https://software.intel.com/en-us/license/intel-simplified-software-license). +You can redistribute not only dynamic libraries but also headers, examples and static libraries on accepting the license [Intel Simplified license](https://software.intel.com/en-us/license/intel-simplified-software-license). Installing the full MKL installation enables MKL support for all operators under the linalg namespace. 1. Download and install the latest full MKL version following instructions on the [intel website.](https://software.intel.com/en-us/mkl) @@ -317,7 +317,7 @@ This limitations of this experimental feature are:

Quantization and Inference with INT8

-Benefiting from Intel® MKL-DNN, MXNet built with Intel® MKL-DNN brings outstanding performance improvement on quantization and inference with INT8 Intel® CPU Platform on Intel® Xeon® Scalable Platform. +Benefiting from Intel MKL-DNN, MXNet built with Intel MKL-DNN brings outstanding performance improvement on quantization and inference with INT8 Intel CPU Platform on Intel Xeon Scalable Platform. - [CNN Quantization Examples](https://github.com/apache/incubator-mxnet/tree/master/example/quantization). From 936d4f74753f70a7cda152cd1e6117229e927145 Mon Sep 17 00:00:00 2001 From: taolv Date: Thu, 21 Feb 2019 15:15:27 +0800 Subject: [PATCH 6/7] add to webpage --- docs/tutorials/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index b8627d4a9501..d42903688e0b 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -143,6 +143,7 @@ Select API:  * [Large-Scale Multi-Host Multi-GPU Image Classification](/tutorials/vision/large_scale_classification.html) * [Importing an ONNX model into MXNet](/tutorials/onnx/super_resolution.html) * [Optimizing Deep Learning Computation Graphs with TensorRT](/tutorials/tensorrt/inference_with_trt.html) + * [How to build and install MXNet with MKL-DNN backend](/tutorials/mkldnn/MKLDNN_README.html) * API Guides * Core APIs * NDArray From 5238f4a25714eed19f4fbfaf9b44fde01e0a8e33 Mon Sep 17 00:00:00 2001 From: taolv Date: Fri, 22 Feb 2019 21:28:30 +0800 Subject: [PATCH 7/7] Hint message and avoid 404 --- MKLDNN_README.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 MKLDNN_README.md diff --git a/MKLDNN_README.md b/MKLDNN_README.md new file mode 100644 index 000000000000..34790c9c513d --- /dev/null +++ b/MKLDNN_README.md @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + + + +File is moved to [docs/tutorials/mkldnn/MKLDNN_README.md](docs/tutorials/mkldnn/MKLDNN_README.md).