-
Notifications
You must be signed in to change notification settings - Fork 6.8k
Refine mxnet python installation #12696
Changes from 15 commits
a9d40c4
429684a
602b7d0
85f9408
5a01234
bfd9012
a758d61
69d5358
daec3f2
2bb79b0
f8ce81c
5692c4f
bcc2ead
c7d1895
0e1eb01
bc48137
7a43d0d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,7 +11,7 @@ The following describes how to install with pip for computers with CPUs, Intel C | |
- [Build from Source](#build-from-source) | ||
- Install MXNet with a Programming Language API | ||
- [Python](#install-the-mxnet-package-for-python) | ||
- [R](#install-mxnet-package-for-r) | ||
- [R](#install-the-mxnet-package-for-r) | ||
- [Julia](#install-the-mxnet-package-for-julia) | ||
|
||
|
||
|
@@ -137,14 +137,14 @@ To build and install MXNet yourself using [VS2017](https://www.visualstudio.com/ | |
``` | ||
1. Download and install [CMake](https://cmake.org/download) if it is not already installed. [CMake v3.12.2](https://cmake.org/files/v3.12/cmake-3.12.2-win64-x64.msi) has been tested with MXNet. | ||
1. Download and run the [OpenCV](https://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.4.1/opencv-3.4.1-vc14_vc15.exe/download) package. There are more recent versions of OpenCV, so please create an issue/PR to update this info if you validate one of these later versions. | ||
1. This will unzip several files. You can place them in another directory if you wish. | ||
1. Set the environment variable `OpenCV_DIR` to point to the OpenCV build directory that you just unzipped (e.g., `OpenCV_DIR = C:\utils\opencv\build`). | ||
1. This will unzip several files. You can place them in another directory if you wish. We will use `C:\utils` as our default path. | ||
1. Set the environment variable `OpenCV_DIR` to point to the OpenCV build directory that you just unzipped. Start ```cmd``` and type `set OpenCV_DIR=C:\utils\opencv\build`. | ||
1. If you don’t have the Intel Math Kernel Library (MKL) installed, you can install it and follow the [MKLDNN_README](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md) from here, or you can use OpenBLAS. These instructions will assume you're using OpenBLAS. | ||
1. Download the [OpenBlas](https://sourceforge.net/projects/openblas/files/v0.2.19/OpenBLAS-v0.2.19-Win64-int32.zip/download) package. Later versions of OpenBLAS are available, but you would need to build from source. v0.2.19 is the most recent version that ships with binaries. Contributions of more recent binaries would be appreciated. | ||
1. Unzip the file. You can place the unzipped files and folders in another directory if you wish. | ||
1. Set the environment variable `OpenBLAS_HOME` to point to the OpenBLAS directory that contains the `include` and `lib` directories (e.g., `OpenBLAS_HOME = C:\utils\OpenBLAS`). | ||
1. Download and install [CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal). If you already had CUDA, then installed VS2017, you should reinstall CUDA now so that you get the CUDA toolkit components for VS2017 integration. | ||
1. Download and install cuDNN. To get access to the download link, register as an NVIDIA community user. Then Follow the [link](http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#install-windows) to install the cuDNN. | ||
1. Unzip the file, rename it to ```OpenBLAS``` and put it under `C:\utils`. You can place the unzipped files and folders in another directory if you wish. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Did we provide command before to create C:\utils? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add this on line 140 |
||
1. Set the environment variable `OpenBLAS_HOME` to point to the OpenBLAS directory that contains the `include` and `lib` directories and type `set OpenBLAS_HOME=C:\utils\OpenBLAS` on the command prompt(```cmd```). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since the file can be unzipped and placed anywhere, what is significance of C:\utils There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, it could be anywhere but original tutorial use C"\utils so I follow it. It makes users less confused? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Usually environment variables are all caps. why is it OpenBLAS_HOME? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not sure why OpenBLAS_HOME and OpenCV_DIR environment variables are not all cps |
||
1. Download and install [CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal). If you already had CUDA, then installed VS2017, you should reinstall CUDA now so that you get the CUDA toolkit components for VS2017 integration. Note that the lastest CUDA version supported by MXNet is 9.2. You can find it on the `Legacy Releases`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. lastest -> latest ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where is Legacy Releases? |
||
1. Download and install cuDNN. To get access to the download link, register as an NVIDIA community user. Then follow the [link](http://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#install-windows) to install the cuDNN and put those libraries into ```C:\cuda```. | ||
1. Download and install [git](https://git-for-windows.github.io/) if you haven't already. | ||
|
||
After you have installed all of the required dependencies, build the MXNet source code: | ||
|
@@ -158,13 +158,14 @@ git clone https://github.com/apache/incubator-mxnet.git --recursive | |
3. Verify that the `DCUDNN_INCLUDE` and `DCUDNN_LIBRARY` environment variables are pointing to the `include` folder and `cudnn.lib` file of your CUDA installed location, and `C:\incubator-mxnet` is the location of the source code you just cloned in the previous step. | ||
4. Create a build dir using the following command and go to the directory, for example: | ||
``` | ||
mkdir C:\build | ||
cd C:\build | ||
mkdir C:\incubator-mxnet\build | ||
cd C:\incubator-mxnet\build | ||
``` | ||
5. Compile the MXNet source code with `cmake` by using following command: | ||
``` | ||
cmake -G "Visual Studio 15 2017 Win64" -T cuda=9.2,host=x64 -DUSE_CUDA=1 -DUSE_CUDNN=1 -DUSE_NVRTC=1 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_LIST=Common -DCUDA_TOOLSET=9.2 -DCUDNN_INCLUDE=C:\cuda\include -DCUDNN_LIBRARY=C:\cuda\lib\x64\cudnn.lib "C:\incubator-mxnet" | ||
``` | ||
* Make sure you set the environment variables correctly (OpenBLAS_HOME, OpenCV_DIR) and change the version of the Visual studio 2017 to v14.11 before enter above command. | ||
6. After the CMake successfully completed, compile the the MXNet source code by using following command: | ||
``` | ||
msbuild mxnet.sln /p:Configuration=Release;Platform=x64 /maxcpucount | ||
|
@@ -206,14 +207,28 @@ We have installed MXNet core library. Next, we will install MXNet interface pack | |
- [Julia](#install-the-mxnet-package-for-julia) | ||
- **Scala** is not yet available for Windows | ||
|
||
## Install MXNet for Python | ||
## Install the MXNet Package for Python | ||
|
||
These steps are required after building from source. If you already installed MXNet by using pip, you do not need to do these steps to use MXNet with Python. | ||
|
||
1. Install ```Python``` using windows installer available [here](https://www.python.org/downloads/release/python-2712/). | ||
2. Install ```Numpy``` using windows installer available [here](https://scipy.org/index.html). | ||
3. Next, we install Python package interface for MXNet. You can find the Python interface package for [MXNet on GitHub](https://github.com/dmlc/mxnet/tree/master/python/mxnet). | ||
|
||
3. Start ```cmd``` and create a folder named ```common```(```mkdir C:\common```) | ||
4. Download the [mingw64_dll.zip](https://sourceforge.net/projects/openblas/files/v0.2.12/mingw64_dll.zip/download), unzip and copy three libraries (.dll files) that openblas.dll depends on to ```C:\common```. | ||
5. Copy the required .dll file to ```C:\common``` and make sure following libraries (.dll files) in the folder. | ||
``` | ||
libgcc_s_seh-1.dll (in mingw64_dll) | ||
libgfortran-3.dll (in mingw64_dll) | ||
libquadmath-0.dll (in mingw64_dll) | ||
libopenblas.dll (in OpenBlas folder you download) | ||
opencv_world341.dll (in OpenCV folder you download) | ||
``` | ||
6. Add ```C:\common``` to Environment Variables. | ||
* Type ```control sysdm.cpl``` on ```cmp``` | ||
* Select the **Advanced tab** and click **Environment Variables** | ||
* Double click the **Path** and click **New** | ||
* Add ```C:\common``` and click OK | ||
7. Use setup.py to insatll the package. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. spelling... |
||
```bash | ||
# Assuming you are in root mxnet source code folder | ||
cd python | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why why is this new order of operations for CUDA and cuDNN?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the order is still OpenBLAS -> CUDA -> cuDNN. Did you mean that?