Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-641] fix R windows install docs #11805

Merged
merged 7 commits into from
Jul 30, 2018
Merged

[MXNET-641] fix R windows install docs #11805

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
844124c
fix R windows install docs
ankkhedia Jul 18, 2018
d8dbb98
Merge remote-tracking branch 'upstream/master'
ankkhedia Jul 23, 2018
5f66a10
addressed PR comments
ankkhedia Jul 23, 2018
956a4e4
PR comments
ankkhedia Jul 24, 2018
31f7d5e
PR comments
ankkhedia Jul 24, 2018
dad0712
fixed line wrappings
ankkhedia Jul 24, 2018
40a1098
fixed line wrappings
ankkhedia Jul 24, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 127 additions & 42 deletions docs/install/windows_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Next, we install ```graphviz``` library that we use for visualizing network grap

We have installed MXNet core library. Next, we will install MXNet interface package for programming language of your choice:
- [Python](#install-the-mxnet-package-for-python)
- [R](#install-mxnet-for-r)
- [R](#install-mxnet-package-for-r)
- [Julia](#install-the-mxnet-package-for-julia)
- **Scala** is not yet available for Windows

Expand Down Expand Up @@ -91,7 +91,7 @@ Done! We have installed MXNet with Python interface. Run below commands to verif
```
We actually did a small tensor computation using MXNet! You are all set with MXNet on your Windows machine.

## Install MXNet for R
## Install MXNet Package for R
MXNet for R is available for both CPUs and GPUs.

### Installing MXNet on a Computer with a CPU Processor
Expand All @@ -101,7 +101,7 @@ To install MXNet on a computer with a CPU processor, choose from two options:
* Use the prebuilt binary package
* Build the library from source code

#### Installing MXNet with the Prebuilt Binary Package
#### Installing MXNet with the Prebuilt Binary Package(CPU)
For Windows users, MXNet provides prebuilt binary packages.
You can install the package directly in the R console.

Expand All @@ -114,81 +114,166 @@ For CPU-only package:
install.packages("mxnet")
```

For GPU-enabled package:
#### Building MXNet from Source Code(CPU)
1. Clone the MXNet github repo.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

View the markdown output. This numbering is wrapping instead of using new lines.


```r
cran <- getOption("repos")
cran["dmlc"] <- "https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU"
options(repos = cran)
install.packages("mxnet")
```sh
git clone --recursive https://github.com/apache/incubator-mxnet
```

#### Building MXNet from Source Code
The `--recursive` is to clone all the submodules used by MXNet. You will be editing the ```"/mxnet/R-package"``` folder.
2. Download prebuilt GPU-enabled MXNet libraries for Windows from [Windows release](https://github.com/yajiedesign/mxnet/releases). You will need `mxnet_x64_vc14_cpu.7z` and `prebuildbase_win10_x64_vc14.7z` where X stands for your CUDA toolkit version
3. Create a folder called ```R-package/inst/libs/x64```. MXNet supports only 64-bit operating systems, so you need the x64 folder.
4. Copy the following shared libraries (.dll files) into the ```R-package/inst/libs/x64``` folder:
```
libgcc_s_seh-1.dll
libgfortran-3.dll
libmxnet.dll
libmxnet.lib
libopenblas.dll
libquadmath-0.dll
mxnet.dll
unzip.exe
unzip32.dll
vcomp140.dll
wget.exe
```
These dlls can be found in `prebuildbase_win10_x64_vc14/3rdparty`, `mxnet_x64_vc14_cpu/build`, `mxnet_x64_vc14_cpu/lib`.

Run the following commands to install the MXNet dependencies and build the MXNet R package.
5. Copy the header files from `dmlc`, `mxnet`, `mxshadow` and `nnvm` from mxnet_x64_vc14_cpu/include and mxnet_x64_vc14_cpu/nvnm/include into `./R-package/inst/include`. It should look like:

```r
Rscript -e "install.packages('devtools', repo = 'https://cloud.r-project.org/')"
```
./R-package/inst
└── include
├── dmlc
├── mxnet
├── mshadow
└── nnvm

```
6. Make sure that R executable is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
7. Also make sure that Rtools is installed and the executable is added to your ```PATH``` in the environment variables.
8. Temporary patch - im2rec currently results in crashes during the build. Remove the im2rec.h and im2rec.cc files in R-package/src/ from cloned repository and comment out the two im2rec lines in [R-package/src/mxnet.cc](https://github.com/apache/incubator-mxnet/blob/master/R-package/src/mxnet.cc) as shown below.
```bat
#include "./kvstore.h"
#include "./export.h"
//#include "./im2rec.h"
......
......
DataIterCreateFunction::InitRcppModule();
KVStore::InitRcppModule();
Exporter::InitRcppModule();
// IM2REC::InitRcppModule();
}

```bash
cd R-package
Rscript -e "library(devtools); library(methods); options(repos=c(CRAN='https://cloud.r-project.org/')); install_deps(dependencies = TRUE)"
cd ..
make rpkg
```

9. Now open the Windows CMD with admin rights and change the directory to the `mxnet` folder(cloned repository). Then use the following commands
to build R package:

```bat
echo import(Rcpp) > R-package\NAMESPACE
echo import(methods) >> R-package\NAMESPACE
Rscript -e "install.packages('devtools', repos = 'https://cloud.r-project.org')"
cd R-package
Rscript -e "library(devtools); library(methods); options(repos=c(CRAN='https://cloud.r-project.org')); install_deps(dependencies = TRUE)"
cd ..

R CMD INSTALL --no-multiarch R-package

Rscript -e "require(mxnet); mxnet:::mxnet.export('R-package')"
rm R-package/NAMESPACE
Rscript -e "require(devtools); install_version('roxygen2', version = '5.0.1', repos = 'https://cloud.r-project.org/', quiet = TRUE)"
Rscript -e "require(roxygen2); roxygen2::roxygenise('R-package')"

R CMD INSTALL --build --no-multiarch R-package
```


### Installing MXNet on a Computer with a GPU Processor
To install MXNet on a computer with a GPU processor, choose from two options:

To install MXNet R package on a computer with a GPU processor, you need the following:
* Use the prebuilt binary package
* Build the library from source code

* Microsoft Visual Studio 2013
However, few dependencies remains same for both options. You will need the following:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few dependencies remain for both options.

* Install [Nvidia-drivers](http://www.nvidia.com/Download/index.aspx?lang=en-us) if not installed. Latest driver based on your system configuration is recommended.

* The NVidia CUDA Toolkit
* Install [Microsoft Visual Studio](https://visualstudio.microsoft.com/downloads/) (VS2015 or VS2017 is required by CUDA)

* The MXNet package
* Install [NVidia CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit)(cu92 is recommended though we support cu80, cu90, cu91 and cu92)

* CuDNN (to provide a Deep Neural Network library)
* Download and install [CuDNN](https://developer.nvidia.com/cudnn) (to provide a Deep Neural Network library). Latest version recommended.

To install the required dependencies and install MXNet for R:
Note: A pre-requisite to above softwares is [Nvidia-drivers](http://www.nvidia.com/Download/index.aspx?lang=en-us) which we assume is installed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not list as a regular dependency and not a Note?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually it comes pre-installed with Windows system, however this needs to be an additional step in Windows Server images on ec2. So, added as note

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again it might be very helpful to identify a version number. I've had issues with this before and it took a lot of digging to figure out that I needed a driver upgrade.


#### Installing MXNet with the Prebuilt Binary Package(GPU)
For Windows users, MXNet provides prebuilt binary packages.
You can install the package directly in the R console after you have the above software installed.

1. Install the [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit). The CUDA Toolkit depends on Visual Studio. To check whether your GPU is compatible with the CUDA Toolkit and for information on installing it, see NVidia's [CUDA Installation Guide](http://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/).
3. Clone the MXNet github repo.
For GPU package:

```r
cran <- getOption("repos")
cran["dmlc"] <- "https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cuX"
options(repos = cran)
install.packages("mxnet")
```
Change X to 80,90,91 or 92 based on your CUDA toolkit version. Currently, MXNet supports these versions of CUDA.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there's some issue with 9.1 and it is not recommended.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does one check their current version #? If you are installing everything for the first time, maybe the recommendation should be made for the user to install 9.2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested 9.2 for first time installation

#### Building MXNet from Source Code(GPU)
After you have installed above software, continue with the following steps to build MXNet-R:
1. Clone the MXNet github repo.

```sh
git clone --recursive https://github.com/dmlc/mxnet
git clone --recursive https://github.com/apache/incubator-mxnet
```

The `--recursive` is to clone all the submodules used by MXNet. You will be editing the ```"/mxnet/R-package"``` folder.
4. Download prebuilt GPU-enabled MXNet libraries for Windows from https://github.com/yajiedesign/mxnet/releases. You will need `mxnet_x64_vc14_gpu.7z` and `prebuildbase_win10_x64_vc14.7z`.
5. Download and install [CuDNN](https://developer.nvidia.com/cudnn).
6. Create a folder called ```R-package/inst/libs/x64```. MXNet supports only 64-bit operating systems, so you need the x64 folder.
7. Copy the following shared libraries (.dll files) into the ```R-package/inst/libs/x64``` folder:
```
cublas64_80.dll
cudart64_80.dll
cudnn64_5.dll
curand64_80.dll
2. Download prebuilt GPU-enabled MXNet libraries for Windows from https://github.com/yajiedesign/mxnet/releases. You will need `mxnet_x64_vc14_gpu_cuX.7z` and `prebuildbase_win10_x64_vc14.7z` where X stands for your CUDA toolkit version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems odd. Aren't these are binaries for VS2015 (vc14), but the prerequisite was for VS2017? Is there a binary tagged with vc15 available?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added that both 2015 and 2017 are supported

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numbering is wrapping here too.

3. Create a folder called ```R-package/inst/libs/x64```. MXNet supports only 64-bit operating systems, so you need the x64 folder.
4. Copy the following shared libraries (.dll files) into the ```R-package/inst/libs/x64``` folder:
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why were these removed -
-cublas64_80.dll
-cudart64_80.dll
-cudnn64_5.dll
-curand64_80.dll

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't need to be there in the R package libraries,it is taken from cuda toolkit installation and not here

libgcc_s_seh-1.dll
libgfortran-3.dll
libmxnet.dll
libmxnet.lib
libopenblas.dll
libquadmath-0.dll
nvrtc64_80.dll
mxnet.dll
unzip.exe
unzip32.dll
vcomp140.dll
wget.exe
```
These dlls can be found in `prebuildbase_win10_x64_vc14/3rdparty/cudart`, `prebuildbase_win10_x64_vc14/3rdparty/openblas/bin`, `mxnet_x64_vc14_gpu/build`, `mxnet_x64_vc14_gpu/lib` and the `cuDNN` downloaded from NVIDIA.
8. Copy the header files from `dmlc`, `mxnet` and `nnvm` into `./R-package/inst/include`. It should look like:
These dlls can be found in `prebuildbase_win10_x64_vc14/3rdparty`, `mxnet_x64_vc14_gpu_cuX/build`, `mxnet_x64_vc14_gpu_cuX/lib`.
5. Copy the header files from `dmlc`, `mxnet`, `mxshadow` and `nnvm` from mxnet_x64_vc14_gpuX/include and mxnet_x64_vc14_gpuX/nvnm/include into `./R-package/inst/include`. It should look like:

```
./R-package/inst
└── include
├── dmlc
├── mxnet
└── nnvm
├── mshadow
└── nnvm

```
6. Make sure that R executable is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
7. Also make sure that Rtools is installed and the executable is added to your ```PATH``` in the environment variables.
8. Temporary patch - im2rec currently results in crashes during the build. Remove the im2rec.h and im2rec.cc files in R-package/src/ from cloned repository and comment out the two im2rec lines in [R-package/src/mxnet.cc](https://github.com/apache/incubator-mxnet/blob/master/R-package/src/mxnet.cc) as shown below.
```bat
#include "./kvstore.h"
#include "./export.h"
//#include "./im2rec.h"
......
......
DataIterCreateFunction::InitRcppModule();
KVStore::InitRcppModule();
Exporter::InitRcppModule();
// IM2REC::InitRcppModule();
}

```
9. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
10. Now open the Windows CMD and change the directory to the `mxnet` folder. Then use the following commands
9. Now open the Windows CMD with admin rights and change the directory to the `mxnet` folder(cloned repository). Then use the following commands
to build R package:

```bat
Expand Down