Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-641] fix R windows install docs #11805

Merged
merged 7 commits into from
Jul 30, 2018
Merged

[MXNET-641] fix R windows install docs #11805

Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
844124c
fix R windows install docs
ankkhedia Jul 18, 2018
d8dbb98
Merge remote-tracking branch 'upstream/master'
ankkhedia Jul 23, 2018
5f66a10
addressed PR comments
ankkhedia Jul 23, 2018
956a4e4
PR comments
ankkhedia Jul 24, 2018
31f7d5e
PR comments
ankkhedia Jul 24, 2018
dad0712
fixed line wrappings
ankkhedia Jul 24, 2018
40a1098
fixed line wrappings
ankkhedia Jul 24, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 99 additions & 42 deletions docs/install/windows_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Next, we install ```graphviz``` library that we use for visualizing network grap

We have installed MXNet core library. Next, we will install MXNet interface package for programming language of your choice:
- [Python](#install-the-mxnet-package-for-python)
- [R](#install-mxnet-for-r)
- [R](#install-mxnet-package-for-r)
- [Julia](#install-the-mxnet-package-for-julia)
- **Scala** is not yet available for Windows

Expand Down Expand Up @@ -91,7 +91,7 @@ Done! We have installed MXNet with Python interface. Run below commands to verif
```
We actually did a small tensor computation using MXNet! You are all set with MXNet on your Windows machine.

## Install MXNet for R
## Install MXNet Package for R
MXNet for R is available for both CPUs and GPUs.

### Installing MXNet on a Computer with a CPU Processor
Expand All @@ -101,7 +101,7 @@ To install MXNet on a computer with a CPU processor, choose from two options:
* Use the prebuilt binary package
* Build the library from source code

#### Installing MXNet with the Prebuilt Binary Package
#### Installing MXNet with the Prebuilt Binary Package(CPU)
For Windows users, MXNet provides prebuilt binary packages.
You can install the package directly in the R console.

Expand All @@ -114,81 +114,138 @@ For CPU-only package:
install.packages("mxnet")
```

For GPU-enabled package:
#### Building MXNet from Source Code(CPU)
1. Clone the MXNet github repo.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

View the markdown output. This numbering is wrapping instead of using new lines.


```r
cran <- getOption("repos")
cran["dmlc"] <- "https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU"
options(repos = cran)
install.packages("mxnet")
```sh
git clone --recursive https://github.com/apache/incubator-mxnet
```

#### Building MXNet from Source Code
The `--recursive` is to clone all the submodules used by MXNet. You will be editing the ```"/mxnet/R-package"``` folder.
2. Download prebuilt GPU-enabled MXNet libraries for Windows from [Windows release](https://github.com/yajiedesign/mxnet/releases). You will need `mxnet_x64_vc14_cpu.7z` and `prebuildbase_win10_x64_vc14.7z` where X stands for your CUDA toolkit version
3. Create a folder called ```R-package/inst/libs/x64```. MXNet supports only 64-bit operating systems, so you need the x64 folder.
4. Copy the following shared libraries (.dll files) into the ```R-package/inst/libs/x64``` folder:
```
libgcc_s_seh-1.dll
libgfortran-3.dll
libmxnet.dll
libmxnet.lib
libopenblas.dll
libquadmath-0.dll
mxnet.dll
unzip.exe
unzip32.dll
vcomp140.dll
wget.exe
```
These dlls can be found in `prebuildbase_win10_x64_vc14/3rdparty`, `mxnet_x64_vc14_cpu/build`, `mxnet_x64_vc14_cpu/lib`.

Run the following commands to install the MXNet dependencies and build the MXNet R package.
5. Copy the header files from `dmlc`, `mxnet`, `mxshadow` and `nnvm` from mxnet_x64_vc14_cpu/include and mxnet_x64_vc14_cpu/nvnm/include into `./R-package/inst/include`. It should look like:

```r
Rscript -e "install.packages('devtools', repo = 'https://cloud.r-project.org/')"
```
./R-package/inst
└── include
├── dmlc
├── mxnet
├── mshadow
└── nnvm

```
6. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
7. Also make sure that Rtools in installed and added to your ```PATH``` in the environment variables.
8. Temporary patch - im2rec currently results in crashes during the build. Remove the im2rec.h and im2rec.cc files in R-package/src/ from cloned repository and comment out the two im2rec lines in R-package/src/mxnet.cc.
9. Now open the Windows CMD with admin rights and change the directory to the `mxnet` folder(cloned repository). Then use the following commands
to build R package:

```bash
cd R-package
Rscript -e "library(devtools); library(methods); options(repos=c(CRAN='https://cloud.r-project.org/')); install_deps(dependencies = TRUE)"
cd ..
make rpkg
```bat
echo import(Rcpp) > R-package\NAMESPACE
echo import(methods) >> R-package\NAMESPACE
Rscript -e "install.packages('devtools', repos = 'https://cloud.r-project.org')"
cd R-package
Rscript -e "library(devtools); library(methods); options(repos=c(CRAN='https://cloud.r-project.org')); install_deps(dependencies = TRUE)"
cd ..

R CMD INSTALL --no-multiarch R-package

Rscript -e "require(mxnet); mxnet:::mxnet.export('R-package')"
rm R-package/NAMESPACE
Rscript -e "require(devtools); install_version('roxygen2', version = '5.0.1', repos = 'https://cloud.r-project.org/', quiet = TRUE)"
Rscript -e "require(roxygen2); roxygen2::roxygenise('R-package')"

R CMD INSTALL --build --no-multiarch R-package
```


### Installing MXNet on a Computer with a GPU Processor
To install MXNet on a computer with a GPU processor, choose from two options:

* Use the prebuilt binary package
* Build the library from source code

However, few dependencies remains same for both options. You will need the following:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few dependencies remain for both options.


To install MXNet R package on a computer with a GPU processor, you need the following:
* Install [Microsoft Visual Studio 2017](https://visualstudio.microsoft.com/downloads/) (required by CUDA)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several other MXNet-related instructions ask for 2015 (MKL-DNN for example). I think we should have both here and identify any nuance in the setups

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed


* Microsoft Visual Studio 2013
* Install [NVidia CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit)

* The NVidia CUDA Toolkit
* Download and install [CuDNN 7](https://developer.nvidia.com/cudnn) (to provide a Deep Neural Network library)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we specify a minor version, or at least say cuDNN > 7.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggested to use latest version


* The MXNet package
Note: A pre-requisite to above softwares is [Nvidia-drivers](http://www.nvidia.com/Download/index.aspx?lang=en-us) which we assume is installed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not list as a regular dependency and not a Note?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually it comes pre-installed with Windows system, however this needs to be an additional step in Windows Server images on ec2. So, added as note

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again it might be very helpful to identify a version number. I've had issues with this before and it took a lot of digging to figure out that I needed a driver upgrade.


* CuDNN (to provide a Deep Neural Network library)
#### Installing MXNet with the Prebuilt Binary Package(GPU)
For Windows users, MXNet provides prebuilt binary packages.
You can install the package directly in the R console after you have the above softwares installed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

software (it's both singular and plural)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also add a period at the end.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out. Taken care


To install the required dependencies and install MXNet for R:
For GPU package:

1. Install the [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit). The CUDA Toolkit depends on Visual Studio. To check whether your GPU is compatible with the CUDA Toolkit and for information on installing it, see NVidia's [CUDA Installation Guide](http://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/).
3. Clone the MXNet github repo.
```r
cran <- getOption("repos")
cran["dmlc"] <- "https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/CRAN/GPU/cuX"
options(repos = cran)
install.packages("mxnet")
```
Change X to 80,90,91 or 92 based on your CUDA toolkit version. Currently, MXNet supports these versions of CUDA.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there's some issue with 9.1 and it is not recommended.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does one check their current version #? If you are installing everything for the first time, maybe the recommendation should be made for the user to install 9.2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested 9.2 for first time installation

#### Building MXNet from Source Code(GPU)
After you have installed above softwares
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

software, continue with the following steps to build MXNet:

1. Clone the MXNet github repo.

```sh
git clone --recursive https://github.com/dmlc/mxnet
git clone --recursive https://github.com/apache/incubator-mxnet
```

The `--recursive` is to clone all the submodules used by MXNet. You will be editing the ```"/mxnet/R-package"``` folder.
4. Download prebuilt GPU-enabled MXNet libraries for Windows from https://github.com/yajiedesign/mxnet/releases. You will need `mxnet_x64_vc14_gpu.7z` and `prebuildbase_win10_x64_vc14.7z`.
5. Download and install [CuDNN](https://developer.nvidia.com/cudnn).
6. Create a folder called ```R-package/inst/libs/x64```. MXNet supports only 64-bit operating systems, so you need the x64 folder.
7. Copy the following shared libraries (.dll files) into the ```R-package/inst/libs/x64``` folder:
```
cublas64_80.dll
cudart64_80.dll
cudnn64_5.dll
curand64_80.dll
2. Download prebuilt GPU-enabled MXNet libraries for Windows from https://github.com/yajiedesign/mxnet/releases. You will need `mxnet_x64_vc14_gpu_cuX.7z` and `prebuildbase_win10_x64_vc14.7z` where X stands for your CUDA toolkit version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems odd. Aren't these are binaries for VS2015 (vc14), but the prerequisite was for VS2017? Is there a binary tagged with vc15 available?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added that both 2015 and 2017 are supported

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numbering is wrapping here too.

3. Create a folder called ```R-package/inst/libs/x64```. MXNet supports only 64-bit operating systems, so you need the x64 folder.
4. Copy the following shared libraries (.dll files) into the ```R-package/inst/libs/x64``` folder:
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why were these removed -
-cublas64_80.dll
-cudart64_80.dll
-cudnn64_5.dll
-curand64_80.dll

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't need to be there in the R package libraries,it is taken from cuda toolkit installation and not here

libgcc_s_seh-1.dll
libgfortran-3.dll
libmxnet.dll
libmxnet.lib
libopenblas.dll
libquadmath-0.dll
nvrtc64_80.dll
mxnet.dll
unzip.exe
unzip32.dll
vcomp140.dll
wget.exe
```
These dlls can be found in `prebuildbase_win10_x64_vc14/3rdparty/cudart`, `prebuildbase_win10_x64_vc14/3rdparty/openblas/bin`, `mxnet_x64_vc14_gpu/build`, `mxnet_x64_vc14_gpu/lib` and the `cuDNN` downloaded from NVIDIA.
8. Copy the header files from `dmlc`, `mxnet` and `nnvm` into `./R-package/inst/include`. It should look like:
These dlls can be found in `prebuildbase_win10_x64_vc14/3rdparty`, `mxnet_x64_vc14_gpu_cuX/build`, `mxnet_x64_vc14_gpu_cuX/lib`.
5. Copy the header files from `dmlc`, `mxnet`, `mxshadow` and `nnvm` from mxnet_x64_vc14_gpuX/include and mxnet_x64_vc14_gpuX/nvnm/include into `./R-package/inst/include`. It should look like:

```
./R-package/inst
└── include
├── dmlc
├── mxnet
└── nnvm
├── mshadow
└── nnvm

```
9. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
10. Now open the Windows CMD and change the directory to the `mxnet` folder. Then use the following commands
6. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure that the R executable is....

7. Also make sure that Rtools in installed and added to your ```PATH``` in the environment variables.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rtools executable (or folder path if there's more than one in there...)

8. Temporary patch - im2rec currently results in crashes during the build. Remove the im2rec.h and im2rec.cc files in R-package/src/ from cloned repository and comment out the two im2rec lines in R-package/src/mxnet.cc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to show the two lines. These seems kind of vague and easy to mess up.

9. Now open the Windows CMD with admin rights and change the directory to the `mxnet` folder(cloned repository). Then use the following commands
to build R package:

```bat
Expand Down