Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Issue on MXNet R installation with GPU support on Windows #8936

Closed
nicolasjamet opened this issue Dec 4, 2017 · 13 comments
Closed

Issue on MXNet R installation with GPU support on Windows #8936

nicolasjamet opened this issue Dec 4, 2017 · 13 comments

Comments

@nicolasjamet
Copy link

Description

When following the installation tutorial : https://gist.github.com/thirdwing/89aa9bfc588ade138496e6932072152c, the step : R CMD INSTALL --no-multiarch R-package returns
"Error: compilation failed for package 'mxnet'" ( several errors including : compiler and library support for the ISO C++ 2011 standard, im2rec.cc:20:30: fatal error, make: *** [im2rec.o] Error 1,...)

Environment info (Required)

Windows 10
GPU : GeForce GTX 1080 Ti
R 3.3.2
Rtools 3.4
Visual Studio 14
Cuda 9
Cudnn 7
OpenCV 3.3.1
prebuilt GPU-enabled MXNet libraries :
-> 20171204_mxnet_x64_vc141_gpu_cu90
-> prebuildbase_win10_x64_vc14_v2

All environment variables seem to be well defined.
For Rtools, paths are : C:\Rtools\mingw_64\bin & C:\Rtools\bin

R sessionInfo:

R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=French_Switzerland.1252 LC_CTYPE=French_Switzerland.1252 LC_MONETARY=French_Switzerland.1252
[4] LC_NUMERIC=C LC_TIME=French_Switzerland.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached):
[1] tools_3.3.2

Error Message:

c:\mxnet>R CMD INSTALL --no-multiarch R-package

  • installing to library 'C:/R-3.3.2/library'
  • installing source package 'mxnet' ...
    ** libs
    c:/Rtools/mingw_64/bin/g++ -I"C:/R-3.3.2/include" -DNDEBUG -I../inst/include -I"C:/R-3.3.2/library/Rcpp/include" -I"C:/TDM-GCC-64//include" -O2 -Wall -mtune=core2 -c executor.cc -o executor.o
    In file included from executor.cc:9:0:
    ./base.h: In function 'std::string mxnet::R::toPyString(const string&, const RObject&)':
    ./base.h:276:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < vec.size(); ++i) {
    ^
    ./base.h: In function 'std::vector mxnet::R::Dim2InternalShape(const Rcpp::Dimension&)':
    ./base.h:340:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < rshape.size(); ++i) {
    ^
    executor.cc: In static member function 'static void mxnet::R::Executor::UpdateArray(const char*, const List&, Rcpp::List*, bool, bool)':
    executor.cc:43:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < from.size(); ++i) {
    ^
    executor.cc: In static member function 'static Rcpp::List mxnet::R::Executor::CloneArray(const List&)':
    executor.cc:87:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < src.size(); ++i) {
    ^
    executor.cc: In function 'Rcpp::List* mxnet::R::CreateArrayList(const List&, const string&, const std::vector<std::basic_string >&, const RObjectType&, std::vector<void*>)':
    executor.cc:123:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < source_array.size(); ++i) {
    ^
    executor.cc: In function 'Rcpp::List
    mxnet::R::CreateGradList(const List&, const List&, const std::vector<std::basic_string >&, const RObjectType&, std::vector<void*>, std::vector)':
    executor.cc:155:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < grad_reqs.size(); ++i) {
    ^
    c:/Rtools/mingw_64/bin/g++ -I"C:/R-3.3.2/include" -DNDEBUG -I../inst/include -I"C:/R-3.3.2/library/Rcpp/include" -I"C:/TDM-GCC-64//include" -O2 -Wall -mtune=core2 -c export.cc -o export.o
    In file included from export.cc:9:0:
    ./base.h: In function 'std::string mxnet::R::toPyString(const string&, const RObject&)':
    ./base.h:276:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < vec.size(); ++i) {
    ^
    ./base.h: In function 'std::vector mxnet::R::Dim2InternalShape(const Rcpp::Dimension&)':
    ./base.h:340:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < rshape.size(); ++i) {
    ^
    export.cc: In static member function 'static void mxnet::R::Exporter::Export(const string&)':
    export.cc:92:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
    for (size_t i = 0; i < func_names.size(); ++i) {
    ^
    c:/Rtools/mingw_64/bin/g++ -I"C:/R-3.3.2/include" -DNDEBUG -I../inst/include -I"C:/R-3.3.2/library/Rcpp/include" -I"C:/TDM-GCC-64//include" -O2 -Wall -mtune=core2 -c im2rec.cc -o im2rec.o
    In file included from C:/Rtools/mingw_64/x86_64-w64-mingw32/include/c++/random:35:0,
    from im2rec.cc:14:
    C:/Rtools/mingw_64/x86_64-w64-mingw32/include/c++/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
    #error This file requires compiler and library support for the
    ^

    In file included from ../inst/include/dmlc/io.h:14:0,
    from im2rec.cc:16:
    ../inst/include/dmlc/./logging.h:67:3: warning: identifier 'nullptr' is a keyword in C++11 [-Wc++0x-compat]
    LogCheckError() : str(nullptr) {}
    ^
    im2rec.cc:20:30: fatal error: opencv2/opencv.hpp: No such file or directory
    #include <opencv2/opencv.hpp>

    ^
    compilation terminated.
    make: *** [im2rec.o] Error 1
    Warning: l'exécution de la commande 'make -f "Makevars.win" -f "C:/R-3.3.2/etc/x64/Makeconf" -f "C:/R-3.3.2/share/make/winshlib.mk" SHLIB_LDFLAGS='$(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="mxnet.dll" WIN=64 TCLBIN=64 OBJECTS="executor.o export.o im2rec.o io.o kvstore.o mxnet.o ndarray.o symbol.o"' renvoie un statut 2
    ERROR: compilation failed for package 'mxnet'
  • removing 'C:/R-3.3.2/library/mxnet'
@nicolasjamet nicolasjamet changed the title Issue with MXNet R installation with GPU support on Windows Issue on MXNet R installation with GPU support on Windows Dec 4, 2017
@mjmg
Copy link

mjmg commented Dec 4, 2017

Isn't your R version quite old compared to the other toolchains released this year?

R 3.3.2 (released OCT 2016)
Rtools 3.4
Visual Studio 14
Cuda 9
Cudnn 7

Have you tried using R 3.4.X (which has better support for C++11) to see if the errors disappeared? 3.4.3 was just released last Nov. 30
https://stat.ethz.ch/pipermail/r-announce/2017/000623.html

@nicolasjamet
Copy link
Author

Thanks for your answer. I've installed R 3.4.2 and It returns the same error message ( I also tried with R 3.4.3 but It returns bin\x64\Rscript.exe" -e "install.packages' is not recognized as an internal or external command, operable program or batch file. with the command Rscript -e "install.packages)

@mjmg
Copy link

mjmg commented Dec 5, 2017

These lines seems suspect:

c:/Rtools/mingw_64/bin/g++ -I"C:/R-3.3.2/include" -DNDEBUG -I../inst/include -I"C:/R-3.3.2/library/Rcpp/include" -I"C:/TDM-GCC-64//include" -O2 -Wall -mtune=core2 -c executor.cc -o executor.o
...
c:/Rtools/mingw_64/bin/g++ -I"C:/R-3.3.2/include" -DNDEBUG -I../inst/include -I"C:/R-3.3.2/library/Rcpp/include" -I"C:/TDM-GCC-64//include" -O2 -Wall -mtune=core2 -c im2rec.cc -o im2rec.o

"C:/TDM-GCC-64//include" indicates you have the TDM GCC compiler include files referenced in your Include path. Rtools provides its own complete GCC toolchain. Rtools might attempt to include header files in the TDM GCC installation path which can lead to conflicts.

This one:

im2rec.cc:20:30: fatal error: opencv2/opencv.hpp: No such file or directory
#include <opencv2/opencv.hpp>

seems to indicate problems finding the include path of the opencv installation.

@nicolasjamet
Copy link
Author

I have uninstalled TDM-GCC-64. The lines are now as below. It seems that the path "d:/Compiler/gcc-4.9.3/local330/include" is taken from Makeconf file in C:\R-3.4.2\etc\x64. This path doesn't exist - should I replace it with Rtools path?

c:/Rtools/mingw_64/bin/g++ -I"C:/R-3.4.2/include" -DNDEBUG -I../inst/include -I"C:/R-3.4.2/library/Rcpp/include" -I"d:/Compiler/gcc-4.9.3/local330/include" -O2 -Wall -mtune=core2 -c export.cc -o export.o
c:/Rtools/mingw_64/bin/g++ -I"C:/R-3.4.2/include" -DNDEBUG -I../inst/include -I"C:/R-3.4.2/library/Rcpp/include" -I"d:/Compiler/gcc-4.9.3/local330/include" -O2 -Wall -mtune=core2 -c im2rec.cc -o im2rec.o

Regarding opencv, I referenced the following :

Within Path variable :
C:\Program Files\opencv\build\bin
C:\Program Files\opencv\build\include
C:\Program Files\opencv\build\include\opencv2

And as System Variables
OPENCV_DIR = C:\Program Files\opencv\build\bin
OPENCV_DIR_BUILD = C:\Program Files\opencv\build\include
OPENCV_DIR_OPENCV2 = C:\Program Files\opencv\build\include\opencv2

@mjmg
Copy link

mjmg commented Dec 6, 2017

It seems most of your problems are due to environment variables and their order.

Make sure Rtools is the highest priority followed by opencv. "d:/Compiler/gcc-4.9.3/local330/include" should not be in the paths.

For reference maybe should should enumerate all your environment variables to check problems in their order.

@nicolasjamet
Copy link
Author

Path variable :

C:\Rtools\bin
C:\Rtools\mingw_64\bin
C:\Program Files\opencv\build
C:\Program Files\opencv\build\bin
C:\Program Files\opencv\build\include
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp
C:\NVIDIA\CUDNN\cudnn-9.0-windows10-x64-v7
C:\Program Files\Microsoft MPI\Bin
C:\ProgramData\Oracle\Java\javapath
C:\Program Files (x86)\Intel\iCLS Client
C:\Program Files\Intel\iCLS Client
C:\windows\system32
C:\windows
C:\windows\System32\Wbem
C:\windows\System32\WindowsPowerShell\v1.0
C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL
C:\Program Files\Intel\Intel(R) Management Engine Components\DAL
C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT
C:\Program Files\Intel\Intel(R) Management Engine Components\IPT
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common
C:\Program Files\Git\cmd
C:\WINDOWS\system32
C:\WINDOWS
C:\WINDOWS\System32\Wbem
C:\WINDOWS\System32\WindowsPowerShell\v1.0
C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018.1.156\windows\mkl
C:\Program Files\CMake\bin
C:\Users\admnja.dnx\bin
C:\Program Files\Microsoft DNX\Dnvm
C:\Program Files\Microsoft SQL Server\130\Tools\Binn
C:\Graphviz\bin
C:\R-3.4.2
C:\R-3.4.2\bin
C:\R-3.4.2\bin\x64\

Other system variables :

CUDA_BIN_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin
CUDA_LIBNVVP_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp
CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0
CUDNN_PATH=C:\NVIDIA\CUDNN\cudnn-9.0-windows10-x64-v7
HOMEDRIVE=C:
HOMEPATH=\Users\nja
INTEL_LICENSE_FILE=C:\Program Files (x86)\Common Files\Intel\Licenses
LOCALAPPDATA=C:\Users\nja\AppData\Local
LOGONSERVER=\DC01-RAM-GVA
MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin
MXNET_HOME=C:\MXNet\prebuildbase_win10_x64_vc14_v2
NUMBER_OF_PROCESSORS=20
NVCUDASAMPLES9_0_ROOT=C:\ProgramData\NVIDIA Corporation\CUDA Samples\v9.0
NVCUDASAMPLES_ROOT=C:\ProgramData\NVIDIA Corporation\CUDA Samples\v9.0
NVTOOLSEXT_PATH=C:\Program Files\NVIDIA Corporation\NvToolsExt
OneDrive=C:\Users\nja\OneDrive
OpenBLAS_HOME=C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018.1.156\windows\mkl
OPENCV_DIR=C:\Program Files\opencv\build
OPENCV_DIR_BUILD=C:\Program Files\opencv\build\include
OPENCV_DIR_OPENCV2=C:\Program Files\opencv\build\include\opencv2
OS=Windows_NT
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 85 Stepping 4, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=5504
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=$P$G
PUBLIC=C:\Users\Public
PYTHON_EXECUTABLE=C:\Python\python.exe
PYTHON_INCLUDE_DIR=C:\Python\include
PYTHON_LIBRARY=C:\Python\libs\python36.lib
R_PATH=C:\R-3.4.2
SESSIONNAME=Console
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEMP=C:\Users\nja\AppData\Local\Temp
TMP=C:\Users\nja\AppData\Local\Temp
VS140COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools
windir=C:\WINDOWS

@mjmg
Copy link

mjmg commented Dec 7, 2017

Maybe you should reinstall opencv on the root drive C just to rule out errors due to the space in "C:\Program Files\opencv\build" to be sure.

@nicolasjamet
Copy link
Author

Done, but unfortunately it doesn't change the outcome. For reference, in paths I now have:
C:\opencv\build
C:\opencv\build\bin
C:\opencv\build\include

@nicolasjamet
Copy link
Author

Any idea on something else I could try ?

@mayer79
Copy link

mayer79 commented Feb 26, 2018

I have exactly the same problem.

@alihashmiii
Copy link

alihashmiii commented Mar 7, 2018

I am facing the same problem here. why is not the GPU support easier for R mxnet. Python GPU support was so much easier

@leleamol
Copy link
Contributor

Proposed Labels: "Windows", "R","Build","Installation"

@ankkhedia
Copy link
Contributor

Hi @alihashmiii @mayer79 @nicolasjamet
The latest prebuilt mxnet-cu90, cu80, cu91 and cu91 binary for win10 haas been updated. You can take a look at install instructions. Also the instructions for building it from source has been updates. Feel free to try it out and let me know if you face any issue

@sandeep-krishnamurthy Could you please close this issue as it has been resolved.

@alihashmiii @mayer79 @nicolasjamet
Please feel free to reopen if closed in error.

@nswamy nswamy closed this as completed Aug 9, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

8 participants