CMake build with MKL_USE_ILP64 throws type mismatch

[dszeto2]

When setting MKL_USE_ILP64 to On in the CMake build, the build fails:

In file included from /home/dszeto2/Desktop/mxnet/src/operator/c_lapack_api.cc:20:0:
/home/dszeto2/Desktop/mxnet/src/operator/c_lapack_api.h: In function `int MXNET_LAPACK_sgetri(int, int, float*, int, int*, float*, int)`:
/home/dszeto2/Desktop/mxnet/src/operator/c_lapack_api.h:249:68: error: cannot convert `int*` to `const long long int*` for argument `5` to `long long int LAPACKE_sgetri(int, long long int, float*, long long int, const long long int*)`
       return LAPACKE_##prefix##getri(matrix_layout, n, a, lda, ipiv); \
                                                                    ^
/home/dszeto2/Desktop/mxnet/src/operator/c_lapack_api.h:254:3: note: in expansion of macro `MXNET_LAPACK_CWRAP_GETRI`
   MXNET_LAPACK_CWRAP_GETRI(s, float)
   ^
/home/dszeto2/Desktop/mxnet/src/operator/c_lapack_api.h: In function`int MXNET_LAPACK_dgetri(int, int, double*, int, int*, double*, int)`:
/home/dszeto2/Desktop/mxnet/src/operator/c_lapack_api.h:249:68: error: cannot convert `int*` to `const long long int*` for argument `5` to `long long int LAPACKE_dgetri(int, long long int, double*, long long int, const long long int*)`
       return LAPACKE_##prefix##getri(matrix_layout, n, a, lda, ipiv); \
                                                                    ^
/home/dszeto2/Desktop/mxnet/src/operator/c_lapack_api.h:255:3: note: in expansion of macro `MXNET_LAPACK_CWRAP_GETRI`
   MXNET_LAPACK_CWRAP_GETRI(d, double)

The rest of my CMake build command is here:

cmake \
-DUSE_CUDA=Off \
-DUSE_CUDNN=Off \
-DUSE_LAPACK=On \
-DUSE_F16C=Off \
-DBLAS=MKL \
-DUSE_OPENCV=Off \
-DUSE_PROFILER=On \
-DUSE_MKLDNN=Off \
-DUSE_OPENMP=On \
-DBUILD_CPP_EXAMPLES=Off \
-DMKL_USE_SINGLE_DYNAMIC_LIBRARY=Off \
-DMKL_USE_STATIC_LIBS=Off \
-DCMAKE_SHARED_LINKER_FLAGS="-lrt" \
-DCMAKE_SKIP_BUILD_RPATH=On \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=On \
-DCMAKE_INSTALL_RPATH="\$ORIGIN" \
-DINTEL_ROOT=/opt/intel \
-DMKL_ROOT=/opt/intel/mkl \
-DMKL_USE_ILP64=On \
-DMKL_USE_CLUSTER=Off \
-DUSE_MKL_IF_AVAILABLE=On \
-DINTEL_RTL_ROOT=/opt/intel/lib/intel64_lin \
-DCMAKE_BUILD_TYPE=Release \
..

I'm using gcc 4.8.2 on CentOS-6 from commit f52ddfd880ad9d0e5a9d26eb41aabdd96fa4ac65. Setting MKL_USE_ILP64 to Off gets mxnet to build and successfully link against libmkl_intel_lp64.so, which is expected. Any help would be appreciated.

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended label(s): Build

[ddavydenko]

@mxnet-label-bot, add [Bug, Build]

[TaoLv]

AFAIK, MXNet doesn't work with ILP64. Could you please elaborate what's the problem you want to solve?

[TaoLv]

cc @yinghu5

[matteosal]

AFAIK, MXNet doesn't work with ILP64. Could you please elaborate what's the problem you want to solve?

@TaoLv we are trying to get ILP64 builds to work. Your comment is very unexpected for us, because we have regularly used mxnet builds with ILP64 in the past without any problem. Also, the same commit referenced in the issue builds and runs just fine with ILP64 on mac. The issue we're having is with linux only.

[TaoLv]

Hi @matteosal I'm not sure I understand your question. Since you're trying to use ILP64, I guess your tensor size is > INT32_MAX. Recently, INT64 tensor size is supported by #14570 via a new compilation flag USE_INT64_TENSOR_SIZE. Have you ever tried this flag?

Even though this flag is set, the linkage of MKL BLAS in mshadow is still LP64 and the gemm API in mshadow is using int. That's what I meant by doesn't work with ILP64.

See https://github.com/apache/incubator-mxnet/blob/master/3rdparty/mshadow/make/mshadow.mk#L93 and https://github.com/apache/incubator-mxnet/blob/master/3rdparty/mshadow/mshadow/dot_engine-inl.h#L281.
@apeforest @pengzhao-intel

[samskalicky]

@matteosal Can you try with the USE_INT64_TENSOR_SIZE flag?

[Zha0q1]

building with ilp64 mkl/openblas is now supported. Setting USE_INT64_TENSOR_SIZE=1 will automatically link to ilp64 mkl now