From 3367b7d18abc38cdc5d7838f257a4ff54f67df46 Mon Sep 17 00:00:00 2001 From: Pedro Larroy Date: Tue, 26 Mar 2019 12:53:40 -0700 Subject: [PATCH 1/4] Update ubuntu install instructions from source --- docs/install/ubuntu_setup.md | 97 +++++++++++++++++++++++++++--------- 1 file changed, 73 insertions(+), 24 deletions(-) diff --git a/docs/install/ubuntu_setup.md b/docs/install/ubuntu_setup.md index f225023d18d5..4bd0efab393e 100644 --- a/docs/install/ubuntu_setup.md +++ b/docs/install/ubuntu_setup.md @@ -140,9 +140,30 @@ On Ubuntu versions 16.04 or later, you need the following dependencies: **Step 1:** Install build tools and git. ```bash sudo apt-get update - sudo apt-get install -y build-essential git + sudo apt-get install -y build-essential git ninja-build ccache ``` +**For Ubuntu 18.04 and CUDA builds you need to update CMake** + +```bash +#!/usr/bin/env bash +set -exuo pipefail +sudo apt remove --purge --auto-remove cmake + +# Update CMAKE for correct cuda autotedetection: https://github.com/clab/dynet/issues/1457 +version=3.14 +build=0 +mkdir -p ~/tmp +cd ~/tmp +wget https://cmake.org/files/v$version/cmake-$version.$build.tar.gz +tar -xzvf cmake-$version.$build.tar.gz +cd cmake-$version.$build/ +./bootstrap +make -j$(nproc) +sudo make install +``` + + **Step 2:** Install a Math Library. Details on the different math libraries are found in the build from source guide's [Math Library Selection](build_from_source.html#math-library-selection) section. @@ -170,44 +191,72 @@ If building on CPU and using OpenBLAS: ```bash git clone --recursive https://github.com/apache/incubator-mxnet.git cd incubator-mxnet - echo "USE_OPENCV = 1" >> ./config.mk - echo "USE_BLAS = openblas" >> ./config.mk - make -j $(nproc) +``` + + +```bash + rm -rf build + mkdir -p build && cd build + cmake -GNinja \ + -DUSE_CUDA=OFF \ + -DUSE_MKL_IF_AVAILABLE=OFF \ + -DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \ + -DCMAKE_C_COMPILER_LAUNCHER=ccache \ + -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ + .. + ninja ``` If building on CPU and using MKL and MKL-DNN (make sure MKL is installed according to [Math Library Selection](build_from_source.html#math-library-selection) and [MKL-DNN README](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md)): ```bash - git clone --recursive https://github.com/apache/incubator-mxnet.git - cd incubator-mxnet - echo "USE_OPENCV = 1" >> ./config.mk - echo "USE_BLAS = openblas" >> ./config.mk - echo "USE_CUDA = 0" >> ./config.mk - echo "USE_MKLDNN = 1" >> ./config.mk - make -j $(nproc) + rm -rf build + mkdir -p build && cd build + cmake -GNinja \ + -DUSE_CUDA=OFF \ + -DUSE_MKL_IF_AVAILABLE=ON \ + -DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \ + -DCMAKE_C_COMPILER_LAUNCHER=ccache \ + -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ + .. + ninja ``` -If building on GPU and you want OpenCV and OpenBLAS (make sure you have installed the [CUDA dependencies first](#cuda-dependencies)): +If building on GPU (make sure you have installed the [CUDA dependencies first](#cuda-dependencies)): +Cuda 10.1 in Ubuntu 18.04 builds fine but is not currently tested in CI. ```bash - git clone --recursive https://github.com/apache/incubator-mxnet.git - cd incubator-mxnet - echo "USE_OPENCV = 1" >> ./config.mk - echo "USE_BLAS = openblas" >> ./config.mk - echo "USE_CUDA = 1" >> ./config.mk - echo "USE_CUDA_PATH = /usr/local/cuda" >> ./config.mk - echo "USE_CUDNN = 1" >> ./config.mk - make -j $(nproc) + rm -rf build + mkdir -p build && cd build + cmake -GNinja \ + -DUSE_CUDA=ON \ + -DUSE_MKL_IF_AVAILABLE=OFF \ + -DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \ + -DCMAKE_C_COMPILER_LAUNCHER=ccache \ + -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ + .. + ninja ``` -*Note* - USE_OPENCV and USE_BLAS are make file flags to set compilation options to use OpenCV and BLAS library. You can explore and use more compilation options in `make/config.mk` and also review common [usage examples](build_from_source.html#usage-examples). +*Note* - You can explore and use more compilation options in `CMakeLists.txt` and also review common [usage examples](build_from_source.html#usage-examples). +You can also use a scripted version of the above with an editable CMake options file by doing the +following: + +```bash +cp cmake/cmake_options.yml . +# Edit to your taste +$EDITOR cmake_options.yml +# Launch a local CMake build +./dev_menu.py menu 1 +``` -Building from source creates a library called ```libmxnet.so``` in the `lib` folder in your MXNet project root. +Building from source creates a library called ```libmxnet.so``` in the `build` folder in your MXNet project root. -You may also want to add the MXNet shared library to your `LD_LIBRARY_PATH`: +You may also want to add the MXNet shared library to your `LD_LIBRARY_PATH` and `MXNET_LIBRARY_PATH` accordingly: ```bash -export LD_LIBRARY_PATH=$PWD/lib +export LD_LIBRARY_PATH=`realpath build` +export MXNET_LIBRARY_PATH=`realpath build/libmxnet.so` ``` After building the MXNet library, you may install language bindings. From 7024cf10289c1b146f5deee7d992e3ee9737f198 Mon Sep 17 00:00:00 2001 From: Aaron Markham Date: Wed, 3 Apr 2019 12:26:09 -0700 Subject: [PATCH 2/4] Update docs/install/ubuntu_setup.md Co-Authored-By: larroy --- docs/install/ubuntu_setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/install/ubuntu_setup.md b/docs/install/ubuntu_setup.md index 4bd0efab393e..7527f608025d 100644 --- a/docs/install/ubuntu_setup.md +++ b/docs/install/ubuntu_setup.md @@ -137,7 +137,7 @@ It is recommended that you review the general [build from source](build_from_sou On Ubuntu versions 16.04 or later, you need the following dependencies: -**Step 1:** Install build tools and git. +**Step 1:** Install prerequisite packages. ```bash sudo apt-get update sudo apt-get install -y build-essential git ninja-build ccache From beab7aba35f2062287f49cbe62a4d6f38d8e5a89 Mon Sep 17 00:00:00 2001 From: Pedro Larroy Date: Wed, 10 Apr 2019 17:13:52 -0700 Subject: [PATCH 3/4] Address CR comments --- docs/install/ubuntu_setup.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/install/ubuntu_setup.md b/docs/install/ubuntu_setup.md index 7527f608025d..c8fd8d0fec3b 100644 --- a/docs/install/ubuntu_setup.md +++ b/docs/install/ubuntu_setup.md @@ -188,11 +188,14 @@ For other libraries, visit the [Math Library Selection](build_from_source.html#m If building on CPU and using OpenBLAS: +Clone the repository: + ```bash git clone --recursive https://github.com/apache/incubator-mxnet.git cd incubator-mxnet ``` +Build with CMake and ninja, without GPU and without MKL. ```bash rm -rf build @@ -239,12 +242,12 @@ Cuda 10.1 in Ubuntu 18.04 builds fine but is not currently tested in CI. ``` *Note* - You can explore and use more compilation options in `CMakeLists.txt` and also review common [usage examples](build_from_source.html#usage-examples). -You can also use a scripted version of the above with an editable CMake options file by doing the +Optionally, you can also use a higher level, scripted version of the above with an editable CMake options file by doing the following: ```bash cp cmake/cmake_options.yml . -# Edit to your taste +# Edit cmake_options.yml in the MXNet root to your taste $EDITOR cmake_options.yml # Launch a local CMake build ./dev_menu.py menu 1 From 1b8821e4a94acf7a0bfec1b9186d12aa728537fb Mon Sep 17 00:00:00 2001 From: Pedro Larroy Date: Mon, 15 Apr 2019 15:41:42 -0700 Subject: [PATCH 4/4] Address CR comments --- docs/install/ubuntu_setup.md | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/docs/install/ubuntu_setup.md b/docs/install/ubuntu_setup.md index c8fd8d0fec3b..01b11cdc11ab 100644 --- a/docs/install/ubuntu_setup.md +++ b/docs/install/ubuntu_setup.md @@ -241,7 +241,7 @@ Cuda 10.1 in Ubuntu 18.04 builds fine but is not currently tested in CI. ninja ``` -*Note* - You can explore and use more compilation options in `CMakeLists.txt` and also review common [usage examples](build_from_source.html#usage-examples). +*Note* - You can explore and use more compilation options as they are delcared in the top of `CMakeLists.txt` and also review common [usage examples](build_from_source.html#usage-examples). Optionally, you can also use a higher level, scripted version of the above with an editable CMake options file by doing the following: @@ -250,18 +250,11 @@ cp cmake/cmake_options.yml . # Edit cmake_options.yml in the MXNet root to your taste $EDITOR cmake_options.yml # Launch a local CMake build -./dev_menu.py menu 1 +./dev_menu.py build ``` Building from source creates a library called ```libmxnet.so``` in the `build` folder in your MXNet project root. -You may also want to add the MXNet shared library to your `LD_LIBRARY_PATH` and `MXNET_LIBRARY_PATH` accordingly: - -```bash -export LD_LIBRARY_PATH=`realpath build` -export MXNET_LIBRARY_PATH=`realpath build/libmxnet.so` -``` - After building the MXNet library, you may install language bindings.