Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[DOC] Update ubuntu install instructions from source #14534

Merged
merged 4 commits into from
Apr 24, 2019
Merged

[DOC] Update ubuntu install instructions from source #14534

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3367b7d
Update ubuntu install instructions from source
larroy Mar 26, 2019
7024cf1
Update docs/install/ubuntu_setup.md
aaronmarkham Apr 3, 2019
beab7ab
Address CR comments
larroy Apr 11, 2019
1b8821e
Address CR comments
larroy Apr 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 71 additions & 26 deletions docs/install/ubuntu_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,33 @@ It is recommended that you review the general [build from source](build_from_sou

On Ubuntu versions 16.04 or later, you need the following dependencies:

**Step 1:** Install build tools and git.
**Step 1:** Install prerequisite packages.
```bash
sudo apt-get update
sudo apt-get install -y build-essential git
sudo apt-get install -y build-essential git ninja-build ccache
larroy marked this conversation as resolved.
Show resolved Hide resolved
```

**For Ubuntu 18.04 and CUDA builds you need to update CMake**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the implication for older Ubuntu OS? Is there a version >= that must be met?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For older versions the distribution's provided CMake is working.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we clarify that in the doc?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I guess it is fine if there is no action required.


```bash
#!/usr/bin/env bash
set -exuo pipefail
sudo apt remove --purge --auto-remove cmake

# Update CMAKE for correct cuda autotedetection: https://github.com/clab/dynet/issues/1457
version=3.14
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting in version number in the docs makes them go stale pretty fast. Is there a way to consolidate versions in a file so it can be managed there instead of sprinkled throughout code blocks in the docs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine. This is the minimum version that works with ubuntu 18.04 I think is ok to put it there, and people can update it. If you put an indirect reference becomes more messy to maintain.

build=0
mkdir -p ~/tmp
cd ~/tmp
wget https://cmake.org/files/v$version/cmake-$version.$build.tar.gz
tar -xzvf cmake-$version.$build.tar.gz
cd cmake-$version.$build/
./bootstrap
make -j$(nproc)
sudo make install
```


**Step 2:** Install a Math Library.

Details on the different math libraries are found in the build from source guide's [Math Library Selection](build_from_source.html#math-library-selection) section.
Expand All @@ -167,49 +188,73 @@ For other libraries, visit the [Math Library Selection](build_from_source.html#m

If building on CPU and using OpenBLAS:

Clone the repository:

```bash
git clone --recursive https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
echo "USE_OPENCV = 1" >> ./config.mk
echo "USE_BLAS = openblas" >> ./config.mk
make -j $(nproc)
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should put something here instead of having two code blocks back to back... some description...

If building on CPU and using MKL and MKL-DNN (make sure MKL is installed according to [Math Library Selection](build_from_source.html#math-library-selection) and [MKL-DNN README](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md)):
Build with CMake and ninja, without GPU and without MKL.

```bash
git clone --recursive https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
echo "USE_OPENCV = 1" >> ./config.mk
echo "USE_BLAS = openblas" >> ./config.mk
echo "USE_CUDA = 0" >> ./config.mk
echo "USE_MKLDNN = 1" >> ./config.mk
make -j $(nproc)
rm -rf build
mkdir -p build && cd build
cmake -GNinja \
-DUSE_CUDA=OFF \
-DUSE_MKL_IF_AVAILABLE=OFF \
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \
Copy link
Contributor

@apeforest apeforest Apr 9, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not an expert in cmake but is ccache a de facto build option for cmake build system? Can we leave this option to user?

Copy link
Contributor Author

@larroy larroy Apr 11, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs to be enabled explicitly with this flag, is not default. I think we should enable it in our instructions to speed up development process. Otherwise it becomes an obscure trick.

-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
..
ninja
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-DUSE_MKLDNN will be turned on by default in this cmake line? Seems we need change the description in the next section and document the default behavior somewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MKL install requires additional steps in Ubuntu that are not part of the installation, so it's disabled. by -DUSE_MKL_IF_AVAILABLE=OFF even if any other MKL variables are on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TaoLv happy if you can help with MKL and CMake: #14670

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All MKL is disabled, check the end of the line: https://github.com/apache/incubator-mxnet/blob/master/CMakeLists.txt#L31

```

If building on GPU and you want OpenCV and OpenBLAS (make sure you have installed the [CUDA dependencies first](#cuda-dependencies)):
If building on CPU and using MKL and MKL-DNN (make sure MKL is installed according to [Math Library Selection](build_from_source.html#math-library-selection) and [MKL-DNN README](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/mkldnn/MKLDNN_README.md)):

```bash
git clone --recursive https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
echo "USE_OPENCV = 1" >> ./config.mk
echo "USE_BLAS = openblas" >> ./config.mk
echo "USE_CUDA = 1" >> ./config.mk
echo "USE_CUDA_PATH = /usr/local/cuda" >> ./config.mk
echo "USE_CUDNN = 1" >> ./config.mk
make -j $(nproc)
rm -rf build
mkdir -p build && cd build
cmake -GNinja \
-DUSE_CUDA=OFF \
-DUSE_MKL_IF_AVAILABLE=ON \
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
..
ninja
```

*Note* - USE_OPENCV and USE_BLAS are make file flags to set compilation options to use OpenCV and BLAS library. You can explore and use more compilation options in `make/config.mk` and also review common [usage examples](build_from_source.html#usage-examples).
If building on GPU (make sure you have installed the [CUDA dependencies first](#cuda-dependencies)):
Cuda 10.1 in Ubuntu 18.04 builds fine but is not currently tested in CI.

Building from source creates a library called ```libmxnet.so``` in the `lib` folder in your MXNet project root.
```bash
rm -rf build
mkdir -p build && cd build
cmake -GNinja \
-DUSE_CUDA=ON \
-DUSE_MKL_IF_AVAILABLE=OFF \
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
..
ninja
```

You may also want to add the MXNet shared library to your `LD_LIBRARY_PATH`:
*Note* - You can explore and use more compilation options as they are delcared in the top of `CMakeLists.txt` and also review common [usage examples](build_from_source.html#usage-examples).
Optionally, you can also use a higher level, scripted version of the above with an editable CMake options file by doing the
following:

```bash
export LD_LIBRARY_PATH=$PWD/lib
cp cmake/cmake_options.yml .
# Edit cmake_options.yml in the MXNet root to your taste
$EDITOR cmake_options.yml
# Launch a local CMake build
./dev_menu.py build
```

Building from source creates a library called ```libmxnet.so``` in the `build` folder in your MXNet project root.

After building the MXNet library, you may install language bindings.

<hr>
Expand Down