Installation instructions consolidation #12388

aaronmarkham · 2018-08-29T00:25:28Z

Description

Different installation pages were out of date or out of sync. The install page was over-complicated. The build from source instructions weren't really working.

This PR consolidates install info, directs users to where the more recent updates happen to be, and simplifies the basic user install. I started off just trying to fix the C++ setup info, but found so many other issues, I worked on those too.

Features

Fixes a bug that made it so you couldn't preview the dev version's install page without it redirecting you. (options.js)
Updates the build_from_source page
- Point to the OS-focused pages where this information is most up-to-date.
- Added a table that maps links to each binding.
- Leaves the centos/other linux sections there since this info isn't covered elsewhere
- Leaves the NCCL section alone (is this even current info?)
- Links this page in the Download Source section found at the bottom of install
Creates a new C++ setup page that brings together C++ info found around the project.
Refactors the install page.
- Moves the validation section to a new page
- Removes virtualenv (this was repeated multiple times in the file w/o much real value
- Super simplifies the pip install sections
- Links back to each OS's detailed setup guides
- Links to specific install guide sections for Perl and Julia
- and more... the file went from 2531 lines to 1062. So much easier to deal with... probably could take more pruning...
Fixes some issues on the OS guide pages - the Python binding steps were missing

Preview

http://34.201.8.176/versions/cplusplus_install/install/index.html

lebeg · 2018-08-29T09:18:32Z

docs/install/build_from_source.md

@@ -28,184 +32,102 @@ You need C++ build tools and BLAS library to build MXNet shared library. If you
   for Windows) to build the library.


A C++ compiler that supports C++ 11.
G++ (4.8 or later), Clang or Visual Studio 2015 (on Windows) is required.

Can we also make CMake the default (instead of GNU Make and cmake only for Windows)?

I'd be happy to do that, but so many other downstream instructions use make. Are we ready to update all of those too?

Well, maybe it can wait then and be done in a separate PR

lebeg · 2018-08-29T09:20:40Z

docs/install/build_from_source.md

-   ```
+MXNet relies on the
+[BLAS](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms) (Basic
+Linear Algebra Subprograms) library for numerical computations. You can install


... library for numerical computations. Those can be extended with the LAPACK (Linear Algebra Package) - an additional set of mathematical functions.

seems like this would be a good spot to mention mkl-dnn? Also could link to those instructions within the project... wdyt?

lebeg · 2018-08-29T09:21:53Z

docs/install/build_from_source.md

+Linear Algebra Subprograms) library for numerical computations. You can install
+any one among [ATLAS](http://math-atlas.sourceforge.net/),
+[OpenBLAS](http://www.openblas.net/) and
+[MKL](https://software.intel.com/en-us/intel-mkl).


[Accelerate](https://developer.apple.com/documentation/accelerate) on MAC OSX.

lebeg · 2018-08-29T09:23:54Z

docs/install/build_from_source.md

-</div>
+### Other Linux
+Installing both `git` and `make` by following instructions on the websites is
+straightforward. Here we provide the instructions to build `gcc-4.8` from source codes.


Are we sure that we need instructions how to build gcc? Is there any use case where it can not be installed with a package manager?

Say: sudo apt install gcc-4.8 or sudo apt install gcc-4.9

So, with Caffe2 on CentOS, I had to build gcc from scratch. I haven't tried MXNet on CentOS yet, but I wonder if whoever wrote this ran into a similar issue.

Since this is for "other linux" the apt instructions won't cut it.
My preference would be to split out a separate page for CentOS and have thorough, tested instructions. (maybe as a separate PR?)

lebeg · 2018-08-29T09:29:47Z

docs/install/build_from_source.md


-In Visual Studio, open the solution file,```.sln```, and compile it.
-These commands produce a library called ```mxnet.dll``` in the ```./build/Release/``` or ```./build/Debug``` folder.
+* Build on macOS with the default BLAS library and clang installed with `xcode` (OPENMP is disabled because it is not supported in default by clang):


To use OpenMP on MacOS you need to install the Clang compiler brew install llvm (the one provided by Apple does not support OpenMP)

lebeg · 2018-08-29T09:31:07Z

docs/install/build_from_source.md


-</div>
+```bash
+make USE_OPENCV=0


Maybe this would be a good place to put the BLAS libraries explanation from #11148. What do you think?

This caused me to rearrange the page for a better flow. This is all good info. It should be double-checked in case it is only for cmake and won't work for make.

Ah, yes you're right

lebeg · 2018-08-29T09:35:36Z

docs/install/index.md

-**Experimental Choice** If You would like to install mxnet with Intel MKL, try the experimental pip package with MKL:
-```bash
-$ pip install mxnet-mkl
+$ pip install mxnet


Not mxnet-mkl anymore? There are a lot of other options as well: https://pypi.org/search/?q=mxnet Maybe add some recommendation or at least list some options here?

I added the link to PyPI as a "footer" for every pip install instruction. That way users can look for them. I could also put in a footer note that adding *-mkl to a package would get them the experimental mkl support.... that would be good?

lebeg · 2018-08-29T09:38:51Z

docs/install/index.md


+- [G++ (4.8 or later)](https://gcc.gnu.org/gcc-4.8/). Make sure to use gcc 4 and not 5 or 6 as there


Clang 3.9 - 6 as well

lebeg · 2018-08-29T09:40:56Z

docs/install/index.md

+        -DUSE_SIGNAL_HANDLER=ON \
+        -DCMAKE_BUILD_TYPE=Release \
+        -GNinja ..
+    ninja -j1


-j $(nproc)

lebeg · 2018-08-29T09:42:08Z

docs/install/osx_setup.md

+
+```bash
+$ cd python
+$ pip install -e .


Note that the `-e` flag is optional. It is equivalent to `--editable` and means that if you edit the source files, these changes will be reflected in the package installed.

aaronmarkham · 2018-08-29T18:45:13Z

@lebeg thanks for the review! I've addressed most of you comments. Please take a look at the build from source page again since I made a lot of changes after incorporating your cmake documentation on BLAS. That was all really good stuff to add here!

A big update would be to switch using cmake as the default and change all of the examples, but I think we should hold off until your cmake PR is merged, right?

aaronmarkham · 2018-08-29T21:34:05Z

I could also add this table to the pip footer...

lebeg · 2018-08-30T08:45:34Z

A big update would be to switch using cmake as the default and change all of the examples, but I think we should hold off until your cmake PR is merged, right?

Yes, absolutely.

lebeg · 2018-08-30T08:50:46Z

docs/install/build_from_source.md

-Both JDK and Maven are required to build the Scala package.
-
-<div class="ubuntu">
+* Build on **macOS** with the default BLAS library and clang installed with `xcode` (OPENMP is disabled because it is not supported in default by clang):


Build on macOS with the default BLAS library and Clang installed with xcode (OpenMP is disabled because it is not supported by default by Apple version of Clang):

aaronmarkham · 2018-08-30T18:58:26Z

This PR also fixes #10020

* initial edits for c++ instructions * consolidation and reorg of install docs * simplify python section * fix url rewrite to allow for testing * further simplify install * fix render issues * adjust formatting * adjust formatting * fix python/gpu/master * install footers * update footers * add mac dev setup link for osx * adjust formatting for footer * separate validation page * add toc * adjust formatting * break out julia and perl * fix pypi link * fix formatting * fix formatting * fix formatting * add build from source link * updating from PR feedback * add c++ and clojure to ubuntu guide; clarify c++ setup * added pip chart to install; updates from PR feedback * correct reference to c infer api

marcoabreu · 2018-09-14T11:49:49Z

This PR broke the installation guide nightly test: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTests/detail/master/79/pipeline

aaronmarkham · 2018-09-14T14:13:40Z

I'll take a look. I removed the virtualenv section so it should be an easy fix.

* initial edits for c++ instructions * consolidation and reorg of install docs * simplify python section * fix url rewrite to allow for testing * further simplify install * fix render issues * adjust formatting * adjust formatting * fix python/gpu/master * install footers * update footers * add mac dev setup link for osx * adjust formatting for footer * separate validation page * add toc * adjust formatting * break out julia and perl * fix pypi link * fix formatting * fix formatting * fix formatting * add build from source link * updating from PR feedback * add c++ and clojure to ubuntu guide; clarify c++ setup * added pip chart to install; updates from PR feedback * correct reference to c infer api

aaronmarkham added 22 commits August 28, 2018 11:23

initial edits for c++ instructions

90587ba

consolidation and reorg of install docs

a407afb

simplify python section

ea088a1

fix url rewrite to allow for testing

d237bfe

further simplify install

2e35cb8

fix render issues

788fdf1

adjust formatting

a13d8ba

adjust formatting

ae041d0

fix python/gpu/master

3b05cfe

install footers

b2d26f2

update footers

aae8804

add mac dev setup link for osx

c743cdd

adjust formatting for footer

09ae549

separate validation page

e4ecfb7

add toc

9065c25

adjust formatting

2675a77

break out julia and perl

7d9ce90

fix pypi link

1321bbf

fix formatting

88f43c2

fix formatting

8289b74

fix formatting

7813c37

add build from source link

78ce36d

aaronmarkham requested a review from szha as a code owner August 29, 2018 00:25

lebeg reviewed Aug 29, 2018

View reviewed changes

updating from PR feedback

e70d319

add c++ and clojure to ubuntu guide; clarify c++ setup

d43e6f9

lebeg approved these changes Aug 30, 2018

View reviewed changes

added pip chart to install; updates from PR feedback

06d4be6

correct reference to c infer api

db66150

This was referenced Sep 4, 2018

macOS installation (build from source) fails #12373

Closed

mxnet install page for linux mentions two different versions of Cuda #12437

Closed

indhub merged commit d7111d3 into apache:master Sep 5, 2018

aaronmarkham mentioned this pull request Sep 26, 2018

How can I build C++ package? #10020

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Installation instructions consolidation #12388

Installation instructions consolidation #12388

aaronmarkham commented Aug 29, 2018

lebeg Aug 29, 2018

lebeg Aug 29, 2018

aaronmarkham Aug 29, 2018

lebeg Aug 30, 2018

lebeg Aug 29, 2018

aaronmarkham Aug 29, 2018

lebeg Aug 30, 2018

lebeg Aug 29, 2018

lebeg Aug 29, 2018

aaronmarkham Aug 29, 2018

lebeg Aug 29, 2018

lebeg Aug 29, 2018

aaronmarkham Aug 29, 2018

lebeg Aug 30, 2018

lebeg Aug 29, 2018

aaronmarkham Aug 29, 2018

lebeg Aug 30, 2018

lebeg Aug 29, 2018

lebeg Aug 29, 2018

lebeg Aug 29, 2018

aaronmarkham commented Aug 29, 2018 •

edited

Loading

aaronmarkham commented Aug 29, 2018

lebeg commented Aug 30, 2018

lebeg Aug 30, 2018

aaronmarkham commented Aug 30, 2018

marcoabreu commented Sep 14, 2018

aaronmarkham commented Sep 14, 2018

		@@ -28,184 +32,102 @@ You need C++ build tools and BLAS library to build MXNet shared library. If you
		for Windows) to build the library.


		- [G++ (4.8 or later)](https://gcc.gnu.org/gcc-4.8/). Make sure to use gcc 4 and not 5 or 6 as there

Installation instructions consolidation #12388

Installation instructions consolidation #12388

Conversation

aaronmarkham commented Aug 29, 2018

Description

Features

Preview

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronmarkham commented Aug 29, 2018 • edited Loading

aaronmarkham commented Aug 29, 2018

lebeg commented Aug 30, 2018

Choose a reason for hiding this comment

aaronmarkham commented Aug 30, 2018

marcoabreu commented Sep 14, 2018

aaronmarkham commented Sep 14, 2018

aaronmarkham commented Aug 29, 2018 •

edited

Loading