Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

add binary and docs build command options to devmenu #16514

Merged
merged 1 commit into from
Oct 18, 2019

Conversation

aaronmarkham
Copy link
Contributor

@aaronmarkham aaronmarkham commented Oct 17, 2019

Description

This PR will make it easier for users/contributors to build MXNet from source and then build any documentation set they need.

Prerequisites

  • Clone the repo
  • Install Python3 as well as pip3 (this script isn't compatible with Python 2.x)
  • Install Docker CE
  • Install Docker for Python
pip3 install docker
  • Get docker to run without sudo (this is for Ubuntu):
#!/bin/bash
set -e
set -x
export DEBIAN_FRONTEND=noninteractive
apt-get -y install curl
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
      $(lsb_release -cs) \
         stable"
apt-get update
apt-get -y install docker-ce
service docker restart
usermod -a -G docker $SUDO_USER

Usage

Go to the root of the repo then run:

python3 devmenu.py

It'll look like this:


-- MXNet dev menu --

Available actions
------------------------------
1. [Local] BUILD CMake/Ninja (using cmake_options.yaml (cp cmake/cmake_options.yml .) and edit) (python3 virtualenv in "py3_venv")
2. [Local] Python Unit tests
3. [Docker] Build the MXNet binary - outputs to "lib/"
4. [Docker] Build the Jekyll website - outputs to "docs/static_site/build/html/"
5. [Docker] Build the Python API docs - outputs to "docs/python_docs/python/build/_build/html/"
6. [Docker] Build the C++ API docs - outputs to "docs/cpp_docs/build/html/html/"
7. [Docker] Build the Clojure API docs - outputs to "contrib/clojure-package/target/doc"
8. [Docker] Build the Java API docs - outputs to "docs/scala-package/build/docs/java"
9. [Docker] Build the Julia API docs - outputs to "julia/docs/site/"
10. [Docker] Build the R API docs - outputs to "R-package/build/mxnet-r-reference-manual.pdf"
11. [Docker] Build the Scala API docs - outputs to "scala-package/docs/build/docs/scala"
12. [Docker] sanity_check. Check for linting and code formatting and licenses.
13. [Docker] Python3 CPU unittests
14. [Docker] Python3 GPU unittests
15. [Docker] Python3 GPU+MKLDNN unittests
16. [Docker] Python3 CPU Intel MKLDNN unittests
17. [Docker] Python3 ARMv7 unittests (QEMU)
18. Clean (RESET HARD) repository (Warning! erases local changes / DATA LOSS)
------------------------------
Choose option> 

This PR adds options 3-11.

With the exception of Jekyll, the other documentation sets are likely going to need you to build the MXNet binary first (running option 3). But only once. When the binary is there, you can run the rest of the doc sets.

@aaronmarkham
Copy link
Contributor Author

@larroy Can you review?

@@ -132,8 +132,24 @@ def provision_virtualenv(venv_path=DEFAULT_PYENV):
('[Local] Python Unit tests',
"./py3_venv/bin/nosetests -v tests/python/unittest/"
),
('[Docker] Website and docs build outputs to "docs/_build/html/"',
"ci/build.py --platform ubuntu_cpu /work/runtime_functions.sh deploy_docs"),
('[Docker] Build the MXNet binary - outputs to "lib/"',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mxnet binary? not clear what this does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It builds the .so and .a file that the docs need for API docs generation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only the .so file is needed AFAIK

Copy link
Contributor

@larroy larroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you explain tradeoff / motivation between building all docs or one by one?

@aaronmarkham
Copy link
Contributor Author

could you explain tradeoff / motivation between building all docs or one by one?

CI was updated to parallelize the docs generation. It cut the build time in half. While you can build them manually, using docker and the same flow as CI helps consolidate any issues and helps with troubleshooting.

Also, contributors that focus on one language binding now can do just that, rather than needing to use Sphinx/Python to get their docs generated.

Copy link
Contributor

@larroy larroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aaronmarkham aaronmarkham merged commit d1200c9 into apache:master Oct 18, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants