From 5fa529ca3993f2f061d5d2d3a7e0e6dbab0a25fc Mon Sep 17 00:00:00 2001 From: driazati Date: Tue, 18 Jan 2022 16:29:19 -0800 Subject: [PATCH 01/12] Use ci.py explicitly in docs building instructions This adds `ci.py` to the docs to make it more clear how to easily build the docs locally. This also re-arranges CI following the merging of all CI steps to run concurrently since there's no need to run the Sphinx precheck during GPU unit tests. This still preserves it though in the docs step as a way to quickly bail out if there are formatting errors so the full tutorials don't get built. --- Jenkinsfile | 4 -- docs/README.md | 67 +++++++++++--------- tests/scripts/ci.py | 8 +-- tests/scripts/task_python_docs.sh | 88 +++++++++++++++++++++++---- tests/scripts/task_sphinx_precheck.sh | 37 ++--------- 5 files changed, 121 insertions(+), 83 deletions(-) diff --git a/Jenkinsfile b/Jenkinsfile index 1cd8575fa3fd..aa4b73480270 100755 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -388,10 +388,6 @@ stage('Test') { unpack_lib('gpu', tvm_multilib) timeout(time: max_time, unit: 'MINUTES') { ci_setup(ci_gpu) - sh ( - script: "${docker_run} ${ci_gpu} ./tests/scripts/task_sphinx_precheck.sh", - label: 'Check Sphinx warnings in docs', - ) sh ( script: "${docker_run} ${ci_gpu} ./tests/scripts/task_java_unittest.sh", label: 'Run Java unit tests', diff --git a/docs/README.md b/docs/README.md index 0573e9595aa3..11c342862bb4 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,52 +16,61 @@ # TVM Documentation + This folder contains the source of TVM's documentation, hosted at https://tvm.apache.org/docs ## Build Locally -See also the instructions below to run a specific tutorial. Note that some of the tutorials need GPU support. Once build, either of these can be served using Python's build in HTTP server: +### With Docker (recommended) -```bash -# Run this and then visit http://localhost:8000 in your browser -cd docs/_build/html && python3 -m http.server -``` +1. Build TVM and the docs inside the [tlcpack/ci-gpu image](https://hub.docker.com/r/tlcpack/ci-gpu) using the [`ci.py`](../tests/scripts/ci.py) script. -### With Docker (recommended) + ```bash + # If this runs into errors, try cleaning your 'build' directory + python tests/scripts/ci.py docs -1. Build TVM and the docs inside the [tlcpack/ci-gpu image](https://hub.docker.com/r/tlcpack/ci-gpu) + # See other doc building options + python tests/scripts/ci.py docs --help + ``` - ```bash - # If this runs into errors, try cleaning your 'build' directory - make docs - ``` +2. Serve the docs and visit http://localhost:8000 in your browser + ```bash + # Run an HTTP server you can visit to view the docs in your browser + python tests/scripts/ci.py serve-docs + ``` ### Native 1. [Build TVM](https://tvm.apache.org/docs/install/from_source.html) first in the repo root folder 2. Install dependencies - ```bash - # Pillow on Ubuntu may require libjpeg-dev from apt - docker run tlcpack/ci-gpu:v0.78 bash -c \ - 'python3 -m pip install --quiet tlcpack-sphinx-addon==0.2.1 synr==0.5.0 && python3 -m pip freeze' > frozen-requirements.txt + ```bash + # Pillow on Ubuntu may require libjpeg-dev from apt + docker run tlcpack/ci-gpu:v0.78 bash -c \ + 'python3 -m pip install --quiet tlcpack-sphinx-addon==0.2.1 synr==0.5.0 && python3 -m pip freeze' > frozen-requirements.txt - pip install -r frozen-requirements.txt - ``` + pip install -r frozen-requirements.txt + ``` 3. Generate the docs - ```bash - # TVM_TUTORIAL_EXEC_PATTERN=none skips the tutorial execution to the build - # work on most environments (e.g. MacOS). - export TVM_TUTORIAL_EXEC_PATTERN=none + ```bash + # TVM_TUTORIAL_EXEC_PATTERN=none skips the tutorial execution to the build + # work on most environments (e.g. MacOS). + export TVM_TUTORIAL_EXEC_PATTERN=none + + make html + ``` - make html - ``` +4. Run an HTTP server and visit http://localhost:8000 in your browser + ```bash + cd docs/_build/html && python3 -m http.server + ``` ## Only Execute Specified Tutorials + The document build process will execute all the tutorials in the sphinx gallery. This will cause failure in some cases when certain machines do not have necessary environment. You can set `TVM_TUTORIAL_EXEC_PATTERN` to only execute @@ -70,14 +79,14 @@ the path that matches the regular expression pattern. For example, to only build tutorials under `/vta/tutorials`, run ```bash -TVM_TUTORIAL_EXEC_PATTERN=/vta/tutorials make html +python tests/scripts/ci.py docs --tutorials=/vta/tutorials ``` To only build one specific file, do ```bash # The slash \ is used to get . in regular expression -TVM_TUTORIAL_EXEC_PATTERN=file_name\.py make html +python tests/scripts/ci.py docs --tutorials=file_name\.py ``` ## Helper Scripts @@ -86,7 +95,7 @@ You can run the following script to reproduce the CI sphinx pre-check stage. This script skips the tutorial executions and is useful to quickly check the content. ```bash -./tests/scripts/task_sphinx_precheck.sh +python tests/scripts/ci.py docs --precheck ``` The following script runs the full build which includes tutorial executions. @@ -97,5 +106,7 @@ python tests/scripts/ci.py --precheck --full ``` ## Define the Order of Tutorials -You can define the order of tutorials with `conf.py::subsection_order` and `conf.py::within_subsection_order`. -By default, the tutorials within one subsection is sorted by filename. + +You can define the order of tutorials with `subsection_order` and +`within_subsection_order` in [`conf.py`](conf.py). +By default, the tutorials within one subsection are sorted by filename. diff --git a/tests/scripts/ci.py b/tests/scripts/ci.py index c9c3cc2cbfcf..7765e5de2202 100644 --- a/tests/scripts/ci.py +++ b/tests/scripts/ci.py @@ -183,13 +183,11 @@ def docker(name: str, image: str, scripts: List[str], env: Dict[str, str]): def docs( tutorial_pattern: Optional[str] = None, full: bool = False, - precheck: bool = False, cpu: bool = False, ) -> None: """ Build the documentation from gallery/ and docs/. By default this builds only - the Python docs. If you are on a CPU machine, you can skip the tutorials - and build the docs with the '--precheck --cpu' options. + the Python docs. arguments: full -- Build all language docs, not just Python @@ -242,9 +240,7 @@ def docs( config, f"./tests/scripts/task_build.sh build -j{NPROC}", "./tests/scripts/task_ci_setup.sh", - "./tests/scripts/task_sphinx_precheck.sh" - if precheck - else "./tests/scripts/task_python_docs.sh", + "./tests/scripts/task_python_docs.sh", ] if tutorial_pattern is None: diff --git a/tests/scripts/task_python_docs.sh b/tests/scripts/task_python_docs.sh index 312dd575c9ea..4e42b82c37be 100755 --- a/tests/scripts/task_python_docs.sh +++ b/tests/scripts/task_python_docs.sh @@ -16,15 +16,13 @@ # specific language governing permissions and limitations # under the License. -set -e -set -u -set -o pipefail +set -euo pipefail source tests/scripts/setup-pytest-env.sh # to avoid CI CPU thread throttling. export TVM_BIND_THREADS=0 -export OMP_NUM_THREADS=4 +export OMP_NUM_THREADS=1 IS_LOCAL=${IS_LOCAL:-0} PYTHON_DOCS_ONLY=${PYTHON_DOCS_ONLY:-0} @@ -34,14 +32,77 @@ cleanup() } trap cleanup 0 -# cleanup old states -rm -rf docs/_build -rm -rf docs/_staging -mkdir -p docs/_build/html -mkdir -p docs/_staging/html -rm -rf docs/gen_modules -rm -rf docs/doxygen +clean_files() { + # cleanup old states + rm -rf docs/_build + rm -rf docs/_staging + mkdir -p docs/_build/html + mkdir -p docs/_staging/html + rm -rf docs/gen_modules + rm -rf docs/doxygen + find . -type f -path "*.pyc" | xargs rm -f +} + +sphinx_precheck() { + clean_files + echo "PreCheck sphinx doc generation WARNINGS.." + make cython3 + + pushd docs + make clean + TVM_TUTORIAL_EXEC_PATTERN=none make html 2>&1 | tee /tmp/$$.log.txt + check_sphinx_warnings + popd +} + + +function join_by { local IFS="$1"; shift; echo "$*"; } + +IGNORED_WARNINGS=( + '__mro__' + 'UserWarning' + 'FutureWarning' + 'tensorflow' + 'Keras' + 'pytorch' + 'TensorFlow' + 'coremltools' + '403' + 'git describe' + 'scikit-learn version' + 'doing serial write' + 'gen_gallery extension is not safe for parallel' + 'strategy:conv2d NHWC layout is not optimized for x86 with autotvm.' + 'strategy:depthwise_conv2d NHWC layout is not optimized for x86 with autotvm.' + 'autotvm:Cannot find config for target=llvm -keys=cpu -link-params=0' + 'autotvm:One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.' +) + +JOINED_WARNINGS=$(join_by '|' "${IGNORED_WARNINGS[@]}") + +check_sphinx_warnings() { + grep -v -E "$JOINED_WARNINGS" < /tmp/$$.log.txt > /tmp/$$.logclean.txt || true + if grep --quiet -E "WARN" < /tmp/$$.logclean.txt; then + echo "Lines with 'WARNING' found in the log, please fix them:" + grep -E "WARN" < /tmp/$$.logclean.txt + echo "You can reproduce locally by running 'python tests/scripts/ci.py docs'" + exit 1 + fi + echo "No WARNINGS to be fixed." +} +# run precheck step first to fast-fail if there are problems with the docs +if [ "$IS_LOCAL" != "1" ]; then + echo "Running precheck" + sphinx_precheck +else + # skip the precheck when doing local builds since it would add overhead to + # re-runs (and tutorials are usually not enabled anyways) + echo "Skipping precheck" +fi + + +clean_files # prepare auto scheduler tutorials rm -rf gallery/how_to/tune_with_auto_scheduler/*.json rm -rf gallery/tutorial/*.json @@ -55,11 +116,14 @@ find . -type f -path "*.pyc" | xargs rm -f make cython3 cd docs -PYTHONPATH=`pwd`/../python make html SPHINXOPTS='-j auto' |& tee /tmp/$$.log.txt +PYTHONPATH=$(pwd)/../python make html SPHINXOPTS='-j auto' |& tee /tmp/$$.log.txt if grep -E "failed to execute|Segmentation fault" < /tmp/$$.log.txt; then echo "Some of sphinx-gallery item example failed to execute." exit 1 fi + +check_sphinx_warnings + cd .. if [ "$IS_LOCAL" == "1" ] && [ "$PYTHON_DOCS_ONLY" == "1" ]; then diff --git a/tests/scripts/task_sphinx_precheck.sh b/tests/scripts/task_sphinx_precheck.sh index 544d88af8b04..af0afd4fbadb 100755 --- a/tests/scripts/task_sphinx_precheck.sh +++ b/tests/scripts/task_sphinx_precheck.sh @@ -16,36 +16,7 @@ # specific language governing permissions and limitations # under the License. -# Precheck if sphinx docs build can fail. -set -e -set -u -set -o pipefail - -cleanup() -{ - rm -rf /tmp/$$.* -} -trap cleanup 0 - -# cleanup cache -rm -rf docs/_staging -rm -rf docs/_build -find . -type f -path "*.pyc" | xargs rm -f -make cython3 - -echo "PreCheck sphinx doc generation WARNINGS.." -cd docs -make clean -TVM_TUTORIAL_EXEC_PATTERN=none make html 2>&1 | tee /tmp/$$.log.txt - -grep -v -E "__mro__|UserWarning|FutureWarning|tensorflow|Keras|pytorch|TensorFlow|coremltools|403|git describe|scikit-learn version" < /tmp/$$.log.txt > /tmp/$$.logclean.txt || true -echo "---------Sphinx Log----------" -cat /tmp/$$.logclean.txt -echo "-----------------------------" -if grep --quiet -E "WARN" < /tmp/$$.logclean.txt; then - echo "WARNING found in the log, please fix them." - grep -E "WARN" < /tmp/$$.logclean.txt - echo "You can reproduce locally by running ./tests/scripts/task_sphinx_precheck.sh" - exit 1 -fi -echo "No WARNINGS to be fixed." +# Intentionally do nothing, see https://github.com/apache/tvm/pull/9971. This +# file cannot be deleted from the Jenkinsfile and deleted from the repo in the +# same PR (since it wouldn't be able to pass CI). Once that PR is merged, this +# file can be safely deleted. From 07b22996cdf1fc7bc084d263de7fe0fb049a7234 Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 12:44:46 -0800 Subject: [PATCH 02/12] trigger ci From 997fd2963b28a4d0466e963c00e0678be9a0bf7a Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 13:14:41 -0800 Subject: [PATCH 03/12] trigger ci From 1c879111943da02acf41141e8983c963161c1920 Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 13:52:43 -0800 Subject: [PATCH 04/12] trigger ci From d1ed23aa0be058b37fbddf97cfe319acc6169d2b Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 14:17:03 -0800 Subject: [PATCH 05/12] trigger ci From 5f528e803efe951c87e89cc2e0419e4733c8901b Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 15:00:57 -0800 Subject: [PATCH 06/12] trigger ci From 6ebcff3aa98e5b3df32447c4e3b49385bda16e4d Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 15:42:35 -0800 Subject: [PATCH 07/12] trigger ci From c12013f8a168c9c22f09f6ac642fe55bf3a74e1b Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 15:48:30 -0800 Subject: [PATCH 08/12] trigger ci From 2bb5f589b64c819e4fd130a675f7ab84ff2bcf4b Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 15:49:03 -0800 Subject: [PATCH 09/12] trigger ci From d35a2624139c59e5362cd87d3768c729f79f3a90 Mon Sep 17 00:00:00 2001 From: driazati Date: Mon, 24 Jan 2022 15:53:24 -0800 Subject: [PATCH 10/12] trigger ci From 63b80e3263406cfe08fd984b80a697f5743de76b Mon Sep 17 00:00:00 2001 From: driazati Date: Wed, 26 Jan 2022 12:31:23 -0800 Subject: [PATCH 11/12] add more warnings --- tests/scripts/task_python_docs.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/scripts/task_python_docs.sh b/tests/scripts/task_python_docs.sh index 4e42b82c37be..8a5fbb74ba4d 100755 --- a/tests/scripts/task_python_docs.sh +++ b/tests/scripts/task_python_docs.sh @@ -51,7 +51,7 @@ sphinx_precheck() { pushd docs make clean TVM_TUTORIAL_EXEC_PATTERN=none make html 2>&1 | tee /tmp/$$.log.txt - check_sphinx_warnings + check_sphinx_warnings "docs" popd } @@ -76,6 +76,7 @@ IGNORED_WARNINGS=( 'strategy:depthwise_conv2d NHWC layout is not optimized for x86 with autotvm.' 'autotvm:Cannot find config for target=llvm -keys=cpu -link-params=0' 'autotvm:One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.' + 'autotvm:Cannot find config for target=cuda -keys=cuda,gpu' ) JOINED_WARNINGS=$(join_by '|' "${IGNORED_WARNINGS[@]}") @@ -85,7 +86,7 @@ check_sphinx_warnings() { if grep --quiet -E "WARN" < /tmp/$$.logclean.txt; then echo "Lines with 'WARNING' found in the log, please fix them:" grep -E "WARN" < /tmp/$$.logclean.txt - echo "You can reproduce locally by running 'python tests/scripts/ci.py docs'" + echo "You can reproduce locally by running 'python tests/scripts/ci.py $1'" exit 1 fi echo "No WARNINGS to be fixed." @@ -122,7 +123,7 @@ if grep -E "failed to execute|Segmentation fault" < /tmp/$$.log.txt; then exit 1 fi -check_sphinx_warnings +check_sphinx_warnings "docs --tutorial-pattern=.*" cd .. From 07177ccbf9b5eab875f3073a295f909ae6334f1c Mon Sep 17 00:00:00 2001 From: driazati Date: Thu, 27 Jan 2022 11:42:57 -0800 Subject: [PATCH 12/12] address comments --- docs/README.md | 2 +- tests/scripts/task_python_docs.sh | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 11c342862bb4..520fea60ca28 100644 --- a/docs/README.md +++ b/docs/README.md @@ -47,7 +47,7 @@ This folder contains the source of TVM's documentation, hosted at https://tvm.ap ```bash # Pillow on Ubuntu may require libjpeg-dev from apt - docker run tlcpack/ci-gpu:v0.78 bash -c \ + ./docker/bash.sh ci_gpu -c \ 'python3 -m pip install --quiet tlcpack-sphinx-addon==0.2.1 synr==0.5.0 && python3 -m pip freeze' > frozen-requirements.txt pip install -r frozen-requirements.txt diff --git a/tests/scripts/task_python_docs.sh b/tests/scripts/task_python_docs.sh index 8a5fbb74ba4d..6d2dde58b97d 100755 --- a/tests/scripts/task_python_docs.sh +++ b/tests/scripts/task_python_docs.sh @@ -58,6 +58,10 @@ sphinx_precheck() { function join_by { local IFS="$1"; shift; echo "$*"; } +# These warnings are produced during the docs build for various reasons and are +# known to not signficantly affect the output. Don't add anything new to this +# list without special consideration of its effects, and don't add anything with +# a '|' character. IGNORED_WARNINGS=( '__mro__' 'UserWarning'