[doc] Add sphinx-autobuild and documentation for make local (#47275)

- Split doc building into 2 options: incremental build with caching &
full build from scratch.
- Add description & explanation of the process for incremental build
option.
- Use `sphinx-autobuild` instead of `sphinx-build` for auto reload upon
doc changes.

---------

Signed-off-by: khluu <[email protected]>
Signed-off-by: Kevin H. Luu <[email protected]>
Co-authored-by: angelinalg <[email protected]>

doc/.gitignore

@@ -24,3 +24,6 @@ source/serve/examples.rst
 
 # Ignore generated versions
 source/_static/versions.json
+
+# Ignore doc/serve
+serve/

doc/Makefile

@@ -8,6 +8,7 @@ SPHINXOPTS    = -a -E -W -j auto
 LOCALSPHINXOPTS = -W -j auto
 LINKCHECKOPTS = -a -E -j auto
 SPHINXBUILD   = sphinx-build
+SPHINXAUTOBUILD = sphinx-autobuild --port 0 --open-browser --ignore "*.log" --ignore "*data/examples*" --ignore "*train/examples*" --ignore "*serve/examples*"
 PAPER         =
 BUILDDIR      = _build
 
@@ -84,7 +85,7 @@ RAY_DIR := $(shell pwd | rev | cut -d'/' -f2- | rev)
 local:
 	python load_doc_cache.py --ray-dir=$(RAY_DIR)
 	python update_cache_env.py --ray-dir=$(RAY_DIR)
-	$(SPHINXBUILD) -W --keep-going -b html $(ALLLOCALSPHINXOPTS) $(BUILDDIR)/html
+	$(SPHINXAUTOBUILD) -W --keep-going -b html $(ALLLOCALSPHINXOPTS) $(BUILDDIR)/html
 
 dirhtml:
 	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml

doc/source/ray-contribute/docs.md

@@ -56,7 +56,38 @@ pip install -r requirements-doc.txt
 Don't use `-U` in this step. You don't want to upgrade dependencies because `requirements-doc.txt` pins exact versions you need to build the docs.
 
 ### Build documentation
-Build the documentation by running the following command:
+Before building, clean your environment first by running:
+```shell
+make clean
+```
+
+Choose from the following 2 options to build documentation locally:
+
+- Incremental build
+- Full build
+
+#### 1. Incremental build with global cache and live rendering
+
+To use this option, you can run:
+```shell
+make local
+```
+
+This option is recommended if you need to make frequent uncomplicated changes.
+
+In this approach, Sphinx only builds the changes you made in your branch compared to your last pull from upstream master. The rest of doc is cached with pre-built doc pages from your last commit from upstream (for every new commit pushed to Ray, CI builds all the documentation pages from that commit and store them on S3 as cache).
+
+The build first traces your commit tree to find the latest commit that CI already cached on S3. 
+Once the build finds the commit, it fetches the corresponding cache from S3 and extracts it into the `doc/` directory. Simultaneously, CI tracks all the files that have changed from that commit to current `HEAD`, including any un-staged changes.
+
+Sphinx then rebuilds only the pages that your changes affect, leaving the rest untouched from the cache.
+
+When build finishes, the doc page would automatically pop up on your browser. If any change is made in the `doc/` directory, Sphinx would automatically rebuild and reload your doc page. You can stop it by interrupting with `Ctrl+C`.
+
+
+#### 2. Full build from scratch
+In the full build option, Sphinx rebuilds all files in `doc/` directory, ignoring all cache and saved environment.
+Because of this behavior, you get a really clean build but it's much slower.
 
 ```shell
 make develop