Improve and refactor noxfile to prepare for translation commands (#379)

Author: ccordoba12

CONTRIBUTING.md

@@ -6,7 +6,7 @@ First off, thanks for your interest in helping out with the documentation for Sp
 For more information about Spyder, please see the [website](https://www.spyder-ide.org/), and for the core Spyder codebase, visit the [main repo](https://github.com/spyder-ide/spyder).
 You can view the live documentation for current and past Spyder versions at [docs.Spyder-IDE.org](https://docs.spyder-ide.org).
 
-Spyder-Docs is part of the Spyder IDE Github org, and is developed with standard Github flow.
+Spyder-Docs is part of the Spyder IDE GitHub org, and is developed with standard GitHub flow.
 If you're not comfortable with at least the basics of ``git`` and GitHub, we recommend reading beginner tutorials such as [GitHub's Git Guide](https://github.com/git-guides/), its [introduction to basic Git commands](https://docs.github.com/en/get-started/using-git/about-git#basic-git) and its [guide to the fork workflow](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project).
 However, this contributing guide should fill you in on most of the basics you need to know.
 
@@ -20,13 +20,15 @@ Let us know if you have any further questions, and we look forward to your contr
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
 - [Reporting Issues](#reporting-issues)
-- [Cloning the repository](#cloning-the-repository)
+- [Cloning the Repository](#cloning-the-repository)
 - [Setting Up a Development Environment with Nox (Recommended)](#setting-up-a-development-environment-with-nox-recommended)
 - [Setting Up a Development Environment Manually](#setting-up-a-development-environment-manually)
   - [Create and activate a fresh environment](#create-and-activate-a-fresh-environment)
   - [Install dependencies](#install-dependencies)
 - [Installing the Pre-Commit Hooks](#installing-the-pre-commit-hooks)
-- [Building the docs](#building-the-docs)
+- [Building the Docs](#building-the-docs)
+  - [Build with Nox](#build-with-nox)
+  - [Build manually](#build-manually)
 - [Deciding Which Branch to Use](#deciding-which-branch-to-use)
 - [Making Your Changes](#making-your-changes)
 - [Pushing your Branch](#pushing-your-branch)
@@ -49,9 +51,9 @@ If referring to a particular word, line or section, please be sure to provide a
 
 
 
-## Cloning the repository
+## Cloning the Repository
 
-First, navigate to the [project repository](https://github.com/spyder-ide/spyder-docs) in your web browser and press the ``Fork`` button to make a personal copy of the repository on your own Github account.
+First, navigate to the [project repository](https://github.com/spyder-ide/spyder-docs) in your web browser and press the ``Fork`` button to make a personal copy of the repository on your own GitHub account.
 Then, click the ``Clone or Download`` button on your repository, copy the link and run the following on the command line to clone the repo:
 
 ```shell
@@ -208,39 +210,54 @@ Once you're satisfied, ``git add .`` and commit again.
 
 
 
-## Building the docs
+## Building the Docs
 
-To build the docs locally with Sphinx, if you've installed Nox you can just run
+The documentation is built with [Sphinx](https://www.sphinx-doc.org/), which you can invoke either using [Nox](https://nox.thea.codes/) or manually.
+
+
+### Build with Nox
+
+To build the docs using Nox, just run
 
 ```shell
 nox -s build
 ```
 
-For manual installations, you can invoke Sphinx yourself with the appropriate options:
+and can then open the rendered documentation in your default web browser with
 
 ```shell
-python -m sphinx -n -W --keep-going doc doc/_build/html
+nox -s serve
 ```
 
-If using Nox, you can open the rendered documentation in your default web browser with
+Alternatively, to automatically rebuild the docs when changes occur, you can invoke
 
 ```shell
-nox -s serve
+nox -s autobuild
 ```
 
-and to additionally automatically keep rebuilding the docs when changes occur, you can invoke
+You can also pass your own custom [Sphinx build options](https://www.sphinx-doc.org/en/master/man/sphinx-build.html) after a ``--`` separator which are added to the default set.
+For example, to rebuild just the install guide and FAQ in verbose mode with the ``dirhtml`` builder (our noxfile automatically prepends the source directory for you, so typing the full relative path is optional):
 
 ```shell
-nox -s autobuild
+nox -s build -- --verbose --builder dirhtml -- installation.rst faq.rst
+```
+
+
+### Build manually
+
+For manual installations, you can invoke Sphinx yourself with the appropriate options:
+
+```shell
+python -m sphinx -n -W --keep-going doc doc/_build/html
 ```
 
-Otherwise, navigate to the ``_build/html`` directory inside the ``spyder-docs`` repository and open ``index.html`` (the main page of the docs) in your preferred browser.
+Then, navigate to the ``_build/html`` directory inside the ``spyder-docs`` repository and open ``index.html`` (the main page of the docs) in your preferred browser.
 
 
 
 ## Deciding Which Branch to Use
 
-When you start to work on a new pull request (PR), you need to be sure that your work is done on top of the correct branch, and that you base your PR on Github against it.
+When you start to work on a new pull request (PR), you need to be sure that your work is done on top of the correct branch, and that you base your PR on GitHub against it.
 
 To guide you, issues on GitHub are marked with a milestone that indicates the correct branch to use.
 If not, follow these guidelines:

noxfile.py

@@ -1,6 +1,8 @@
 """Common tasks to build, check and publish Spyder-Docs."""
 
 # Standard library imports
+import contextlib
+import logging
 import os
 import tempfile
 import shutil
@@ -10,17 +12,20 @@
 
 # Third party imports
 import nox  # pylint: disable=import-error
+import nox.logger  # pylint: disable=import-error
 
 
 # --- Global constants --- #
 
+nox.options.error_on_external_run = True
 nox.options.sessions = ["build"]
 nox.options.default_venv_backend = "none"
 
-LATEST_VERSION = 5
-BASE_URL = "https://docs.spyder-ide.org"
 
-BUILD_INVOCATION = ("python", "-m", "sphinx", "--color")
+CI = "CI" in os.environ
+CANARY_COMMAND = ("sphinx-build", "--version")
+
+BUILD_INVOCATION = ("python", "-m", "sphinx")
 SOURCE_DIR = Path("doc").resolve()
 BUILD_DIR = Path("doc/_build").resolve()
 BUILD_OPTIONS = ("-n", "-W", "--keep-going")
@@ -31,11 +36,23 @@
 
 SCRIPT_DIR = Path("scripts").resolve()
 
-CI = "CI" in os.environ
+LATEST_VERSION = 5
+BASE_URL = "https://docs.spyder-ide.org"
 
 
 # ---- Helpers ---- #
 
+@contextlib.contextmanager
+def set_log_level(logger=nox.logger.logger, level=logging.CRITICAL):
+    """Context manager to set a logger log level and reset it after."""
+    prev_level = logger.level
+    logger.setLevel(level)
+    try:
+        yield
+    finally:
+        logger.setLevel(prev_level)
+
+
 def split_sequence(seq, sep="--"):
     """Split a sequence by a single separator."""
     if sep not in seq:
@@ -56,6 +73,21 @@ def process_filenames(filenames, source_dir=SOURCE_DIR):
     return filenames
 
 
+def extract_builder_name(options):
+    """Extract the Sphinx builder name from a sequence of options."""
+    try:
+        builder_index = options.index("--builder")
+    except ValueError:
+        try:
+            builder_index = options.index("-b")
+        except ValueError:
+            return None
+
+    options.pop(builder_index)
+    builder = options.pop(builder_index)
+    return builder
+
+
 def construct_sphinx_invocation(
     posargs=(),
     builder=HTML_BUILDER,
@@ -65,16 +97,22 @@ def construct_sphinx_invocation(
     build_invocation=BUILD_INVOCATION,
 ):
     """Reusably build a Sphinx invocation string from the given arguments."""
-    extra_options, filenames = split_sequence(posargs)
+    extra_options, filenames = split_sequence(list(posargs))
     filenames = process_filenames(filenames, source_dir=source_dir)
+    builder = extract_builder_name(extra_options) or builder
+
+    if CI:
+        build_options = list(build_options) + ["--color"]
+
     sphinx_invocation = [
         *build_invocation,
         "-b",
         builder,
         *build_options,
         *extra_options,
+        "--",
         str(source_dir),
-        str(Path(build_dir) / builder),
+        str(build_dir / builder),
         *filenames,
     ]
     return sphinx_invocation
@@ -86,37 +124,51 @@ def construct_sphinx_invocation(
 # See: https://github.com/wntrblm/nox/issues/167
 @nox.session(venv_backend="virtualenv", reuse_venv=True)
 def _execute(session):
-    """Dispatch tasks to run in a common environment. Don not run directly."""
-    _install(session)
+    """Dispatch tasks to run in a common environment. Do not run directly."""
+    if not session.posargs or isinstance(session.posargs[0], str):
+        raise ValueError(
+            "Must pass a list of functions to execute as first posarg")
+
+    if not session.posargs or session.posargs[0] is not _install:
+        # pylint: disable=too-many-try-statements
+        try:
+            with set_log_level():
+                session.run(
+                    *CANARY_COMMAND, include_outer_env=False, silent=True)
+        except nox.command.CommandFailed:
+            print("Installing dependencies in isolated environment...")
+            _install(session, use_posargs=False)
+
     if session.posargs:
         for task in session.posargs[0]:
             task(session)
 
 
 # ---- Install ---- #
 
-def _install(session):
+def _install(session, use_posargs=True):
     """Execute the dependency installation."""
-    session.install("-r", "requirements.txt")
+    posargs = session.posargs[1:] if use_posargs else ()
+    session.install("-r", "requirements.txt", *posargs)
 
 
 @nox.session
 def install(session):
-    """Install the project's dependencies in a virtual environment."""
-    session.notify("_execute", posargs=())
+    """Install the project's dependencies (passes through args to pip)."""
+    session.notify("_execute", posargs=([_install], *session.posargs))
 
 
 # ---- Utility ---- #
 
-def _sphinx_help(session):
+def _build_help(session):
     """Print Sphinx --help."""
     session.run(*BUILD_INVOCATION, "--help")
 
 
 @nox.session(name="help")
-def sphinx_help(session):
-    """Get help with Sphinx."""
-    session.notify("_execute", posargs=([_sphinx_help],))
+def build_help(session):
+    """Get help with the project build."""
+    session.notify("_execute", posargs=([_build_help],))
 
 
 def _run(session):
@@ -130,18 +182,24 @@ def run(session):
     session.notify("_execute", posargs=([_run], *session.posargs))
 
 
-def _clean():
-    """Remove the Sphinx build directory."""
+def _clean(session):
+    """Remove the build directory."""
+    print(f"Removing build directory {BUILD_DIR.as_posix()!r}")
+    ignore = session.posargs and session.posargs[0] in {"-i", "--ignore"}
+
     try:
-        BUILD_DIR.unlink()
+        shutil.rmtree(BUILD_DIR, ignore_errors=ignore)
     except FileNotFoundError:
         pass
+    except Exception:
+        print("\nError removing files; pass '-i'/'--ignore' flag to ignore\n")
+        raise
 
 
 @nox.session
-def clean(_session):
-    """Clean build artifacts."""
-    _clean()
+def clean(session):
+    """Clean build artifacts (pass -i/--ignore to ignore errors)."""
+    _clean(session)
 
 
 # ---- Build ---- #

requirements.txt

@@ -1,4 +1,4 @@
 pre-commit>=2.10.0,<4  # For pre-commit hooks; Lowercap to config syntax
 pydata-sphinx-theme>=0.14.1,<0.16  #  Theme; Lowercap to add version warning
-sphinx>=5,<8  # Docs generator; Cap to range confirmed supported by deps
-sphinx-design>=0.5.0,<0.6  # For dropdowns and panels; lowercap to sphinx>=5,<8
+sphinx>=5,<9  # Docs generator; Cap to range confirmed supported by deps
+sphinx-design>=0.5.0,<0.7  # For dropdowns and panels; lowercap to sphinx>=5,<8