Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sphinx - core #1930

Merged
merged 10 commits into from
Jun 8, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/deploy-gh-pages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Deploy to GitHub Pages
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AA-Turner I think we don't deploy on the GitHub pages... What's the reason for this workflow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't currently. #1385 has extensive discussion on this (#1385 (comment) etc) - the main drive is that deploying to GH pages is easy and free, and enables previewing the changes before any switchovers etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also discussion in various other issues (#2, #3, #25) and others I've likely forgoton (may be some on the pythondotorg repo)


on: [push]

jobs:
deploy-to-pages:
runs-on: ubuntu-latest

steps:
- name: 🛎️ Checkout
uses: actions/checkout@v2

- name: 🐍 Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9

- name: 🧳 Cache pip
uses: actions/cache@v2
with:
# This path is specific to Ubuntu
path: ~/.cache/pip
# Look to see if there is a cache hit for the corresponding requirements file
key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-

- name: 👷‍ Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: 🔧 Build PEPs
run: make pages -j$(nproc)

- name: 🚀 Deploy to GitHub pages
uses: JamesIves/[email protected]
with:
branch: gh-pages # The branch to deploy to.
folder: build # Synchronise with build.py -> build_directory
single-commit: true # Delete existing files
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ __pycache__
.vscode
*.swp
/build
/package
/venv
26 changes: 20 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Rules to only make the required HTML versions, not all of them,
# without the user having to keep track of which.
#
# Not really important, but convenient.
# Builds PEP files to HTML using docutils or sphinx
# Also contains testing targets

PEP2HTML=pep2html.py

Expand Down Expand Up @@ -34,7 +32,6 @@ install:

clean:
-rm pep-0000.rst
-rm pep-0000.txt
-rm *.html
-rm -rf build

Expand All @@ -43,7 +40,7 @@ update:

venv:
$(PYTHON) -m venv $(VENV_DIR)
./$(VENV_DIR)/bin/python -m pip install -U docutils
./$(VENV_DIR)/bin/python -m pip install -r requirements.txt

package: all rss
mkdir -p build/peps
Expand All @@ -57,3 +54,20 @@ package: all rss
lint:
pre-commit --version > /dev/null || python3 -m pip install pre-commit
pre-commit run --all-files

# New Sphinx targets:

SPHINX_JOBS=8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 8?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arbitrary number, beyond that diminishing returns. This was tested on GHA CI infrastructure, so a different number may be faster locally.

I'm considering disabiling search capabilities, as people will likely be better served with site:peps.python.org blah searches, and I think that the word stemming process takes a bit of time. Sphinx builds on CI took around 7 minutes, which does seem quite long (3-4x straight docutils)

(From #1934 (comment) - the makefile changes there come from this core PR)

SPHINX_BUILD=$(PYTHON) build.py -j $(SPHINX_JOBS)

pages: rss
$(SPHINX_BUILD) --index-file

sphinx:
$(SPHINX_BUILD)

fail_on_warning:
$(SPHINX_BUILD) --fail-on-warning

check_links:
$(SPHINX_BUILD) --check-links
54 changes: 54 additions & 0 deletions build.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
"""Build script for Sphinx documentation"""

import argparse
from pathlib import Path

from sphinx.application import Sphinx


def create_parser():
parser = argparse.ArgumentParser(description="Build PEP documents")
# alternative builders:
parser.add_argument("-l", "--check-links", action="store_true")

# flags / options
parser.add_argument("-f", "--fail-on-warning", action="store_true")
parser.add_argument("-n", "--nitpicky", action="store_true")
parser.add_argument("-j", "--jobs", type=int)

# extra build steps
parser.add_argument("-i", "--index-file", action="store_true") # for PEP 0

return parser.parse_args()


if __name__ == "__main__":
args = create_parser()

root_directory = Path(".").absolute()
source_directory = root_directory
build_directory = root_directory / "build" # synchronise with deploy-gh-pages.yaml -> deploy step
doctree_directory = build_directory / ".doctrees"

# builder configuration
sphinx_builder = "dirhtml"
if args.check_links:
sphinx_builder = "linkcheck"

# other configuration
config_overrides = {}
if args.nitpicky:
config_overrides["nitpicky"] = True

app = Sphinx(
source_directory,
confdir=source_directory,
outdir=build_directory,
doctreedir=doctree_directory,
buildername=sphinx_builder,
confoverrides=config_overrides,
warningiserror=args.fail_on_warning,
parallel=args.jobs,
)
app.builder.copysource = False # Prevent unneeded source copying - we link direct to GitHub
app.build()
37 changes: 37 additions & 0 deletions conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
"""Configuration for building PEPs using Sphinx."""

# -- Project information -----------------------------------------------------

project = "PEPs"
master_doc = "contents"

# -- General configuration ---------------------------------------------------

# The file extensions of source files. Sphinx uses these suffixes as sources.
source_suffix = {
".rst": "restructuredtext",
".txt": "restructuredtext",
}

# List of patterns (relative to source dir) to ignore when looking for source files.
exclude_patterns = [
# Windows:
"Thumbs.db",
".DS_Store",
# Python:
"venv",
"requirements.txt",
# Sphinx:
"build",
"output.txt", # Link-check output
# PEPs:
"README.rst",
"CONTRIBUTING.rst",
]

# -- Options for HTML output -------------------------------------------------

# HTML output settings
html_show_copyright = False # Turn off miscellany
html_show_sphinx = False
html_title = "peps.python.org" # Set <title/>
16 changes: 16 additions & 0 deletions contents.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@

Python Enhancement Proposals (PEPs)
***********************************


This is an internal Sphinx page, please go to the :doc:`PEP Index<pep-0000>`.


.. toctree::
:maxdepth: 3
:titlesonly:
:hidden:
:glob:
:caption: PEP Table of Contents (needed for Sphinx):

pep-*
3 changes: 3 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Requirements for building PEPs with Sphinx
sphinx >= 3.5
docutils >= 0.16