Add new JSON output: supported Python versions (#102)

* Adds output that reflects a package's supported Python interpreters * Fixes whitespace issues * Adds an .editorconfig * Polish * Avoid Node warning * Use descriptive names * Make explicit matrix is json too Maybe we get real dicts in GHA one day... * Switch to tox & explicit list * Switch to structlog as test object We already use it in CI and we have control over it * Whitespace * Add changelog * Rename keys To be explicit where the data is coming from and to leave the door open for other sources. * Add docs --------- Co-authored-by: Frank Sachsenheim <[email protected]>
hynek · Mar 27, 2024 · 4303b19 · 4303b19
1 parent c16461c
commit 4303b19
Show file tree

Hide file tree

Showing 4 changed files with 144 additions and 11 deletions.
diff --git a/.github/workflows/ci-supported-pythons.yml b/.github/workflows/ci-supported-pythons.yml
@@ -0,0 +1,75 @@
+---
+name: Use JSON output to build job matrix
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+env:
+  FORCE_COLOR: "1" # Make tools pretty.
+
+jobs:
+  build-package:
+    name: Build & verify package
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          repository: hynek/structlog
+          path: structlog
+      - uses: actions/checkout@v4
+        with:
+          path: action
+      - uses: ./action
+        id: baipp
+        with:
+          path: structlog
+
+    outputs:
+      python-versions: ${{ steps.baipp.outputs.supported_python_classifiers_json_array }}
+      # If your matrix consists only of Python versions, you can use the
+      # following, too:
+      # python-versions: ${{ steps.baipp.outputs.supported_python_classifiers_json_job_matrix_value }}
+
+  test-package:
+    needs: build-package
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        # Create matrix from the 'python-versions' output from the build-package
+        # job.
+        python-version: ${{ fromJson(needs.build-package.outputs.python-versions) }}
+
+      # If you set 'python-versions' to
+      # 'supported_python_classifiers_json_job_matrix_value'
+      # above, you would set the matrix like this instead:
+      # matrix: ${{ fromJson(needs.build-package.outputs.python-versions) }}
+
+    steps:
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Download built packages from the build-package job.
+        uses: actions/download-artifact@v4
+        with:
+          name: Packages
+          path: dist
+
+      - name: Prepare tests & config
+        run: |
+          # We use tox together with the fast tox-uv plugin.
+          python -Im pip install tox-uv
+
+          # Unpack SDist for tests & config files.
+          tar xf dist/*.tar.gz --strip-components=1
+
+          # Ensure tests run against wheel.
+          rm -rf src
+
+      - run: python -Im tox run --installpkg dist/*.whl -f py$(echo ${{ matrix.python-version }} | tr -d .)
+
+...
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -9,6 +9,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ### Added
 
+- New outputs: `supported_python_classifiers_json_array` and `supported_python_classifiers_json_job_matrix_value`.
+
+  They are extracted from the trove classifiers defined in the package metadata (for example, `Programming Language :: Python :: 3.12`) and allow you to define the Python versions matrix for your CI jobs without duplicating this information.
+  [#80](https://github.com/hynek/build-and-inspect-python-package/pull/80)
+  [#102](https://github.com/hynek/build-and-inspect-python-package/pull/102)
+
 - New input: `skip-wheel` to skip building the wheel in addition to the source distribution.
   This is useful if you need to build your wheels using advanced tools like [*cibuildwheel*](https://cibuildwheel.pypa.io/) anyway.
   [#98](https://github.com/hynek/build-and-inspect-python-package/pull/98)

diff --git a/README.md b/README.md
@@ -6,7 +6,7 @@
 
 *build-and-inspect-python-package* is a GitHub Action that provides the following functionality to Python package maintainers:
 
-**Builds your package** using PyPA's [*build*](https://pypi.org/project/build/) (this works with any [PEP 517](https://peps.python.org/pep-0517/)-compatible build backend, including Hatch, Flit, Setuptools, PDM, or Poetry).
+**Builds your package** using PyPA’s [*build*](https://pypi.org/project/build/) (this works with any [PEP 517](https://peps.python.org/pep-0517/)-compatible build backend, including Hatch, Flit, Setuptools, PDM, or Poetry).
 [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/specs/source-date-epoch/) is set to the timestamp of the last commit, giving you reproducible builds with meaningful file timestamps.
 
 Uploads the **built *wheel* and the source distribution (*SDist*) as GitHub Actions artifacts**, so you can download and inspect them from the Summary view of a run, or [**upload them to PyPI automatically**][automated] once the verification succeeds.
@@ -16,7 +16,7 @@ Lints the **wheel contents** using [*check-wheel-contents*](https://pypi.org/pro
 Lints the **PyPI README** using [Twine](https://pypi.org/project/twine/) and uploads it as an GitHub Actions artifact for further manual inspection.
 To level up your PyPI README game, check out [*hatch-fancy-pypi-readme*](https://github.com/hynek/hatch-fancy-pypi-readme)!
 
-Prints the **tree of both *SDist* and *wheel*** in the CI output, so you don't have to download the packages, if you just want to check the content list.
+Prints the **tree of both *SDist* and *wheel*** in the CI output, so you don’t have to download the packages, if you just want to check the content list.
 
 Prints and uploads the **packaging metadata** as a GitHub Actions artifact.
 
@@ -43,9 +43,16 @@ It uploads every commit on `main` to [Test PyPI](https://test.pypi.org/project/s
   This is easily achievable using tools like [*setuptools-scm*](https://setuptools-scm.readthedocs.io/) or [*hatch-vcs*](https://github.com/ofek/hatch-vcs), but beyond the scope of this humble README.
 
 
+### Define Python Version Matrix Based On Package Metadata
+
+*build-and-inspect-python-package* extracts the Python versions your package supports from the trove classifiers in your package’s metadata and offers them as an action output.
+
+That means that you can define your CI matrix based on the Python versions your package supports without duplicating the information between your package configuration and your CI configuration.
+
+
 ### Applications
 
-If you package an **application** as a Python package, this action is useful to double-check you're shipping everything you need, including all templates, translation files, et cetera.
+If you package an **application** as a Python package, this action is useful to double-check you’re shipping everything you need, including all templates, translation files, et cetera.
 
 
 ## Usage
@@ -92,26 +99,39 @@ While *build-and-inspect-python-package* will build a wheel for you by default,
 
 ### Outputs
 
-- `dist`: the location with the built packages.
+- `dist`: The location with the built packages.
+
+  See, for example, how [*argon2-cffi-bindings*](https://github.com/hynek/argon2-cffi-bindings/blob/daff9ceb693312ab8257c60db4cd1c13cd866a35/.github/workflows/ci.yml#L83-L97) uses this feature to check the built wheels don’t break a package that depends on it.
 
-  See, for example, how [*argon2-cffi-bindings*](https://github.com/hynek/argon2-cffi-bindings/blob/daff9ceb693312ab8257c60db4cd1c13cd866a35/.github/workflows/ci.yml#L83-L97) uses this feature to check the built wheels don't break a package that depends on it.
+- `supported_python_classifiers_json_array`: A JSON array of Python versions that are supported by the package as defined by the trove classifiers in the package metadata (for example, `Programming Language :: Python :: 3.12`).
+
+  You can assign this to a matrix strategy key in your CI job (for example, `strategy.matrix.python-version`) to test against multiple Python versions without duplicating the information.
+  Since GitHub Actions only allows for strings as variables, you have to parse it with `fromJSON` in your workflow.
+
+  If all this sounds confusing: Check out our [supported Pythons CI workflow] for a realistic example.
+
+- `supported_python_classifiers_json_job_matrix_value`: Same as `supported_python_classifiers_json_array`, but it’s a mapping with the JSON array bound to the `python-version` key.
+
+  This is useful if you only want to define a matrix based on Python versions, because then you can just assign this to `strategy.matrix`.
 
 
 ### Artifacts
 
-After a successful run, you'll find multiple artifacts in the run's Summary view:
+After a successful run, you’ll find the following artifacts in the run’s Summary view:
 
 - **Packages**: The built packages.
   Perfect for [automated PyPI upload workflows][automated]!
-- **Package Metadata**: the extracted packaging metadata (*hint*: it's formatted as an email).
-- **PyPI README**: the extracted PyPI README, exactly how it would be used by PyPI as your project's landing page.
-  [PEP 621](https://peps.python.org/pep-0621/) calls it `readme`, in classic *setuptools* it's `long_description`.
+- **Package Metadata**: the extracted packaging metadata (*hint*: it’s formatted as an email).
+- **PyPI README**: the extracted PyPI README, exactly how it would be used by PyPI as your project’s landing page.
+  [PEP 621](https://peps.python.org/pep-0621/) calls it `readme`, in classic *setuptools* it’s `long_description`.
 
 
 ### Examples
 
 [Our CI](.github/workflows/ci.yml) uses all inputs and outputs, if you want to see them in action.
 
+Our [supported Pythons CI workflow] demonstrates how to use `supported_python_classifiers_json_array` to set up a matrix of Python versions for your CI jobs without duplicating the information with your packaging metadata.
+
 
 ## License
 
@@ -120,3 +140,4 @@ The scripts and documentation in this project are released under the [MIT Licens
 [automated]: https://github.com/python-attrs/attrs/blob/main/.github/workflows/pypi-package.yml
 [*cibuildwheel*]: https://cibuildwheel.pypa.io/
 [*setuptools-scm*]: https://setuptools-scm.readthedocs.io/
+[supported Pythons CI workflow]: .github/workflows/ci-supported-pythons.yml
diff --git a/action.yml b/action.yml
@@ -22,7 +22,21 @@ inputs:
 outputs:
   dist:
     description: The location of the built packages.
-    value: ${{ steps.setter.outputs.dist }}
+    value: ${{ steps.dist-location-setter.outputs.dist }}
+  supported_python_classifiers_json_array:
+    description: >
+      A JSON array that contains all classifier-declared supported Python
+      versions. When loaded using the 'fromJson' function, this can be assigned
+      to a matrix strategy key (for example, `python-version`).
+
+    value: ${{ steps.supported-pythons-setter.outputs.supported_python_classifiers_json_array }}
+  supported_python_classifiers_json_job_matrix_value:
+    description: >
+      Same as 'supported_python_classifiers_json_array', except it's already a
+      JSON mapping from "python-version" to a list of all classifier-declared
+      supported Python versions. In other words, you can assign it directly to
+      the 'strategy.matrix' key.
+    value: ${{ steps.supported-pythons-setter.outputs.supported_python_classifiers_json_job_matrix_value }}
 
 runs:
   using: composite
@@ -78,7 +92,7 @@ runs:
       working-directory: ${{ inputs.path }}
 
     - name: Set output
-      id: setter
+      id: dist-location-setter
       shell: bash
       run: echo "dist=/tmp/baipp/dist" >>$GITHUB_OUTPUT
 
@@ -165,3 +179,20 @@ runs:
       with:
         name: PyPI README${{ inputs.upload-name-suffix }}
         path: /tmp/baipp/dist/out/sdist/PyPI-README.*
+
+    - name: Generate JSON objects of supported Python versions
+      id: supported-pythons-setter
+      shell: bash
+      working-directory: /tmp/baipp/dist/out/sdist/
+      run: |
+          cat */PKG-INFO | python -c '
+          import json, re, sys
+          match_classifier = re.compile(
+              r"\s*Classifier: Programming Language :: Python :: (\d+\.\d+)$"
+          ).match
+          version_tokens = [
+              m.group(1).strip() for l in sys.stdin.readlines() if (m := match_classifier(l))
+          ]
+          print(f"supported_python_classifiers_json_array={json.dumps(version_tokens)}")
+          print(f"""supported_python_classifiers_json_job_matrix_value={json.dumps({"python-version": version_tokens})}""")
+          ' >> $GITHUB_OUTPUT