From 514ee0623a77928b17c62a9631308201ea5eb60a Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Wed, 2 Sep 2026 18:13:19 -0700 Subject: [PATCH 1/5] Make the Python Coverage Selector Explicit and Name the Route D4.5 Forecloses Two deferred contract defects, both found by adversarial passes on earlier pull requests and both left for a change that could touch CI behavior rather than wording alone. ## D1.6, the Python coverage invocation Issue #1199 reported that `pytest --cov-report=xml` collects nothing, so no Python repo in the fleet reports coverage. The second half of that is false, and the live runs say so. `Financial-Modeling` and `aiopurpleair` each log `Coverage XML written to file coverage.xml` and upload it, because each carries a `--cov=` entry in its own pyproject `addopts`. `homeassistant-purpleair` has no `uv.lock`, so the step's own `if` never fires there, and PlexCleaner's Python subtree and `ESPHome-Config` are lint-only. The real defect is narrower. D1.6 and `spec/project-types.json` both document the bare invocation as the thing that collects coverage, and it does not: `pytest-cov` reports on what `--cov` selects, so `--cov-report=xml` alone measures nothing, writes no file, and exits zero. A repo onboarded by following D1.6 literally therefore reports nothing while its gate stays green, which is the failure D1.6's own *Prevents* clause names. Nothing downstream notices, because the upload is best-effort by design and reads a missing file exactly as it reads a healthy run. So the contract now states the three obligations that were implicit, that the repo references pytest-cov, selects the coverage source in its own pyproject, and leaves the report at the root path the upload step already reads, and `validate-task.yml` fails the test step when that file was not written. Measured both ways against a real pytest tree: with a selector in `addopts` the step passes as it does today, and without one it exits 1 with the `::error::` line rather than passing. The step deletes any `coverage.xml` before the run, so the check proves this run wrote one rather than that a file exists. Without that, a repo committing the report satisfies the guard while measuring nothing, which is the same false green in a new place. Proven by removing the delete and watching that case pass. Adding `--cov` to the shared validator was the obvious fix and is the wrong one. Alongside a repo's own `--cov=` it widens the measured set to include `tests/`, which moves every existing repo's percentage and would break a repo running `fail_under = 100`. The guard reaches a repo the way any hub task change does. Financial- Modeling calls `validate-task.yml` at a pinned hub release and picks it up at its next Dependabot bump. aiopurpleair still carries its own copy of the file, so it gets nothing here until it adopts the caller stub, and its copy already carries a comment saying the selector lives in pyproject, which is the explanation the hub's copy was missing. ## D4.5, the recovery a moved tip forecloses Issue #1214 reported that D4.5 and D5.2 between them imply a failed package push is unrecoverable once the branch tip moves, without either item having considered GitHub's **Re-run all jobs**, which replays a run at that run's own `github.sha` rather than at the tip. The issue proposed three dispositions and assumed settling between them needed a live publish against a real registry. It does not. 5B settles a question of this shape from the YAML, and the YAML answers this one. `publish-release.yml` passes `ref: ${{ github.sha }}`, and every build leaf checks out `needs.get-version.outputs.GitCommitId`. **Re-run all jobs** replays a run at that run's own `github.sha` rather than at the branch tip, so `get-version` recomputes the same version, the leaves rebuild the same commit, the package artifact D5.2 deleted is rebuilt rather than downloaded, and `publish-` retries the same push. The release needs nothing from the re-run, the failed run having already cut it. What stays open is only whether the registry accepts the retried push, and no route settles that in advance, the prescribed re-dispatch included. So D4.5 now names two recoveries and says which case each covers, rather than one recovery and an implication that the other case is beyond repair. D5.2 narrows its exclusion to **Re-run failed jobs**, the variant whose download really does find nothing. Two claims in the surrounding sentences were false and are corrected with it. "Leaves the first release without its package" is wrong for a NuGet caller, since D4.3 in the same item has the NuGet leaf uploading a `release-asset-*` carrying the package: what the first release lacks is the registry push, which the item already said correctly one sentence earlier. And "since NBGV derives the version from git height" was attached to the claim that a re-dispatch builds the new tip, which git height does not explain. The reason a dispatch resolves the tip is that it names a branch, which the item states separately. Three bounds ship with the route. D4.4 and 5B's S9 describe a re-run whose predecessor push succeeded, where the registry dedupes the second, so this recovery is the case they do not cover. GitHub allows a re-run only within 30 days. And **Re-run failed jobs** is not the route, since D5.2's delete has already taken the artifact its download needs on the path that reaches this case. The issue's claim that 5C reserves a real registry push for the maintainer is not in 5C. 5C reserves the deploy ref gate dispatch and nothing else, so that citation was dropped rather than carried over. The recovery passage took six adversarial rounds and every round but the last introduced a defect the next one caught, twice by correctly applying the previous round's advice. The durable lesson is that prose stating a rule on top of a mechanism fails differently from prose stating the mechanism: when the mechanism turned out to be unverifiable, re-deriving the rule from what was left produced a claim nothing supported, and deleting the rule was the fix. Closes #1199 Closes #1214 --- .../references/d-guarantees.md | 4 ++-- .claude-plugin/fleet-skills/.source-digest | 2 +- .../references/d-guarantees.md | 4 ++-- .../references/d-guarantees.md | 4 ++-- .github/workflows/validate-task.yml | 12 +++++++++- WORKFLOW.md | 6 ++--- docs/reusable-workflows.md | 2 +- reports/canonical-review.json | 24 +++++++++---------- spec/project-types.json | 2 +- 9 files changed, 35 insertions(+), 25 deletions(-) diff --git a/.agents/skills/workflow-ci-contract/references/d-guarantees.md b/.agents/skills/workflow-ci-contract/references/d-guarantees.md index a49f42ce..7e44658c 100644 --- a/.agents/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.agents/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator passes `--cov-report=xml` on the Python leg, which names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and the validator reds the test step when no root `coverage.xml` was written. ## D2: Validation at Entry @@ -32,7 +32,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D4.2** `target_commitish` is the built commit's SHA (NBGV `GitCommitId`), never a branch name and never a separately re-resolved ref. - **D4.3** Every release is a tag plus source zip, README, and LICENSE, `prerelease` equals `branch != default`, file targets attach `release-asset-*`, and a no-file-target caller (Docker-only, PyPI-only, source-only) passes `expect_release_assets: false` or the release-create step fails on unmatched files, a source-only one setting every `enable_*` input false with it. A NuGet caller is not one of those, since its leaf uploads a `release-asset-*` carrying the package. - **D4.4** No-op republish on a schedule or push trigger: an unchanged version re-pushes nothing and the release-create skips when the tag exists, while a dispatch re-run refreshes it and runs the paired asset delete with it, registries dedupe server-side under `dotnet nuget push --skip-duplicate` and PyPI's `skip-existing: true`, and Docker always re-pushes by design. -- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch rather than a cleanup, and it repairs that version only while the branch tip has not moved, since NBGV derives the version from git height, so a re-dispatch after the tip has moved publishes a further version and leaves the first release without its package. +- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch while the tip has not moved, since a dispatch names a branch rather than a commit and so builds that branch's tip at dispatch time. Once the tip has moved a re-dispatch builds the new tip instead, and NBGV deriving the version from git height makes that a further version, so the version whose push failed never reaches the registry. **Re-run all jobs** is the recovery there: GitHub replays under the original event's `GITHUB_SHA` and re-executes every job, and the publisher pins the build to that commit, so the same version is rebuilt, its package artifact rebuilt and re-uploaded rather than left missing by D5.2's delete, and its push retried, the release itself needing nothing from the re-run. Three bounds. D4.4's no-op re-run assumes the earlier push succeeded, so it does not describe this one. GitHub offers a re-run only within 30 days of the initial run. And **Re-run failed jobs** is unreliable rather than unavailable, D5.2's delete usually having taken the artifact its download needs while D5.3 leaves that delete best-effort. - **D4.6** A deploy check asserts which release and which environment answer, waiting for convergence to a bounded timeout, with an unreachable host reported distinctly from an HTTP status. ## D5: Resource Cleanup diff --git a/.claude-plugin/fleet-skills/.source-digest b/.claude-plugin/fleet-skills/.source-digest index 01ad8951..4810f274 100644 --- a/.claude-plugin/fleet-skills/.source-digest +++ b/.claude-plugin/fleet-skills/.source-digest @@ -1 +1 @@ -9cf4ad34f6781963 +b70d364d5a8969ac diff --git a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md index a49f42ce..7e44658c 100644 --- a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator passes `--cov-report=xml` on the Python leg, which names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and the validator reds the test step when no root `coverage.xml` was written. ## D2: Validation at Entry @@ -32,7 +32,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D4.2** `target_commitish` is the built commit's SHA (NBGV `GitCommitId`), never a branch name and never a separately re-resolved ref. - **D4.3** Every release is a tag plus source zip, README, and LICENSE, `prerelease` equals `branch != default`, file targets attach `release-asset-*`, and a no-file-target caller (Docker-only, PyPI-only, source-only) passes `expect_release_assets: false` or the release-create step fails on unmatched files, a source-only one setting every `enable_*` input false with it. A NuGet caller is not one of those, since its leaf uploads a `release-asset-*` carrying the package. - **D4.4** No-op republish on a schedule or push trigger: an unchanged version re-pushes nothing and the release-create skips when the tag exists, while a dispatch re-run refreshes it and runs the paired asset delete with it, registries dedupe server-side under `dotnet nuget push --skip-duplicate` and PyPI's `skip-existing: true`, and Docker always re-pushes by design. -- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch rather than a cleanup, and it repairs that version only while the branch tip has not moved, since NBGV derives the version from git height, so a re-dispatch after the tip has moved publishes a further version and leaves the first release without its package. +- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch while the tip has not moved, since a dispatch names a branch rather than a commit and so builds that branch's tip at dispatch time. Once the tip has moved a re-dispatch builds the new tip instead, and NBGV deriving the version from git height makes that a further version, so the version whose push failed never reaches the registry. **Re-run all jobs** is the recovery there: GitHub replays under the original event's `GITHUB_SHA` and re-executes every job, and the publisher pins the build to that commit, so the same version is rebuilt, its package artifact rebuilt and re-uploaded rather than left missing by D5.2's delete, and its push retried, the release itself needing nothing from the re-run. Three bounds. D4.4's no-op re-run assumes the earlier push succeeded, so it does not describe this one. GitHub offers a re-run only within 30 days of the initial run. And **Re-run failed jobs** is unreliable rather than unavailable, D5.2's delete usually having taken the artifact its download needs while D5.3 leaves that delete best-effort. - **D4.6** A deploy check asserts which release and which environment answer, waiting for convergence to a bounded timeout, with an unreachable host reported distinctly from an HTTP status. ## D5: Resource Cleanup diff --git a/.github/skills/workflow-ci-contract/references/d-guarantees.md b/.github/skills/workflow-ci-contract/references/d-guarantees.md index a49f42ce..7e44658c 100644 --- a/.github/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.github/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator passes `--cov-report=xml` on the Python leg, which names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and the validator reds the test step when no root `coverage.xml` was written. ## D2: Validation at Entry @@ -32,7 +32,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D4.2** `target_commitish` is the built commit's SHA (NBGV `GitCommitId`), never a branch name and never a separately re-resolved ref. - **D4.3** Every release is a tag plus source zip, README, and LICENSE, `prerelease` equals `branch != default`, file targets attach `release-asset-*`, and a no-file-target caller (Docker-only, PyPI-only, source-only) passes `expect_release_assets: false` or the release-create step fails on unmatched files, a source-only one setting every `enable_*` input false with it. A NuGet caller is not one of those, since its leaf uploads a `release-asset-*` carrying the package. - **D4.4** No-op republish on a schedule or push trigger: an unchanged version re-pushes nothing and the release-create skips when the tag exists, while a dispatch re-run refreshes it and runs the paired asset delete with it, registries dedupe server-side under `dotnet nuget push --skip-duplicate` and PyPI's `skip-existing: true`, and Docker always re-pushes by design. -- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch rather than a cleanup, and it repairs that version only while the branch tip has not moved, since NBGV derives the version from git height, so a re-dispatch after the tip has moved publishes a further version and leaves the first release without its package. +- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch while the tip has not moved, since a dispatch names a branch rather than a commit and so builds that branch's tip at dispatch time. Once the tip has moved a re-dispatch builds the new tip instead, and NBGV deriving the version from git height makes that a further version, so the version whose push failed never reaches the registry. **Re-run all jobs** is the recovery there: GitHub replays under the original event's `GITHUB_SHA` and re-executes every job, and the publisher pins the build to that commit, so the same version is rebuilt, its package artifact rebuilt and re-uploaded rather than left missing by D5.2's delete, and its push retried, the release itself needing nothing from the re-run. Three bounds. D4.4's no-op re-run assumes the earlier push succeeded, so it does not describe this one. GitHub offers a re-run only within 30 days of the initial run. And **Re-run failed jobs** is unreliable rather than unavailable, D5.2's delete usually having taken the artifact its download needs while D5.3 leaves that delete best-effort. - **D4.6** A deploy check asserts which release and which environment answer, waiting for convergence to a bounded timeout, with an unreachable host reported distinctly from an HTTP status. ## D5: Resource Cleanup diff --git a/.github/workflows/validate-task.yml b/.github/workflows/validate-task.yml index afec365d..b2aae10e 100644 --- a/.github/workflows/validate-task.yml +++ b/.github/workflows/validate-task.yml @@ -308,9 +308,19 @@ jobs: if: hashFiles('pyproject.toml') != '' && hashFiles('tests/**') != '' && hashFiles('uv.lock') != '' run: uv sync --all-groups --frozen + # --cov-report=xml names the report format and selects nothing to measure, so the repository's own pyproject.toml supplies the --cov selector, per D1.6. + # The report is checked rather than assumed, because the best-effort upload below reads a missing file exactly as it reads a healthy run. + # The pre-run delete makes that a check on what this run wrote, since a committed coverage.xml would otherwise satisfy it without any measurement. - name: Run pytest step if: hashFiles('pyproject.toml') != '' && hashFiles('tests/**') != '' && hashFiles('uv.lock') != '' - run: uv run pytest --cov-report=xml + run: | + set -Eeuo pipefail + rm -f coverage.xml + uv run pytest --cov-report=xml + if [[ ! -s coverage.xml ]]; then + echo "::error::This run wrote no coverage.xml at the repository root. Select a coverage source in this repository's pyproject.toml, an addopts entry of --cov= in practice, since --cov-report=xml alone measures nothing, and leave the report at the root path the upload step below reads." + exit 1 + fi # Best-effort: continue-on-error plus fail_ci_if_error false, so a missing token never reds the gate. - name: Upload coverage to Codecov step (Python) diff --git a/WORKFLOW.md b/WORKFLOW.md index 3a58b3d7..3ee9c86f 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -154,7 +154,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as the o - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated on smoke being false, written `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings). *Prevents: a PR publishing, and orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter marks no target, so smoke-build skips. An inclusion list satisfying D1.1 reaches this by leaving workflow paths out of every target's entry. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, run under `if: always()` so a failed or skipped dependency cannot skip the gate itself, `needs:` the validation job, and the `changes` and `smoke-build` jobs too wherever the repo has a smoke build, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/` and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml` over a repo whose own pytest configuration selects what to measure) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. The Python invocation carries a load-bearing detail of its own, an omission rather than a collision: it names the report format and selects nothing to measure. `pytest-cov` reports on what `--cov` selects, so `--cov-report=xml` on its own measures nothing, writes no file, and exits zero, which the best-effort upload then reads exactly as it reads a healthy run. A Python repo with tests therefore references **`pytest-cov`** in its dev dependency group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repo root as `coverage.xml`, already the one path the upload step names. The validator **fails the test step when that file was not written**, since nothing downstream of it can tell an absent report from an uploaded one, so a repo that redirects the report through `[tool.coverage.xml]` reds the gate rather than uploading nothing from a green run. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/`, `coverage.xml`, and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry @@ -177,13 +177,13 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as the o - **D4.2 Tag the built commit.** Output: the release `target_commitish` is the built commit's SHA (NBGV's `GitCommitId`), never a branch name or a separately re-resolved ref. *Prevents: the tag landing on the default branch instead of the built tree.* - **D4.3 Release contents.** Output: every release contains a tag on the built commit plus the auto source zip, README, and LICENSE. File targets attach `release-asset-*`. The `prerelease` value equals `branch != default`. A no-file-target caller sets `expect_release_assets: false` to reach the no-asset shape. This applies to Docker-only, PyPI-only, and source-only repos. A NuGet target is not among them, since its leaf uploads a `release-asset-*` carrying the package, so a NuGet-only caller keeps the default `true`. The setting relaxes `fail_on_unmatched_files` and skips the asset download. The release-create step fails when no assets exist and the setting retains its default `true`. A source-only caller also sets every `enable_*` input false. - **D4.4 No-op republish.** Input: a re-run whose version is unchanged, on a schedule or push trigger. Output: nothing is re-pushed, because the release-create step is skipped when the tag exists, and the paired asset-delete is skipped with it. A **dispatch** re-run refreshes the release instead and runs that delete with it, which is why a dispatch-only publisher records this item's skip leg as unreachable rather than failed. Registry pushes are no-ops. The NuGet/PyPI publish steps are **not** statically gated on existence. They run and the **server** dedupes (`dotnet nuget push --skip-duplicate` turns a 409 into success, and PyPI does the same under `skip-existing: true`). **Docker always re-pushes** the image (base-image refresh), independently of the release-create skip, within the same run. *Prevents: duplicate releases and wasted pushes.* -- **D4.5 A build failure blocks every publish target.** Input: a real publish where one enabled build fails. Output: nothing publishes. `github-release` needs every build and carries the same `!failure() && !cancelled()` guard the terminal registry pusher (Docker) does, since the implicit `success()` would otherwise skip both on every run that disables a target rather than only on a failed one. A failed build therefore skips the release (no tag, no release), and Docker, which needs every other build, skips with it (no image push) while a disabled or unchanged target (skipped, not failed) still lets docker push. *Prevents: a partial publish, e.g. a Docker image pushed while .NET publish failed and no release was cut.* A package target's separate publish job needs its own gate for the same reason, since it sits outside the `github-release` and Docker `needs:` chains: it `needs:` the release-task call, so a failed build skips it with the rest. What no gate covers is a failed **push**, because that job runs after the whole release task and therefore after `github-release`, for the trusted-publishing reason section 3's package-registry bullet gives. A rejected token exchange, a registry outage, or a trusted-publishing policy naming the wrong workflow file therefore leaves a published release and tag for a version that never reached the registry. The recovery is a re-dispatch rather than a cleanup. A dispatch names a branch, `main` or `develop` per D2.3, and never a commit, so what it builds is that branch's tip at dispatch time. A re-dispatch therefore refreshes the failed version's release (D4.4) and runs its push again only while the tip is still the commit whose push failed. Once the tip has moved a re-dispatch builds the new tip, since NBGV derives the version from git height, so it publishes a further version and leaves the first release without its package. +- **D4.5 A build failure blocks every publish target.** Input: a real publish where one enabled build fails. Output: nothing publishes. `github-release` needs every build and carries the same `!failure() && !cancelled()` guard the terminal registry pusher (Docker) does, since the implicit `success()` would otherwise skip both on every run that disables a target rather than only on a failed one. A failed build therefore skips the release (no tag, no release), and Docker, which needs every other build, skips with it (no image push) while a disabled or unchanged target (skipped, not failed) still lets docker push. *Prevents: a partial publish, e.g. a Docker image pushed while .NET publish failed and no release was cut.* A package target's separate publish job needs its own gate for the same reason, since it sits outside the `github-release` and Docker `needs:` chains: it `needs:` the release-task call, so a failed build skips it with the rest. What no gate covers is a failed **push**, because that job runs after the whole release task and therefore after `github-release`, for the trusted-publishing reason section 3's package-registry bullet gives. A rejected token exchange, a registry outage, or a trusted-publishing policy naming the wrong workflow file therefore leaves a published release and tag for a version that never reached the registry. The recovery is a re-dispatch or a re-run rather than a cleanup, and which of the two applies turns on whether the branch tip has moved. A dispatch names a branch, `main` or `develop` per D2.3, and never a commit, so what it builds is that branch's tip at dispatch time. A re-dispatch therefore refreshes the failed version's release (D4.4) and runs its push again while the tip is still the commit whose push failed. Once the tip has moved a re-dispatch builds the new tip instead. NBGV derives the version from git height, so that is a further version, and the version whose push failed never reaches the registry. **Re-run all jobs** (`gh run rerun `) is the recovery there. GitHub replays a run under the original event's `GITHUB_SHA` and `GITHUB_REF` and re-executes every job rather than only the failed ones, the publisher pins the release task to that commit with `ref: ${{ github.sha }}`, so `get-version` recomputes the version from the same commit and history and each build leaf checks out the `GitCommitId` `get-version` emits, the package artifact D5.2 deleted is rebuilt and re-uploaded rather than missing when `publish-` downloads it, and that job retries the push it failed. The release needs nothing from that re-run, the failed run having already cut it, so whether D4.4's release-create step refreshes or skips does not bear on the recovery. What no route settles in advance is whether the registry accepts the retried push. Three qualifications come with **Re-run all jobs**. D4.4 and 5B's S9 describe a re-run whose predecessor push **succeeded**, where the registry dedupes the second one, so this recovery is the case they do not cover and its retried push is the first the registry ever receives for that version. GitHub offers a re-run only within **30 days** of the initial run, past which a moved tip leaves that version with no route at all. And **Re-run failed jobs** (`--failed`) is unreliable here rather than unavailable. D5.2's delete runs on the path that reaches this case, its gate being `!cancelled()` and the download having succeeded, and it removes the package artifact a `--failed` re-run would download. D5.3 leaves that delete best-effort, so the artifact survives where that delete ran and failed, and `--failed` works in that case alone. - **D4.6 Deploy verification names the release.** Input: a deploy to a filesystem on a host the project owns that completes without error. Output: a check against the running host asserts **which release is answering**, not merely that it answers. The artifact stamps its own version into the configuration it ships, and the check compares that against the version just installed, **waiting for convergence to a bounded timeout** rather than sampling once, because content goes live the instant a pointer moves while server rules wait on an asynchronous reload. The same check asserts **which environment** answered, since several environments serve a byte-identical artifact and a proxy rule aimed at the wrong one answers healthily under the right hostname. An unreachable host is reported distinctly from an HTTP status. *Prevents: a green deploy over a host still serving the previous release's configuration, a URL contract checked against the wrong environment, and a dead config watcher read as a routing fault.* ### D5 - Resource Cleanup - **D5.1 Delete at the point of consumption.** Output: the job that downloads a **cross-job** transfer artifact deletes it (by exact name/pattern) right after consuming it. An intermediate consumed only within the same run MAY instead rely on the `retention-days: 1` backstop. *Prevents: transfer artifacts accumulating against the storage quota.* -- **D5.2 Gate the delete to the condition that made the artifact redundant.** Output: the delete runs exactly when the consumption it follows happened. Where the consumer is a conditional step (the GitHub release create), the delete carries that same condition. Where the consumer is a step that always attempts once its job runs (a package publish job's push), the delete is gated on the **download** having succeeded rather than on the push, as `if: ${{ !cancelled() && steps..outcome == 'success' }}`. A step whose `if:` carries no status-check function, an absent `if:` included, inherits `success()` instead, which skips it on exactly the failed push where the artifact is already downloaded and the release is already cut. So on a no-op re-run that is not a dispatch the `release-asset-*` delete is **skipped** with the release create it follows, while the `nuget-build-*` and `pypi-build-*` deletes still **run**. A dispatch re-run refreshes the release instead (D4.4), so its asset delete runs with it. Deleting the `nuget-build-*` or `pypi-build-*` artifact on the failed-push path costs the run its **Re-run failed jobs** route, since the re-run's download finds nothing, so the recovery for a failed push is the re-dispatch D4.5 names rather than a re-run. *Prevents: deleting freshly built assets on a no-op re-run, and stranding a downloaded artifact when the push it fed fails.* +- **D5.2 Gate the delete to the condition that made the artifact redundant.** Output: the delete runs exactly when the consumption it follows happened. Where the consumer is a conditional step (the GitHub release create), the delete carries that same condition. Where the consumer is a step that always attempts once its job runs (a package publish job's push), the delete is gated on the **download** having succeeded rather than on the push, as `if: ${{ !cancelled() && steps..outcome == 'success' }}`. A step whose `if:` carries no status-check function, an absent `if:` included, inherits `success()` instead, which skips it on exactly the failed push where the artifact is already downloaded and the release is already cut. So on a no-op re-run that is not a dispatch the `release-asset-*` delete is **skipped** with the release create it follows, while the `nuget-build-*` and `pypi-build-*` deletes still **run**. A dispatch re-run refreshes the release instead (D4.4), so its asset delete runs with it. Deleting the `nuget-build-*` or `pypi-build-*` artifact on the failed-push path costs the run its **Re-run failed jobs** route, since the re-run's download then finds nothing, so the recovery for a failed push is the re-dispatch or the full re-run D4.5 names, and D4.5 sets out how far that cost actually reaches. *Prevents: deleting freshly built assets on a no-op re-run, and stranding a downloaded artifact when the push it fed fails.* - **D5.3 Best-effort.** Output: cleanup is `continue-on-error`, tolerates a failed listing, and deletes **all** matching ids. *Prevents: a cleanup hiccup reddening a job whose publish succeeded.* - **D5.4 Retention backstop.** Output: **every** `upload-artifact` sets `retention-days: 1`. - **D5.5 Never blanket-delete.** Output: cleanup MUST NOT enumerate and delete the run's whole artifact set. *Prevents: destroying diagnostic/log artifacts and auto-emitted build-records.* diff --git a/docs/reusable-workflows.md b/docs/reusable-workflows.md index 7b50061b..363c5470 100644 --- a/docs/reusable-workflows.md +++ b/docs/reusable-workflows.md @@ -156,7 +156,7 @@ Hub: `validate-task.yml` hosts a `lint` job (the fleet doc-lint block, language - [x] Hook override path observed on a hub pull request run, [proof run][override-path-run] (runs `./.github/actions/validate`, no hub checkout). Default path observed on PhotoCleaner's adoption pull request, [pilot smoke run][pilot-smoke-run], where the hub's `validate-default` ran because that repo carries no `validate` hook. The follow-up self-reference pilot also runs the bundled prose and repository gates through `$/.github/actions/` without checking out the hub. - [x] PhotoCleaner (pilot, release trigger shape with smoke, the same repo that piloted stage 1): ptr727/PhotoCleaner#55 on `develop` (`c80cb29`), promoted in ptr727/PhotoCleaner#56 (`fa91db0`), both on 2026-08-16. `test-pull-request.yml` calls the hub validate task and no repo hook was needed. - [ ] HomeAutomation-Config (second pilot, operational trigger shape) -- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. +- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job now fails when the run wrote no root `coverage.xml`, so a repo carrying `pyproject.toml`, `tests/`, and `uv.lock` needs `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml`, per D1.6. Measured at the time of that change, aiopurpleair and Financial-Modeling already carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. - [ ] `reports/workflow-reuse.md` regenerated with `validate-task.yml` at 0 copies (a hub-only file no repo carries) and `test-pull-request.yml` showing callers equal to copies. ### Stage 3: The Pure Functions diff --git a/reports/canonical-review.json b/reports/canonical-review.json index 5c5fc848..084ea0d4 100644 --- a/reports/canonical-review.json +++ b/reports/canonical-review.json @@ -283,19 +283,19 @@ }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D1: PR Fast-Feedback (Smoke)", - "digest": "sha256:58e3860e81d92ae6d4ec4a84d04c6770dccad4caa8fdb3c12132f5aa29ffaaa8", + "digest": "sha256:b6250bbeddae5604646198b4e5dc507720b3f065ad9c76ee2b3209e0c4239636", "reviewer": "agent-skill", - "findings": 62, - "hubCommit": "9b33edb7e0f55fe2a05630a49d5be99ba1e3eb26", - "stamp": "2026-09-02T15:16:10Z" + "findings": 0, + "hubCommit": "16daf0dc20e9f11a1485b6835b4f064d6907b273", + "stamp": "2026-09-03T01:05:04Z" }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D4: Release and Publish", - "digest": "sha256:e3fbb2b2de51a2679272e1ff1637681886241da62113ea9020550a3823ef9b43", + "digest": "sha256:2c8b0653258589c31a3f84491da60752deb95131bb5d2a2154a8465bacab69bf", "reviewer": "agent-skill", - "findings": 2, - "hubCommit": "c6e7526cfade19f28f0ab09c4ab283ed00295d7e", - "stamp": "2026-09-02T20:49:13Z" + "findings": 0, + "hubCommit": "16daf0dc20e9f11a1485b6835b4f064d6907b273", + "stamp": "2026-09-03T01:05:09Z" }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D5: Resource Cleanup", @@ -451,11 +451,11 @@ }, { "unit": "WORKFLOW.md > 4. Behavioral Contract: Expected Outcomes", - "digest": "sha256:587d95361c4bd424a65327d3394d266342f13fe05f598b8521bdb7880d781f57", + "digest": "sha256:ea5973704f8a3479bc10ee1aaf2cf2d3d9061b1d0facc8d7b782985026644dcb", "reviewer": "agent-skill", - "findings": 5, - "hubCommit": "7ab81feb4467dd56c0406c1fa98d4f2c94d2ef13", - "stamp": "2026-09-02T21:07:56Z" + "findings": 0, + "hubCommit": "16daf0dc20e9f11a1485b6835b4f064d6907b273", + "stamp": "2026-09-03T01:12:26Z" }, { "unit": "WORKFLOW.md > 5. Test Methodology", diff --git a/spec/project-types.json b/spec/project-types.json index 8c510cc3..eff0d335 100644 --- a/spec/project-types.json +++ b/spec/project-types.json @@ -42,7 +42,7 @@ { "id": "python.pyright.config", "verdict": "intent", "assert": "Build profile: pyright is configured and runs strict on first-party code (src or the integration package) - the strong typing baseline. Third-party strictness is relaxed only where a dependency has no usable types. N/A for the lint-only profile, whose type checker is mypy over stdlib-only code (python.mypy.allowed).", "intentRef": "CODESTYLE.md", "minProfile": "build" }, { "id": "python.config.placement", "verdict": "letter", "assert": "ruff and the type-checker config live in pyproject.toml (canonical); standalone .ruff.toml / pyrightconfig.json is a drift finding. A Home Assistant integration is the exception - it follows home-assistant/core standalone-config conventions and is scored by ha.python.conventions instead.", "intentRef": "CODESTYLE.md" }, { "id": "python.mypy.allowed", "verdict": "intent", "assert": "mypy is permitted as an additional type checker, not banned. It is required for a Home Assistant integration (platinum strict-typing) and is the lint-only profile's type checker. When used it runs in CI and the editor.", "intentRef": "CODESTYLE.md" }, - { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, + { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). Because --cov-report=xml alone measures nothing and writes no file, the repo declares pytest-cov in a dev dependency group and selects the coverage source in its own pyproject.toml, an addopts --cov= entry in practice, leaving the report at the repo root as coverage.xml. The validator deletes any coverage.xml before the run and fails the test step when that file was not written, so a repo missing the selector reds its gate there, while a repo missing pytest-cov reds it earlier, at the pytest invocation that does not recognise the flag. CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, { "id": "python.uvlock.pinned", "verdict": "letter", "assert": "Build profile: the committed uv.lock resolves to LF through the repository-wide .editorconfig and .gitattributes defaults. A CRLF-native operational repo adds a narrow uv.lock LF override only if it adopts the uv build profile. N/A for a non-uv Python repo (a Home Assistant integration on pip/requirements) and for the lint-only profile (no uv.lock by definition).", "intentRef": "GOVERNANCE.md#line-endings", "minProfile": "build" }, { "id": "python.scripts.uvx", "verdict": "letter", "assert": "Lint-only profile only: the tools run via uvx (no project install, no lockfile). A uvx @ pin in a run: step is not Dependabot-trackable, so CI runs uvx ruff@latest / uvx mypy@latest - the fleet rule pins only what Dependabot auto-updates and otherwise runs latest, never a manual pin that goes stale. VS Code tasks, README, and CI all run the unpinned latest. N/A for the build profile (which pins tool versions via uv.lock + uv sync --frozen instead).", "intentRef": "CODESTYLE.md" } ] From 93813f81c86d833aa8be1116d464a9011a641589 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Wed, 2 Sep 2026 19:21:38 -0700 Subject: [PATCH 2/5] State the adopter precondition in the present tense Qodo finding 3 on #1215. The adoption tracker described the guard as a change ("the unit-test job now fails") where comment-and-doc-style requires the present contract: "State what is, never a change from a prior state". Before-and-after framing belongs in the commit message and the PR description, which is where it now lives alone. The same edit drops "Measured at the time of that change" from the per-repo survey for the same reason. --- docs/reusable-workflows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reusable-workflows.md b/docs/reusable-workflows.md index 363c5470..eda895fe 100644 --- a/docs/reusable-workflows.md +++ b/docs/reusable-workflows.md @@ -156,7 +156,7 @@ Hub: `validate-task.yml` hosts a `lint` job (the fleet doc-lint block, language - [x] Hook override path observed on a hub pull request run, [proof run][override-path-run] (runs `./.github/actions/validate`, no hub checkout). Default path observed on PhotoCleaner's adoption pull request, [pilot smoke run][pilot-smoke-run], where the hub's `validate-default` ran because that repo carries no `validate` hook. The follow-up self-reference pilot also runs the bundled prose and repository gates through `$/.github/actions/` without checking out the hub. - [x] PhotoCleaner (pilot, release trigger shape with smoke, the same repo that piloted stage 1): ptr727/PhotoCleaner#55 on `develop` (`c80cb29`), promoted in ptr727/PhotoCleaner#56 (`fa91db0`), both on 2026-08-16. `test-pull-request.yml` calls the hub validate task and no repo hook was needed. - [ ] HomeAutomation-Config (second pilot, operational trigger shape) -- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job now fails when the run wrote no root `coverage.xml`, so a repo carrying `pyproject.toml`, `tests/`, and `uv.lock` needs `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml`, per D1.6. Measured at the time of that change, aiopurpleair and Financial-Modeling already carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. +- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, so a repo carrying `pyproject.toml`, `tests/`, and `uv.lock` needs `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml`, per D1.6. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. - [ ] `reports/workflow-reuse.md` regenerated with `validate-task.yml` at 0 copies (a hub-only file no repo carries) and `test-pull-request.yml` showing callers equal to copies. ### Stage 3: The Pure Functions From 4fdc718663d0149994e496e890c20c7a77f27c96 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Wed, 2 Sep 2026 19:51:27 -0700 Subject: [PATCH 3/5] State D1.6's coverage precondition on all four surfaces CodeRabbit's Major finding on #1215. D1.6 requires coverage for every Python repo with tests, while the hub validator's Python leg runs only where the tree carries pyproject.toml, tests/ and uv.lock, so a tested repo without a lockfile silently gets neither the coverage nor the missing-report failure this branch adds. homeassistant-purpleair is that shape and meets D1.6 through its own workflows. All four surfaces now carry the precondition, attached to the enforcement and never to the obligation: WORKFLOW.md D1.6, the condensed D1.6 bullet, the python.coverage.codecov assert, and the rollout tracker's adopter note. Two earlier attempts at this are why the wording is careful. The first exempted a repo outside that shape from the --cov selector obligation itself, which draws the boundary around the wrong mechanism: selecting a coverage source is pytest-cov's behaviour rather than the validator's, so a repo running the same invocation in its own workflow hits the same silent green. The second reworded that exemption and collided with the lint-only carve-out, since a lint-only tree is defined by carrying no uv.lock and so is always outside the shape. The relaxation is deleted rather than reworded, and the precondition now states only the fact. Also corrects a false identity, the pytest step sits in the Python leg rather than being it, and binds the report-format clause to the flag rather than to the leg. --- .../references/d-guarantees.md | 2 +- .claude-plugin/fleet-skills/.source-digest | 2 +- .../references/d-guarantees.md | 2 +- .../references/d-guarantees.md | 2 +- WORKFLOW.md | 2 +- docs/reusable-workflows.md | 2 +- reports/canonical-review.json | 16 ++++++++-------- spec/project-types.json | 2 +- 8 files changed, 15 insertions(+), 15 deletions(-) diff --git a/.agents/skills/workflow-ci-contract/references/d-guarantees.md b/.agents/skills/workflow-ci-contract/references/d-guarantees.md index 7e44658c..7c0d8245 100644 --- a/.agents/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.agents/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator passes `--cov-report=xml` on the Python leg, which names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and the validator reds the test step when no root `coverage.xml` was written. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, passes `--cov-report=xml`, a flag that names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and that leg reds its test step when no root `coverage.xml` was written. ## D2: Validation at Entry diff --git a/.claude-plugin/fleet-skills/.source-digest b/.claude-plugin/fleet-skills/.source-digest index 4810f274..5ed15cfd 100644 --- a/.claude-plugin/fleet-skills/.source-digest +++ b/.claude-plugin/fleet-skills/.source-digest @@ -1 +1 @@ -b70d364d5a8969ac +a7755b54bec20159 diff --git a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md index 7e44658c..7c0d8245 100644 --- a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator passes `--cov-report=xml` on the Python leg, which names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and the validator reds the test step when no root `coverage.xml` was written. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, passes `--cov-report=xml`, a flag that names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and that leg reds its test step when no root `coverage.xml` was written. ## D2: Validation at Entry diff --git a/.github/skills/workflow-ci-contract/references/d-guarantees.md b/.github/skills/workflow-ci-contract/references/d-guarantees.md index 7e44658c..7c0d8245 100644 --- a/.github/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.github/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator passes `--cov-report=xml` on the Python leg, which names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and the validator reds the test step when no root `coverage.xml` was written. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, passes `--cov-report=xml`, a flag that names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and that leg reds its test step when no root `coverage.xml` was written. ## D2: Validation at Entry diff --git a/WORKFLOW.md b/WORKFLOW.md index 3ee9c86f..0e26533f 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -154,7 +154,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as the o - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated on smoke being false, written `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings). *Prevents: a PR publishing, and orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter marks no target, so smoke-build skips. An inclusion list satisfying D1.1 reaches this by leaving workflow paths out of every target's entry. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, run under `if: always()` so a failed or skipped dependency cannot skip the gate itself, `needs:` the validation job, and the `changes` and `smoke-build` jobs too wherever the repo has a smoke build, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml` over a repo whose own pytest configuration selects what to measure) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. The Python invocation carries a load-bearing detail of its own, an omission rather than a collision: it names the report format and selects nothing to measure. `pytest-cov` reports on what `--cov` selects, so `--cov-report=xml` on its own measures nothing, writes no file, and exits zero, which the best-effort upload then reads exactly as it reads a healthy run. A Python repo with tests therefore references **`pytest-cov`** in its dev dependency group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repo root as `coverage.xml`, already the one path the upload step names. The validator **fails the test step when that file was not written**, since nothing downstream of it can tell an absent report from an uploaded one, so a repo that redirects the report through `[tool.coverage.xml]` reds the gate rather than uploading nothing from a green run. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/`, `coverage.xml`, and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml` over a repo whose own pytest configuration selects what to measure) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. The Python invocation carries a load-bearing detail of its own, an omission rather than a collision: it names the report format and selects nothing to measure. `pytest-cov` reports on what `--cov` selects, so `--cov-report=xml` on its own measures nothing, writes no file, and exits zero, which the best-effort upload then reads exactly as it reads a healthy run. A Python repo with tests therefore references **`pytest-cov`** in its dev dependency group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repo root as `coverage.xml`, already the one path the upload step names. The validator **fails the test step when that file was not written**, since nothing downstream of it can tell an absent report from an uploaded one, so a repo that redirects the report through `[tool.coverage.xml]` reds the gate rather than uploading nothing from a green run. That step sits in the hub validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/`, `coverage.xml`, and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry diff --git a/docs/reusable-workflows.md b/docs/reusable-workflows.md index eda895fe..8c7e4d00 100644 --- a/docs/reusable-workflows.md +++ b/docs/reusable-workflows.md @@ -156,7 +156,7 @@ Hub: `validate-task.yml` hosts a `lint` job (the fleet doc-lint block, language - [x] Hook override path observed on a hub pull request run, [proof run][override-path-run] (runs `./.github/actions/validate`, no hub checkout). Default path observed on PhotoCleaner's adoption pull request, [pilot smoke run][pilot-smoke-run], where the hub's `validate-default` ran because that repo carries no `validate` hook. The follow-up self-reference pilot also runs the bundled prose and repository gates through `$/.github/actions/` without checking out the hub. - [x] PhotoCleaner (pilot, release trigger shape with smoke, the same repo that piloted stage 1): ptr727/PhotoCleaner#55 on `develop` (`c80cb29`), promoted in ptr727/PhotoCleaner#56 (`fa91db0`), both on 2026-08-16. `test-pull-request.yml` calls the hub validate task and no repo hook was needed. - [ ] HomeAutomation-Config (second pilot, operational trigger shape) -- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, so a repo carrying `pyproject.toml`, `tests/`, and `uv.lock` needs `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml`, per D1.6. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. +- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, and that job's Python leg runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, so an adopter of that shape puts `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml` before it bumps. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. - [ ] `reports/workflow-reuse.md` regenerated with `validate-task.yml` at 0 copies (a hub-only file no repo carries) and `test-pull-request.yml` showing callers equal to copies. ### Stage 3: The Pure Functions diff --git a/reports/canonical-review.json b/reports/canonical-review.json index 084ea0d4..83630507 100644 --- a/reports/canonical-review.json +++ b/reports/canonical-review.json @@ -283,11 +283,11 @@ }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D1: PR Fast-Feedback (Smoke)", - "digest": "sha256:b6250bbeddae5604646198b4e5dc507720b3f065ad9c76ee2b3209e0c4239636", + "digest": "sha256:834ab658fac9ea1a4cdef3d4a552e4fd735bc5e715d7da6d89dd8a90458dfdf9", "reviewer": "agent-skill", - "findings": 0, - "hubCommit": "16daf0dc20e9f11a1485b6835b4f064d6907b273", - "stamp": "2026-09-03T01:05:04Z" + "findings": 2, + "hubCommit": "93813f81c86d833aa8be1116d464a9011a641589", + "stamp": "2026-09-03T02:51:09Z" }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D4: Release and Publish", @@ -451,11 +451,11 @@ }, { "unit": "WORKFLOW.md > 4. Behavioral Contract: Expected Outcomes", - "digest": "sha256:ea5973704f8a3479bc10ee1aaf2cf2d3d9061b1d0facc8d7b782985026644dcb", + "digest": "sha256:4e53d2cebad9b3e639abc80cd66ecdddab65fe2f23e919bc86b294ccd54be588", "reviewer": "agent-skill", - "findings": 0, - "hubCommit": "16daf0dc20e9f11a1485b6835b4f064d6907b273", - "stamp": "2026-09-03T01:12:26Z" + "findings": 1, + "hubCommit": "93813f81c86d833aa8be1116d464a9011a641589", + "stamp": "2026-09-03T02:51:03Z" }, { "unit": "WORKFLOW.md > 5. Test Methodology", diff --git a/spec/project-types.json b/spec/project-types.json index eff0d335..9d590282 100644 --- a/spec/project-types.json +++ b/spec/project-types.json @@ -42,7 +42,7 @@ { "id": "python.pyright.config", "verdict": "intent", "assert": "Build profile: pyright is configured and runs strict on first-party code (src or the integration package) - the strong typing baseline. Third-party strictness is relaxed only where a dependency has no usable types. N/A for the lint-only profile, whose type checker is mypy over stdlib-only code (python.mypy.allowed).", "intentRef": "CODESTYLE.md", "minProfile": "build" }, { "id": "python.config.placement", "verdict": "letter", "assert": "ruff and the type-checker config live in pyproject.toml (canonical); standalone .ruff.toml / pyrightconfig.json is a drift finding. A Home Assistant integration is the exception - it follows home-assistant/core standalone-config conventions and is scored by ha.python.conventions instead.", "intentRef": "CODESTYLE.md" }, { "id": "python.mypy.allowed", "verdict": "intent", "assert": "mypy is permitted as an additional type checker, not banned. It is required for a Home Assistant integration (platinum strict-typing) and is the lint-only profile's type checker. When used it runs in CI and the editor.", "intentRef": "CODESTYLE.md" }, - { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). Because --cov-report=xml alone measures nothing and writes no file, the repo declares pytest-cov in a dev dependency group and selects the coverage source in its own pyproject.toml, an addopts --cov= entry in practice, leaving the report at the repo root as coverage.xml. The validator deletes any coverage.xml before the run and fails the test step when that file was not written, so a repo missing the selector reds its gate there, while a repo missing pytest-cov reds it earlier, at the pytest invocation that does not recognise the flag. CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, + { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). Because --cov-report=xml alone measures nothing and writes no file, the repo declares pytest-cov in a dev dependency group and selects the coverage source in its own pyproject.toml, an addopts --cov= entry in practice, leaving the report at the repo root as coverage.xml. The validator's Python leg, which runs where the tree carries pyproject.toml, tests/ and uv.lock, deletes any coverage.xml before the run and fails the test step when that file was not written, so a repo of that shape missing the selector reds its gate there, while one missing pytest-cov reds it earlier, at the pytest invocation that does not recognize the flag. CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, { "id": "python.uvlock.pinned", "verdict": "letter", "assert": "Build profile: the committed uv.lock resolves to LF through the repository-wide .editorconfig and .gitattributes defaults. A CRLF-native operational repo adds a narrow uv.lock LF override only if it adopts the uv build profile. N/A for a non-uv Python repo (a Home Assistant integration on pip/requirements) and for the lint-only profile (no uv.lock by definition).", "intentRef": "GOVERNANCE.md#line-endings", "minProfile": "build" }, { "id": "python.scripts.uvx", "verdict": "letter", "assert": "Lint-only profile only: the tools run via uvx (no project install, no lockfile). A uvx @ pin in a run: step is not Dependabot-trackable, so CI runs uvx ruff@latest / uvx mypy@latest - the fleet rule pins only what Dependabot auto-updates and otherwise runs latest, never a manual pin that goes stale. VS Code tasks, README, and CI all run the unpinned latest. N/A for the build profile (which pins tool versions via uv.lock + uv sync --frozen instead).", "intentRef": "CODESTYLE.md" } ] From 73175e0c985c9302814428f4b96489fbf7eb5d9b Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Wed, 2 Sep 2026 20:12:33 -0700 Subject: [PATCH 4/5] Name the root the coverage guard actually tests Follow-up to the precondition commit, from the review passes on #1215. Three corrections, all to the same rule's four surfaces. hashFiles patterns resolve against GITHUB_WORKSPACE and do not recurse, so the guard tests the repository root rather than the tree. A repo whose Python sits in a subdirectory carries all three files and still never runs the leg. All four surfaces now say root. The condensed D1.6 bullet stated the selector obligation as a consequence of the hub validator's leg, in one so-chain that also said the leg runs only under a three-file root. A reader outside that shape could take the obligation as ungrounded for them. The chain is split: the obligation stands in its own sentence, complete, and enforcement follows in another. The single chain was the structural cause, since whatever precedes the so becomes the reader's ground. That bullet also read 'every Python repo with tests', which is false for the lint-only profile, whose pyproject carries only tool config and no dependency group. A lint-only repo obeying it would have added a group and broken its own profile detection, and this repository is lint-only with tests. It now reads 'lint-only excepted', and carries the report-path obligation that previously existed only inside the leg-preconditioned clause. --- .../references/d-guarantees.md | 2 +- .claude-plugin/fleet-skills/.source-digest | 2 +- .../references/d-guarantees.md | 2 +- .../references/d-guarantees.md | 2 +- WORKFLOW.md | 2 +- docs/reusable-workflows.md | 2 +- reports/canonical-review.json | 16 ++++++++-------- spec/project-types.json | 2 +- 8 files changed, 15 insertions(+), 15 deletions(-) diff --git a/.agents/skills/workflow-ci-contract/references/d-guarantees.md b/.agents/skills/workflow-ci-contract/references/d-guarantees.md index 7c0d8245..31d837d5 100644 --- a/.agents/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.agents/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, passes `--cov-report=xml`, a flag that names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and that leg reds its test step when no root `coverage.xml` was written. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The Python invocation, `pytest --cov-report=xml`, names a report format and selects nothing to measure, so a Python repo with tests, lint-only excepted, carries `pytest-cov` in a dev group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repository root as `coverage.xml`. The hub validator's Python leg, which runs where that root carries `pyproject.toml`, `tests/`, and `uv.lock`, reds its test step when no such report was written. ## D2: Validation at Entry diff --git a/.claude-plugin/fleet-skills/.source-digest b/.claude-plugin/fleet-skills/.source-digest index 5ed15cfd..39c23d5d 100644 --- a/.claude-plugin/fleet-skills/.source-digest +++ b/.claude-plugin/fleet-skills/.source-digest @@ -1 +1 @@ -a7755b54bec20159 +48c6cd114e4cc195 diff --git a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md index 7c0d8245..31d837d5 100644 --- a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, passes `--cov-report=xml`, a flag that names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and that leg reds its test step when no root `coverage.xml` was written. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The Python invocation, `pytest --cov-report=xml`, names a report format and selects nothing to measure, so a Python repo with tests, lint-only excepted, carries `pytest-cov` in a dev group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repository root as `coverage.xml`. The hub validator's Python leg, which runs where that root carries `pyproject.toml`, `tests/`, and `uv.lock`, reds its test step when no such report was written. ## D2: Validation at Entry diff --git a/.github/skills/workflow-ci-contract/references/d-guarantees.md b/.github/skills/workflow-ci-contract/references/d-guarantees.md index 7c0d8245..31d837d5 100644 --- a/.github/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.github/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, passes `--cov-report=xml`, a flag that names a report format and selects nothing to measure, so a Python repo carries `pytest-cov` in a dev group and selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and that leg reds its test step when no root `coverage.xml` was written. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The Python invocation, `pytest --cov-report=xml`, names a report format and selects nothing to measure, so a Python repo with tests, lint-only excepted, carries `pytest-cov` in a dev group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repository root as `coverage.xml`. The hub validator's Python leg, which runs where that root carries `pyproject.toml`, `tests/`, and `uv.lock`, reds its test step when no such report was written. ## D2: Validation at Entry diff --git a/WORKFLOW.md b/WORKFLOW.md index 0e26533f..8e7dbc2f 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -154,7 +154,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as the o - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated on smoke being false, written `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings). *Prevents: a PR publishing, and orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter marks no target, so smoke-build skips. An inclusion list satisfying D1.1 reaches this by leaving workflow paths out of every target's entry. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, run under `if: always()` so a failed or skipped dependency cannot skip the gate itself, `needs:` the validation job, and the `changes` and `smoke-build` jobs too wherever the repo has a smoke build, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml` over a repo whose own pytest configuration selects what to measure) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. The Python invocation carries a load-bearing detail of its own, an omission rather than a collision: it names the report format and selects nothing to measure. `pytest-cov` reports on what `--cov` selects, so `--cov-report=xml` on its own measures nothing, writes no file, and exits zero, which the best-effort upload then reads exactly as it reads a healthy run. A Python repo with tests therefore references **`pytest-cov`** in its dev dependency group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repo root as `coverage.xml`, already the one path the upload step names. The validator **fails the test step when that file was not written**, since nothing downstream of it can tell an absent report from an uploaded one, so a repo that redirects the report through `[tool.coverage.xml]` reds the gate rather than uploading nothing from a green run. That step sits in the hub validator's Python leg, which runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/`, `coverage.xml`, and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml` over a repo whose own pytest configuration selects what to measure) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. The Python invocation carries a load-bearing detail of its own, an omission rather than a collision: it names the report format and selects nothing to measure. `pytest-cov` reports on what `--cov` selects, so `--cov-report=xml` on its own measures nothing, writes no file, and exits zero, which the best-effort upload then reads exactly as it reads a healthy run. A Python repo with tests therefore references **`pytest-cov`** in its dev dependency group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repo root as `coverage.xml`, already the one path the upload step names. The validator **fails the test step when that file was not written**, since nothing downstream of it can tell an absent report from an uploaded one, so a repo that redirects the report through `[tool.coverage.xml]` reds the gate rather than uploading nothing from a green run. That step sits in the hub validator's Python leg, which runs where the repository root carries `pyproject.toml`, `tests/`, and `uv.lock`. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/`, `coverage.xml`, and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry diff --git a/docs/reusable-workflows.md b/docs/reusable-workflows.md index 8c7e4d00..d6e946c1 100644 --- a/docs/reusable-workflows.md +++ b/docs/reusable-workflows.md @@ -156,7 +156,7 @@ Hub: `validate-task.yml` hosts a `lint` job (the fleet doc-lint block, language - [x] Hook override path observed on a hub pull request run, [proof run][override-path-run] (runs `./.github/actions/validate`, no hub checkout). Default path observed on PhotoCleaner's adoption pull request, [pilot smoke run][pilot-smoke-run], where the hub's `validate-default` ran because that repo carries no `validate` hook. The follow-up self-reference pilot also runs the bundled prose and repository gates through `$/.github/actions/` without checking out the hub. - [x] PhotoCleaner (pilot, release trigger shape with smoke, the same repo that piloted stage 1): ptr727/PhotoCleaner#55 on `develop` (`c80cb29`), promoted in ptr727/PhotoCleaner#56 (`fa91db0`), both on 2026-08-16. `test-pull-request.yml` calls the hub validate task and no repo hook was needed. - [ ] HomeAutomation-Config (second pilot, operational trigger shape) -- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, and that job's Python leg runs where the tree carries `pyproject.toml`, `tests/`, and `uv.lock`, so an adopter of that shape puts `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml` before it bumps. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. +- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, and that job's Python leg runs where the repository root carries `pyproject.toml`, `tests/`, and `uv.lock`, so an adopter of that shape puts `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml` before it bumps, per D1.6, which binds that selector for every Python repo with tests. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. - [ ] `reports/workflow-reuse.md` regenerated with `validate-task.yml` at 0 copies (a hub-only file no repo carries) and `test-pull-request.yml` showing callers equal to copies. ### Stage 3: The Pure Functions diff --git a/reports/canonical-review.json b/reports/canonical-review.json index 83630507..4eed92ee 100644 --- a/reports/canonical-review.json +++ b/reports/canonical-review.json @@ -283,11 +283,11 @@ }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D1: PR Fast-Feedback (Smoke)", - "digest": "sha256:834ab658fac9ea1a4cdef3d4a552e4fd735bc5e715d7da6d89dd8a90458dfdf9", + "digest": "sha256:6c57b1c6eed34177bc1f1dacef05725efc19e0b385b103a6d0a01c9265b41e93", "reviewer": "agent-skill", - "findings": 2, - "hubCommit": "93813f81c86d833aa8be1116d464a9011a641589", - "stamp": "2026-09-03T02:51:09Z" + "findings": 0, + "hubCommit": "4fdc718663d0149994e496e890c20c7a77f27c96", + "stamp": "2026-09-03T03:12:20Z" }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D4: Release and Publish", @@ -451,11 +451,11 @@ }, { "unit": "WORKFLOW.md > 4. Behavioral Contract: Expected Outcomes", - "digest": "sha256:4e53d2cebad9b3e639abc80cd66ecdddab65fe2f23e919bc86b294ccd54be588", + "digest": "sha256:a4b85d3b43f98646b6cf1a7161acacc09f3e4b8f3c23c1197af69d13bba74d84", "reviewer": "agent-skill", - "findings": 1, - "hubCommit": "93813f81c86d833aa8be1116d464a9011a641589", - "stamp": "2026-09-03T02:51:03Z" + "findings": 0, + "hubCommit": "4fdc718663d0149994e496e890c20c7a77f27c96", + "stamp": "2026-09-03T03:04:24Z" }, { "unit": "WORKFLOW.md > 5. Test Methodology", diff --git a/spec/project-types.json b/spec/project-types.json index 9d590282..f329ac22 100644 --- a/spec/project-types.json +++ b/spec/project-types.json @@ -42,7 +42,7 @@ { "id": "python.pyright.config", "verdict": "intent", "assert": "Build profile: pyright is configured and runs strict on first-party code (src or the integration package) - the strong typing baseline. Third-party strictness is relaxed only where a dependency has no usable types. N/A for the lint-only profile, whose type checker is mypy over stdlib-only code (python.mypy.allowed).", "intentRef": "CODESTYLE.md", "minProfile": "build" }, { "id": "python.config.placement", "verdict": "letter", "assert": "ruff and the type-checker config live in pyproject.toml (canonical); standalone .ruff.toml / pyrightconfig.json is a drift finding. A Home Assistant integration is the exception - it follows home-assistant/core standalone-config conventions and is scored by ha.python.conventions instead.", "intentRef": "CODESTYLE.md" }, { "id": "python.mypy.allowed", "verdict": "intent", "assert": "mypy is permitted as an additional type checker, not banned. It is required for a Home Assistant integration (platinum strict-typing) and is the lint-only profile's type checker. When used it runs in CI and the editor.", "intentRef": "CODESTYLE.md" }, - { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). Because --cov-report=xml alone measures nothing and writes no file, the repo declares pytest-cov in a dev dependency group and selects the coverage source in its own pyproject.toml, an addopts --cov= entry in practice, leaving the report at the repo root as coverage.xml. The validator's Python leg, which runs where the tree carries pyproject.toml, tests/ and uv.lock, deletes any coverage.xml before the run and fails the test step when that file was not written, so a repo of that shape missing the selector reds its gate there, while one missing pytest-cov reds it earlier, at the pytest invocation that does not recognize the flag. CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, + { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). Because --cov-report=xml alone measures nothing and writes no file, the repo declares pytest-cov in a dev dependency group and selects the coverage source in its own pyproject.toml, an addopts --cov= entry in practice, leaving the report at the repo root as coverage.xml. The validator's Python leg, which runs where the repository root carries pyproject.toml, tests/ and uv.lock, deletes any coverage.xml before the run and fails the test step when that file was not written, so a repo of that shape missing the selector reds its gate there, while one missing pytest-cov reds it earlier, at the pytest invocation that does not recognize the flag. CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, { "id": "python.uvlock.pinned", "verdict": "letter", "assert": "Build profile: the committed uv.lock resolves to LF through the repository-wide .editorconfig and .gitattributes defaults. A CRLF-native operational repo adds a narrow uv.lock LF override only if it adopts the uv build profile. N/A for a non-uv Python repo (a Home Assistant integration on pip/requirements) and for the lint-only profile (no uv.lock by definition).", "intentRef": "GOVERNANCE.md#line-endings", "minProfile": "build" }, { "id": "python.scripts.uvx", "verdict": "letter", "assert": "Lint-only profile only: the tools run via uvx (no project install, no lockfile). A uvx @ pin in a run: step is not Dependabot-trackable, so CI runs uvx ruff@latest / uvx mypy@latest - the fleet rule pins only what Dependabot auto-updates and otherwise runs latest, never a manual pin that goes stale. VS Code tasks, README, and CI all run the unpinned latest. N/A for the build profile (which pins tool versions via uv.lock + uv sync --frozen instead).", "intentRef": "CODESTYLE.md" } ] From a9e8e3157e3c5538f8ceebe5189117400f7ce6a0 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Wed, 2 Sep 2026 20:23:43 -0700 Subject: [PATCH 5/5] Carry the lint-only exception into the adopter note The note cited D1.6 as binding the selector for every Python repo with tests, dropping the exception the other three surfaces carry. A lint-only Python repo does have a test suite, so the unqualified reading pulls one in, and its pyproject may carry only tool config. Found by the diff pass on #1215, the same class as the catalog fix in 73175e0 and the last surface still stating it. --- docs/reusable-workflows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reusable-workflows.md b/docs/reusable-workflows.md index d6e946c1..942bd097 100644 --- a/docs/reusable-workflows.md +++ b/docs/reusable-workflows.md @@ -156,7 +156,7 @@ Hub: `validate-task.yml` hosts a `lint` job (the fleet doc-lint block, language - [x] Hook override path observed on a hub pull request run, [proof run][override-path-run] (runs `./.github/actions/validate`, no hub checkout). Default path observed on PhotoCleaner's adoption pull request, [pilot smoke run][pilot-smoke-run], where the hub's `validate-default` ran because that repo carries no `validate` hook. The follow-up self-reference pilot also runs the bundled prose and repository gates through `$/.github/actions/` without checking out the hub. - [x] PhotoCleaner (pilot, release trigger shape with smoke, the same repo that piloted stage 1): ptr727/PhotoCleaner#55 on `develop` (`c80cb29`), promoted in ptr727/PhotoCleaner#56 (`fa91db0`), both on 2026-08-16. `test-pull-request.yml` calls the hub validate task and no repo hook was needed. - [ ] HomeAutomation-Config (second pilot, operational trigger shape) -- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, and that job's Python leg runs where the repository root carries `pyproject.toml`, `tests/`, and `uv.lock`, so an adopter of that shape puts `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml` before it bumps, per D1.6, which binds that selector for every Python repo with tests. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. +- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, and that job's Python leg runs where the repository root carries `pyproject.toml`, `tests/`, and `uv.lock`, so an adopter of that shape puts `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml` before it bumps, per D1.6, which binds that selector for every Python repo with tests, lint-only excepted. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. - [ ] `reports/workflow-reuse.md` regenerated with `validate-task.yml` at 0 copies (a hub-only file no repo carries) and `test-pull-request.yml` showing callers equal to copies. ### Stage 3: The Pure Functions