diff --git a/.agents/skills/workflow-ci-contract/references/d-guarantees.md b/.agents/skills/workflow-ci-contract/references/d-guarantees.md index a49f42ce..31d837d5 100644 --- a/.agents/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.agents/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The Python invocation, `pytest --cov-report=xml`, names a report format and selects nothing to measure, so a Python repo with tests, lint-only excepted, carries `pytest-cov` in a dev group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repository root as `coverage.xml`. The hub validator's Python leg, which runs where that root carries `pyproject.toml`, `tests/`, and `uv.lock`, reds its test step when no such report was written. ## D2: Validation at Entry @@ -32,7 +32,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D4.2** `target_commitish` is the built commit's SHA (NBGV `GitCommitId`), never a branch name and never a separately re-resolved ref. - **D4.3** Every release is a tag plus source zip, README, and LICENSE, `prerelease` equals `branch != default`, file targets attach `release-asset-*`, and a no-file-target caller (Docker-only, PyPI-only, source-only) passes `expect_release_assets: false` or the release-create step fails on unmatched files, a source-only one setting every `enable_*` input false with it. A NuGet caller is not one of those, since its leaf uploads a `release-asset-*` carrying the package. - **D4.4** No-op republish on a schedule or push trigger: an unchanged version re-pushes nothing and the release-create skips when the tag exists, while a dispatch re-run refreshes it and runs the paired asset delete with it, registries dedupe server-side under `dotnet nuget push --skip-duplicate` and PyPI's `skip-existing: true`, and Docker always re-pushes by design. -- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch rather than a cleanup, and it repairs that version only while the branch tip has not moved, since NBGV derives the version from git height, so a re-dispatch after the tip has moved publishes a further version and leaves the first release without its package. +- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch while the tip has not moved, since a dispatch names a branch rather than a commit and so builds that branch's tip at dispatch time. Once the tip has moved a re-dispatch builds the new tip instead, and NBGV deriving the version from git height makes that a further version, so the version whose push failed never reaches the registry. **Re-run all jobs** is the recovery there: GitHub replays under the original event's `GITHUB_SHA` and re-executes every job, and the publisher pins the build to that commit, so the same version is rebuilt, its package artifact rebuilt and re-uploaded rather than left missing by D5.2's delete, and its push retried, the release itself needing nothing from the re-run. Three bounds. D4.4's no-op re-run assumes the earlier push succeeded, so it does not describe this one. GitHub offers a re-run only within 30 days of the initial run. And **Re-run failed jobs** is unreliable rather than unavailable, D5.2's delete usually having taken the artifact its download needs while D5.3 leaves that delete best-effort. - **D4.6** A deploy check asserts which release and which environment answer, waiting for convergence to a bounded timeout, with an unreachable host reported distinctly from an HTTP status. ## D5: Resource Cleanup diff --git a/.claude-plugin/fleet-skills/.source-digest b/.claude-plugin/fleet-skills/.source-digest index 01ad8951..39c23d5d 100644 --- a/.claude-plugin/fleet-skills/.source-digest +++ b/.claude-plugin/fleet-skills/.source-digest @@ -1 +1 @@ -9cf4ad34f6781963 +48c6cd114e4cc195 diff --git a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md index a49f42ce..31d837d5 100644 --- a/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.claude-plugin/fleet-skills/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The Python invocation, `pytest --cov-report=xml`, names a report format and selects nothing to measure, so a Python repo with tests, lint-only excepted, carries `pytest-cov` in a dev group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repository root as `coverage.xml`. The hub validator's Python leg, which runs where that root carries `pyproject.toml`, `tests/`, and `uv.lock`, reds its test step when no such report was written. ## D2: Validation at Entry @@ -32,7 +32,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D4.2** `target_commitish` is the built commit's SHA (NBGV `GitCommitId`), never a branch name and never a separately re-resolved ref. - **D4.3** Every release is a tag plus source zip, README, and LICENSE, `prerelease` equals `branch != default`, file targets attach `release-asset-*`, and a no-file-target caller (Docker-only, PyPI-only, source-only) passes `expect_release_assets: false` or the release-create step fails on unmatched files, a source-only one setting every `enable_*` input false with it. A NuGet caller is not one of those, since its leaf uploads a `release-asset-*` carrying the package. - **D4.4** No-op republish on a schedule or push trigger: an unchanged version re-pushes nothing and the release-create skips when the tag exists, while a dispatch re-run refreshes it and runs the paired asset delete with it, registries dedupe server-side under `dotnet nuget push --skip-duplicate` and PyPI's `skip-existing: true`, and Docker always re-pushes by design. -- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch rather than a cleanup, and it repairs that version only while the branch tip has not moved, since NBGV derives the version from git height, so a re-dispatch after the tip has moved publishes a further version and leaves the first release without its package. +- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch while the tip has not moved, since a dispatch names a branch rather than a commit and so builds that branch's tip at dispatch time. Once the tip has moved a re-dispatch builds the new tip instead, and NBGV deriving the version from git height makes that a further version, so the version whose push failed never reaches the registry. **Re-run all jobs** is the recovery there: GitHub replays under the original event's `GITHUB_SHA` and re-executes every job, and the publisher pins the build to that commit, so the same version is rebuilt, its package artifact rebuilt and re-uploaded rather than left missing by D5.2's delete, and its push retried, the release itself needing nothing from the re-run. Three bounds. D4.4's no-op re-run assumes the earlier push succeeded, so it does not describe this one. GitHub offers a re-run only within 30 days of the initial run. And **Re-run failed jobs** is unreliable rather than unavailable, D5.2's delete usually having taken the artifact its download needs while D5.3 leaves that delete best-effort. - **D4.6** A deploy check asserts which release and which environment answer, waiting for convergence to a bounded timeout, with an unreachable host reported distinctly from an HTTP status. ## D5: Resource Cleanup diff --git a/.github/skills/workflow-ci-contract/references/d-guarantees.md b/.github/skills/workflow-ci-contract/references/d-guarantees.md index a49f42ce..31d837d5 100644 --- a/.github/skills/workflow-ci-contract/references/d-guarantees.md +++ b/.github/skills/workflow-ci-contract/references/d-guarantees.md @@ -9,7 +9,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D1.3** Smoke never publishes and never uploads: full compile/lint/test, no pushes, every `upload-artifact` gated on smoke being false, `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings. Prevents a PR publishing and orphaned artifacts. - **D1.4** A PR changing only `.github/workflows/**` is not smoke-built, since an inclusion list satisfying D1.1 matches no workflow path, and actionlint still validates them. - **D1.5** One required aggregator gates merge: `if: always()`, `needs:` the validation job plus the `changes` and `smoke-build` jobs wherever the repo has a smoke build, passes on skipped smoke, blocks on failure or cancelled, and its name is ruleset-bound (job `name:` equals ruleset `context:`, renamed together). -- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. +- **D1.6** Coverage reports to Codecov for C# and Python repos with tests, a lint-only profile for that type excepted, the upload best-effort so an outage never reds the gate, with a `codecov.yml` setting statuses informational and `.gitignore` excluding coverage output. The Python invocation, `pytest --cov-report=xml`, names a report format and selects nothing to measure, so a Python repo with tests, lint-only excepted, carries `pytest-cov` in a dev group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repository root as `coverage.xml`. The hub validator's Python leg, which runs where that root carries `pyproject.toml`, `tests/`, and `uv.lock`, reds its test step when no such report was written. ## D2: Validation at Entry @@ -32,7 +32,7 @@ Each guarantee is a MUST from `WORKFLOW.md` section 4, stated as the output a co - **D4.2** `target_commitish` is the built commit's SHA (NBGV `GitCommitId`), never a branch name and never a separately re-resolved ref. - **D4.3** Every release is a tag plus source zip, README, and LICENSE, `prerelease` equals `branch != default`, file targets attach `release-asset-*`, and a no-file-target caller (Docker-only, PyPI-only, source-only) passes `expect_release_assets: false` or the release-create step fails on unmatched files, a source-only one setting every `enable_*` input false with it. A NuGet caller is not one of those, since its leaf uploads a `release-asset-*` carrying the package. - **D4.4** No-op republish on a schedule or push trigger: an unchanged version re-pushes nothing and the release-create skips when the tag exists, while a dispatch re-run refreshes it and runs the paired asset delete with it, registries dedupe server-side under `dotnet nuget push --skip-duplicate` and PyPI's `skip-existing: true`, and Docker always re-pushes by design. -- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch rather than a cleanup, and it repairs that version only while the branch tip has not moved, since NBGV derives the version from git height, so a re-dispatch after the tip has moved publishes a further version and leaves the first release without its package. +- **D4.5** A failed build blocks every publish target: `github-release` needs every build and the terminal registry pusher (Docker) needs every other build, both guarding `!failure() && !cancelled()` so a disabled or unchanged target, skipped rather than failed, still lets the release be cut and the image pushed, and a package target's separate `publish-` job `needs:` the release-task call, so no build failure ships anything partial. A failed **package** push is outside that: the `publish-` job runs after the whole release task and so after `github-release`, and can leave a release and tag for a version the registry never received. The recovery is a re-dispatch while the tip has not moved, since a dispatch names a branch rather than a commit and so builds that branch's tip at dispatch time. Once the tip has moved a re-dispatch builds the new tip instead, and NBGV deriving the version from git height makes that a further version, so the version whose push failed never reaches the registry. **Re-run all jobs** is the recovery there: GitHub replays under the original event's `GITHUB_SHA` and re-executes every job, and the publisher pins the build to that commit, so the same version is rebuilt, its package artifact rebuilt and re-uploaded rather than left missing by D5.2's delete, and its push retried, the release itself needing nothing from the re-run. Three bounds. D4.4's no-op re-run assumes the earlier push succeeded, so it does not describe this one. GitHub offers a re-run only within 30 days of the initial run. And **Re-run failed jobs** is unreliable rather than unavailable, D5.2's delete usually having taken the artifact its download needs while D5.3 leaves that delete best-effort. - **D4.6** A deploy check asserts which release and which environment answer, waiting for convergence to a bounded timeout, with an unreachable host reported distinctly from an HTTP status. ## D5: Resource Cleanup diff --git a/.github/workflows/validate-task.yml b/.github/workflows/validate-task.yml index afec365d..b2aae10e 100644 --- a/.github/workflows/validate-task.yml +++ b/.github/workflows/validate-task.yml @@ -308,9 +308,19 @@ jobs: if: hashFiles('pyproject.toml') != '' && hashFiles('tests/**') != '' && hashFiles('uv.lock') != '' run: uv sync --all-groups --frozen + # --cov-report=xml names the report format and selects nothing to measure, so the repository's own pyproject.toml supplies the --cov selector, per D1.6. + # The report is checked rather than assumed, because the best-effort upload below reads a missing file exactly as it reads a healthy run. + # The pre-run delete makes that a check on what this run wrote, since a committed coverage.xml would otherwise satisfy it without any measurement. - name: Run pytest step if: hashFiles('pyproject.toml') != '' && hashFiles('tests/**') != '' && hashFiles('uv.lock') != '' - run: uv run pytest --cov-report=xml + run: | + set -Eeuo pipefail + rm -f coverage.xml + uv run pytest --cov-report=xml + if [[ ! -s coverage.xml ]]; then + echo "::error::This run wrote no coverage.xml at the repository root. Select a coverage source in this repository's pyproject.toml, an addopts entry of --cov= in practice, since --cov-report=xml alone measures nothing, and leave the report at the root path the upload step below reads." + exit 1 + fi # Best-effort: continue-on-error plus fail_ci_if_error false, so a missing token never reds the gate. - name: Upload coverage to Codecov step (Python) diff --git a/WORKFLOW.md b/WORKFLOW.md index 3a58b3d7..8e7dbc2f 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -154,7 +154,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as the o - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated on smoke being false, written `!inputs.smoke` at the workflow layer and `inputs.smoke != 'true'` in a composite action, whose inputs are strings). *Prevents: a PR publishing, and orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter marks no target, so smoke-build skips. An inclusion list satisfying D1.1 reaches this by leaving workflow paths out of every target's entry. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, run under `if: always()` so a failed or skipped dependency cannot skip the gate itself, `needs:` the validation job, and the `changes` and `smoke-build` jobs too wherever the repo has a smoke build, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/` and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --coverage --coverage-output-format cobertura --results-directory ./coverage` or `pytest --cov-report=xml` over a repo whose own pytest configuration selects what to measure) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate). `CODECOV_TOKEN` lives in the repo's **actions** and **dependabot** secret stores, the second because a run triggered by a Dependabot PR reads the Dependabot store and the upload would otherwise skip silently on every bot PR, and a caller passing it names it (`secrets:` with `CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}`), the way every hub task's declared secrets are passed, since `secrets: inherit` is documented for a caller in the same organization or enterprise and this fleet is a personal account, so a cross-repository call names each secret it passes. A call by local path stays inside one repository and may inherit instead. The publisher stub's validation job names the secret and every pull request stub's validation job passes no `secrets:` key at all, so a repo whose coverage must reach Codecov from its pull requests adds the mapping there itself. Required for **every** C# and Python repo that has tests. That C# invocation runs under **Microsoft.Testing.Platform**, which an MTP-based test project on the .NET 10 SDK and later requires, since running one through the VSTest target fails outright. A repo whose test project is MTP-based, in practice any repo on xunit.v3 4.0.0 or later, therefore also ships a root **`global.json`** declaring `{"test": {"runner": "Microsoft.Testing.Platform"}}`, references **`Microsoft.Testing.Extensions.CodeCoverage`** at **18.9.0 or later** in place of `coverlet.collector`, whose VSTest data collector MTP ignores without failing, and drops `xunit.runner.visualstudio`, the VSTest adapter MTP replaces. A repo whose test project is not yet MTP-based keeps the VSTest collector and its existing pin on the reusable validator, and that lagging state is a migration still owed rather than drift, until its own bump makes the project MTP-based and forces the move. The version floor is load-bearing rather than cautionary. Below 18.1.0 the extension is built against Microsoft.Testing.Platform 1.x, and an 18.0.x resolution, which is what a `>= 18.0.0` range picks, throws a `TypeLoadException` against the 2.x platform xunit.v3 4.0.0 carries, runs zero tests, and still writes a well-formed Cobertura file reporting full coverage, so only the non-zero exit says the run reported nothing. 18.9.0 is the first release on Microsoft.Testing.Platform 2.3.x, where every test project writes into the one shared `--results-directory` the invocation names rather than resolving that relative path per project. Two details of the invocation are equally load-bearing, and neither failure reds the job on its own. `--coverage-output` stays unset, because pinning one filename gives every test project in the solution the same path and a repo with more than one then keeps only whichever ran last. Leaving it unset produces the default name `.cobertura.xml`, which `codecov-cli`'s own file finder does not match. The validator therefore prefixes each report to `coverage-.cobertura.xml` before the upload step reads the directory. The Python invocation carries a load-bearing detail of its own, an omission rather than a collision: it names the report format and selects nothing to measure. `pytest-cov` reports on what `--cov` selects, so `--cov-report=xml` on its own measures nothing, writes no file, and exits zero, which the best-effort upload then reads exactly as it reads a healthy run. A Python repo with tests therefore references **`pytest-cov`** in its dev dependency group, selects the coverage source in its own `pyproject.toml`, an `addopts` entry of `--cov=` in practice, and leaves the report at the repo root as `coverage.xml`, already the one path the upload step names. The validator **fails the test step when that file was not written**, since nothing downstream of it can tell an absent report from an uploaded one, so a repo that redirects the report through `[tool.coverage.xml]` reds the gate rather than uploading nothing from a green run. That step sits in the hub validator's Python leg, which runs where the repository root carries `pyproject.toml`, `tests/`, and `uv.lock`. Where this guarantee does not apply (a `lint-only` profile for that type, per the hub's `registry/repos.json`), the hub's `spec/secrets.json` `typeMechanisms` mapping is not claimed for that repo, and the absence is not drift. The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR (a distinct knob from `fail_ci_if_error`, which only guards the upload step), and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`. A repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact, so `.gitignore` excludes it (e.g. `coverage/`, `coverage.xml`, and `*.cobertura.xml`, with `.gitignore` the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported, a test project stranded on a runner the current SDK refuses, a stale and unused token, a coverage regression blocking an unrelated PR, and a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry @@ -177,13 +177,13 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as the o - **D4.2 Tag the built commit.** Output: the release `target_commitish` is the built commit's SHA (NBGV's `GitCommitId`), never a branch name or a separately re-resolved ref. *Prevents: the tag landing on the default branch instead of the built tree.* - **D4.3 Release contents.** Output: every release contains a tag on the built commit plus the auto source zip, README, and LICENSE. File targets attach `release-asset-*`. The `prerelease` value equals `branch != default`. A no-file-target caller sets `expect_release_assets: false` to reach the no-asset shape. This applies to Docker-only, PyPI-only, and source-only repos. A NuGet target is not among them, since its leaf uploads a `release-asset-*` carrying the package, so a NuGet-only caller keeps the default `true`. The setting relaxes `fail_on_unmatched_files` and skips the asset download. The release-create step fails when no assets exist and the setting retains its default `true`. A source-only caller also sets every `enable_*` input false. - **D4.4 No-op republish.** Input: a re-run whose version is unchanged, on a schedule or push trigger. Output: nothing is re-pushed, because the release-create step is skipped when the tag exists, and the paired asset-delete is skipped with it. A **dispatch** re-run refreshes the release instead and runs that delete with it, which is why a dispatch-only publisher records this item's skip leg as unreachable rather than failed. Registry pushes are no-ops. The NuGet/PyPI publish steps are **not** statically gated on existence. They run and the **server** dedupes (`dotnet nuget push --skip-duplicate` turns a 409 into success, and PyPI does the same under `skip-existing: true`). **Docker always re-pushes** the image (base-image refresh), independently of the release-create skip, within the same run. *Prevents: duplicate releases and wasted pushes.* -- **D4.5 A build failure blocks every publish target.** Input: a real publish where one enabled build fails. Output: nothing publishes. `github-release` needs every build and carries the same `!failure() && !cancelled()` guard the terminal registry pusher (Docker) does, since the implicit `success()` would otherwise skip both on every run that disables a target rather than only on a failed one. A failed build therefore skips the release (no tag, no release), and Docker, which needs every other build, skips with it (no image push) while a disabled or unchanged target (skipped, not failed) still lets docker push. *Prevents: a partial publish, e.g. a Docker image pushed while .NET publish failed and no release was cut.* A package target's separate publish job needs its own gate for the same reason, since it sits outside the `github-release` and Docker `needs:` chains: it `needs:` the release-task call, so a failed build skips it with the rest. What no gate covers is a failed **push**, because that job runs after the whole release task and therefore after `github-release`, for the trusted-publishing reason section 3's package-registry bullet gives. A rejected token exchange, a registry outage, or a trusted-publishing policy naming the wrong workflow file therefore leaves a published release and tag for a version that never reached the registry. The recovery is a re-dispatch rather than a cleanup. A dispatch names a branch, `main` or `develop` per D2.3, and never a commit, so what it builds is that branch's tip at dispatch time. A re-dispatch therefore refreshes the failed version's release (D4.4) and runs its push again only while the tip is still the commit whose push failed. Once the tip has moved a re-dispatch builds the new tip, since NBGV derives the version from git height, so it publishes a further version and leaves the first release without its package. +- **D4.5 A build failure blocks every publish target.** Input: a real publish where one enabled build fails. Output: nothing publishes. `github-release` needs every build and carries the same `!failure() && !cancelled()` guard the terminal registry pusher (Docker) does, since the implicit `success()` would otherwise skip both on every run that disables a target rather than only on a failed one. A failed build therefore skips the release (no tag, no release), and Docker, which needs every other build, skips with it (no image push) while a disabled or unchanged target (skipped, not failed) still lets docker push. *Prevents: a partial publish, e.g. a Docker image pushed while .NET publish failed and no release was cut.* A package target's separate publish job needs its own gate for the same reason, since it sits outside the `github-release` and Docker `needs:` chains: it `needs:` the release-task call, so a failed build skips it with the rest. What no gate covers is a failed **push**, because that job runs after the whole release task and therefore after `github-release`, for the trusted-publishing reason section 3's package-registry bullet gives. A rejected token exchange, a registry outage, or a trusted-publishing policy naming the wrong workflow file therefore leaves a published release and tag for a version that never reached the registry. The recovery is a re-dispatch or a re-run rather than a cleanup, and which of the two applies turns on whether the branch tip has moved. A dispatch names a branch, `main` or `develop` per D2.3, and never a commit, so what it builds is that branch's tip at dispatch time. A re-dispatch therefore refreshes the failed version's release (D4.4) and runs its push again while the tip is still the commit whose push failed. Once the tip has moved a re-dispatch builds the new tip instead. NBGV derives the version from git height, so that is a further version, and the version whose push failed never reaches the registry. **Re-run all jobs** (`gh run rerun `) is the recovery there. GitHub replays a run under the original event's `GITHUB_SHA` and `GITHUB_REF` and re-executes every job rather than only the failed ones, the publisher pins the release task to that commit with `ref: ${{ github.sha }}`, so `get-version` recomputes the version from the same commit and history and each build leaf checks out the `GitCommitId` `get-version` emits, the package artifact D5.2 deleted is rebuilt and re-uploaded rather than missing when `publish-` downloads it, and that job retries the push it failed. The release needs nothing from that re-run, the failed run having already cut it, so whether D4.4's release-create step refreshes or skips does not bear on the recovery. What no route settles in advance is whether the registry accepts the retried push. Three qualifications come with **Re-run all jobs**. D4.4 and 5B's S9 describe a re-run whose predecessor push **succeeded**, where the registry dedupes the second one, so this recovery is the case they do not cover and its retried push is the first the registry ever receives for that version. GitHub offers a re-run only within **30 days** of the initial run, past which a moved tip leaves that version with no route at all. And **Re-run failed jobs** (`--failed`) is unreliable here rather than unavailable. D5.2's delete runs on the path that reaches this case, its gate being `!cancelled()` and the download having succeeded, and it removes the package artifact a `--failed` re-run would download. D5.3 leaves that delete best-effort, so the artifact survives where that delete ran and failed, and `--failed` works in that case alone. - **D4.6 Deploy verification names the release.** Input: a deploy to a filesystem on a host the project owns that completes without error. Output: a check against the running host asserts **which release is answering**, not merely that it answers. The artifact stamps its own version into the configuration it ships, and the check compares that against the version just installed, **waiting for convergence to a bounded timeout** rather than sampling once, because content goes live the instant a pointer moves while server rules wait on an asynchronous reload. The same check asserts **which environment** answered, since several environments serve a byte-identical artifact and a proxy rule aimed at the wrong one answers healthily under the right hostname. An unreachable host is reported distinctly from an HTTP status. *Prevents: a green deploy over a host still serving the previous release's configuration, a URL contract checked against the wrong environment, and a dead config watcher read as a routing fault.* ### D5 - Resource Cleanup - **D5.1 Delete at the point of consumption.** Output: the job that downloads a **cross-job** transfer artifact deletes it (by exact name/pattern) right after consuming it. An intermediate consumed only within the same run MAY instead rely on the `retention-days: 1` backstop. *Prevents: transfer artifacts accumulating against the storage quota.* -- **D5.2 Gate the delete to the condition that made the artifact redundant.** Output: the delete runs exactly when the consumption it follows happened. Where the consumer is a conditional step (the GitHub release create), the delete carries that same condition. Where the consumer is a step that always attempts once its job runs (a package publish job's push), the delete is gated on the **download** having succeeded rather than on the push, as `if: ${{ !cancelled() && steps..outcome == 'success' }}`. A step whose `if:` carries no status-check function, an absent `if:` included, inherits `success()` instead, which skips it on exactly the failed push where the artifact is already downloaded and the release is already cut. So on a no-op re-run that is not a dispatch the `release-asset-*` delete is **skipped** with the release create it follows, while the `nuget-build-*` and `pypi-build-*` deletes still **run**. A dispatch re-run refreshes the release instead (D4.4), so its asset delete runs with it. Deleting the `nuget-build-*` or `pypi-build-*` artifact on the failed-push path costs the run its **Re-run failed jobs** route, since the re-run's download finds nothing, so the recovery for a failed push is the re-dispatch D4.5 names rather than a re-run. *Prevents: deleting freshly built assets on a no-op re-run, and stranding a downloaded artifact when the push it fed fails.* +- **D5.2 Gate the delete to the condition that made the artifact redundant.** Output: the delete runs exactly when the consumption it follows happened. Where the consumer is a conditional step (the GitHub release create), the delete carries that same condition. Where the consumer is a step that always attempts once its job runs (a package publish job's push), the delete is gated on the **download** having succeeded rather than on the push, as `if: ${{ !cancelled() && steps..outcome == 'success' }}`. A step whose `if:` carries no status-check function, an absent `if:` included, inherits `success()` instead, which skips it on exactly the failed push where the artifact is already downloaded and the release is already cut. So on a no-op re-run that is not a dispatch the `release-asset-*` delete is **skipped** with the release create it follows, while the `nuget-build-*` and `pypi-build-*` deletes still **run**. A dispatch re-run refreshes the release instead (D4.4), so its asset delete runs with it. Deleting the `nuget-build-*` or `pypi-build-*` artifact on the failed-push path costs the run its **Re-run failed jobs** route, since the re-run's download then finds nothing, so the recovery for a failed push is the re-dispatch or the full re-run D4.5 names, and D4.5 sets out how far that cost actually reaches. *Prevents: deleting freshly built assets on a no-op re-run, and stranding a downloaded artifact when the push it fed fails.* - **D5.3 Best-effort.** Output: cleanup is `continue-on-error`, tolerates a failed listing, and deletes **all** matching ids. *Prevents: a cleanup hiccup reddening a job whose publish succeeded.* - **D5.4 Retention backstop.** Output: **every** `upload-artifact` sets `retention-days: 1`. - **D5.5 Never blanket-delete.** Output: cleanup MUST NOT enumerate and delete the run's whole artifact set. *Prevents: destroying diagnostic/log artifacts and auto-emitted build-records.* diff --git a/docs/reusable-workflows.md b/docs/reusable-workflows.md index 7b50061b..942bd097 100644 --- a/docs/reusable-workflows.md +++ b/docs/reusable-workflows.md @@ -156,7 +156,7 @@ Hub: `validate-task.yml` hosts a `lint` job (the fleet doc-lint block, language - [x] Hook override path observed on a hub pull request run, [proof run][override-path-run] (runs `./.github/actions/validate`, no hub checkout). Default path observed on PhotoCleaner's adoption pull request, [pilot smoke run][pilot-smoke-run], where the hub's `validate-default` ran because that repo carries no `validate` hook. The follow-up self-reference pilot also runs the bundled prose and repository gates through `$/.github/actions/` without checking out the hub. - [x] PhotoCleaner (pilot, release trigger shape with smoke, the same repo that piloted stage 1): ptr727/PhotoCleaner#55 on `develop` (`c80cb29`), promoted in ptr727/PhotoCleaner#56 (`fa91db0`), both on 2026-08-16. `test-pull-request.yml` calls the hub validate task and no repo hook was needed. - [ ] HomeAutomation-Config (second pilot, operational trigger shape) -- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. +- [ ] The remaining repos, one checkbox each added when the pilots close, since the sweep list is every cataloged repo. A Python adopter owes one precondition before its bump: the `unit-test` job fails when the run wrote no root `coverage.xml`, and that job's Python leg runs where the repository root carries `pyproject.toml`, `tests/`, and `uv.lock`, so an adopter of that shape puts `pytest-cov` in a dev group and a `--cov=` selector in its own `pyproject.toml` before it bumps, per D1.6, which binds that selector for every Python repo with tests, lint-only excepted. aiopurpleair and Financial-Modeling carry both, homeassistant-purpleair and PlexCleaner carry no `uv.lock` so the step never runs there, and ESPHome-Config's Python is lint-only. The hub cannot smoke-test this itself, having no `tests/` and no `uv.lock` of its own. - [ ] `reports/workflow-reuse.md` regenerated with `validate-task.yml` at 0 copies (a hub-only file no repo carries) and `test-pull-request.yml` showing callers equal to copies. ### Stage 3: The Pure Functions diff --git a/reports/canonical-review.json b/reports/canonical-review.json index 5c5fc848..4eed92ee 100644 --- a/reports/canonical-review.json +++ b/reports/canonical-review.json @@ -283,19 +283,19 @@ }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D1: PR Fast-Feedback (Smoke)", - "digest": "sha256:58e3860e81d92ae6d4ec4a84d04c6770dccad4caa8fdb3c12132f5aa29ffaaa8", + "digest": "sha256:6c57b1c6eed34177bc1f1dacef05725efc19e0b385b103a6d0a01c9265b41e93", "reviewer": "agent-skill", - "findings": 62, - "hubCommit": "9b33edb7e0f55fe2a05630a49d5be99ba1e3eb26", - "stamp": "2026-09-02T15:16:10Z" + "findings": 0, + "hubCommit": "4fdc718663d0149994e496e890c20c7a77f27c96", + "stamp": "2026-09-03T03:12:20Z" }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D4: Release and Publish", - "digest": "sha256:e3fbb2b2de51a2679272e1ff1637681886241da62113ea9020550a3823ef9b43", + "digest": "sha256:2c8b0653258589c31a3f84491da60752deb95131bb5d2a2154a8465bacab69bf", "reviewer": "agent-skill", - "findings": 2, - "hubCommit": "c6e7526cfade19f28f0ab09c4ab283ed00295d7e", - "stamp": "2026-09-02T20:49:13Z" + "findings": 0, + "hubCommit": "16daf0dc20e9f11a1485b6835b4f064d6907b273", + "stamp": "2026-09-03T01:05:09Z" }, { "unit": ".agents/skills/workflow-ci-contract/references/d-guarantees.md > D5: Resource Cleanup", @@ -451,11 +451,11 @@ }, { "unit": "WORKFLOW.md > 4. Behavioral Contract: Expected Outcomes", - "digest": "sha256:587d95361c4bd424a65327d3394d266342f13fe05f598b8521bdb7880d781f57", + "digest": "sha256:a4b85d3b43f98646b6cf1a7161acacc09f3e4b8f3c23c1197af69d13bba74d84", "reviewer": "agent-skill", - "findings": 5, - "hubCommit": "7ab81feb4467dd56c0406c1fa98d4f2c94d2ef13", - "stamp": "2026-09-02T21:07:56Z" + "findings": 0, + "hubCommit": "4fdc718663d0149994e496e890c20c7a77f27c96", + "stamp": "2026-09-03T03:04:24Z" }, { "unit": "WORKFLOW.md > 5. Test Methodology", diff --git a/spec/project-types.json b/spec/project-types.json index 8c510cc3..f329ac22 100644 --- a/spec/project-types.json +++ b/spec/project-types.json @@ -42,7 +42,7 @@ { "id": "python.pyright.config", "verdict": "intent", "assert": "Build profile: pyright is configured and runs strict on first-party code (src or the integration package) - the strong typing baseline. Third-party strictness is relaxed only where a dependency has no usable types. N/A for the lint-only profile, whose type checker is mypy over stdlib-only code (python.mypy.allowed).", "intentRef": "CODESTYLE.md", "minProfile": "build" }, { "id": "python.config.placement", "verdict": "letter", "assert": "ruff and the type-checker config live in pyproject.toml (canonical); standalone .ruff.toml / pyrightconfig.json is a drift finding. A Home Assistant integration is the exception - it follows home-assistant/core standalone-config conventions and is scored by ha.python.conventions instead.", "intentRef": "CODESTYLE.md" }, { "id": "python.mypy.allowed", "verdict": "intent", "assert": "mypy is permitted as an additional type checker, not banned. It is required for a Home Assistant integration (platinum strict-typing) and is the lint-only profile's type checker. When used it runs in CI and the editor.", "intentRef": "CODESTYLE.md" }, - { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, + { "id": "python.coverage.codecov", "verdict": "letter", "assert": "The test job collects coverage (pytest --cov-report=xml) and uploads it to Codecov via codecov/codecov-action, best-effort (continue-on-error and fail_ci_if_error: false). Because --cov-report=xml alone measures nothing and writes no file, the repo declares pytest-cov in a dev dependency group and selects the coverage source in its own pyproject.toml, an addopts --cov= entry in practice, leaving the report at the repo root as coverage.xml. The validator's Python leg, which runs where the repository root carries pyproject.toml, tests/ and uv.lock, deletes any coverage.xml before the run and fails the test step when that file was not written, so a repo of that shape missing the selector reds its gate there, while one missing pytest-cov reds it earlier, at the pytest invocation that does not recognize the flag. CODECOV_TOKEN is stored in both the repo actions and dependabot secrets stores, the second so the upload does not skip on a Dependabot PR, and the caller maps it to the reusable validator by name. Required for every Python repo with tests. N/A for the lint-only profile (its unittest suite runs under coverage in CI, reported without a threshold and never uploaded to Codecov). In a mixed repo the codecov.yml file-presence is still required by any co-present type that has tests, e.g. csharp.", "intentRef": "WORKFLOW.md", "minProfile": "build" }, { "id": "python.uvlock.pinned", "verdict": "letter", "assert": "Build profile: the committed uv.lock resolves to LF through the repository-wide .editorconfig and .gitattributes defaults. A CRLF-native operational repo adds a narrow uv.lock LF override only if it adopts the uv build profile. N/A for a non-uv Python repo (a Home Assistant integration on pip/requirements) and for the lint-only profile (no uv.lock by definition).", "intentRef": "GOVERNANCE.md#line-endings", "minProfile": "build" }, { "id": "python.scripts.uvx", "verdict": "letter", "assert": "Lint-only profile only: the tools run via uvx (no project install, no lockfile). A uvx @ pin in a run: step is not Dependabot-trackable, so CI runs uvx ruff@latest / uvx mypy@latest - the fleet rule pins only what Dependabot auto-updates and otherwise runs latest, never a manual pin that goes stale. VS Code tasks, README, and CI all run the unpinned latest. N/A for the build profile (which pins tool versions via uv.lock + uv sync --frozen instead).", "intentRef": "CODESTYLE.md" } ]