From 379db62f7a9cc33f118258216407dd0a83e38a86 Mon Sep 17 00:00:00 2001 From: Don Petry Date: Thu, 2 Jul 2026 09:49:32 -0500 Subject: [PATCH 1/4] =?UTF-8?q?docs(github-settings):=20reconcile=20=C2=A7?= =?UTF-8?q?243=20required-checks=20table=20to=20codified=20code-quality=20?= =?UTF-8?q?(#575)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The required-checks table claimed Coverage + Secret Scan + Dev-Lead Agent as required "All repos" while omitting agent-shield + dependency-audit — out of sync with the codified standards/rulesets/code-quality.json (4 contexts). Reconcile to the codified set and document the safe sequencing. - Point the section at standards/rulesets/code-quality.json as the source of truth. - Required (codified): SonarCloud, CodeQL, agent-shield/AgentShield, dependency-audit/Detect ecosystems (note: only the unconditional Detect job). - Reclassify: Secret Scan + Coverage → template/new repos, NOT yet fleet-wide (added to the ruleset only after a coverage backfill — requiring them now bricks repos lacking the jobs); Dev-Lead Agent → not a required context (per-PR review; requiring it deadlocks workflow-touching PRs); CI Pipeline → repo-specific. - Add a "Sequencing" note explaining why Secret Scan + Coverage are staged. Docs-only; markdownlint clean. Depends on #577 (adds standards/rulesets/). Part of #575 (folded-in from closed #1001). Co-Authored-By: Claude Opus 4.8 (1M context) --- standards/github-settings.md | 41 ++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/standards/github-settings.md b/standards/github-settings.md index 6742cfe72..018def298 100644 --- a/standards/github-settings.md +++ b/standards/github-settings.md @@ -252,20 +252,39 @@ a fresh approval under the current CODEOWNERS. ### `code-quality` — Required Checks Ruleset (All Repositories) -Every repository MUST have the following quality checks configured and -required. The specific check names and ecosystem configurations vary by repo, -but the categories are universal. +The **codified source of truth** for the required-check set is +[`standards/rulesets/code-quality.json`](rulesets/code-quality.json) — the table +below reconciles to it. `apply-rulesets.sh` applies that file; do not add a context +here that the file does not carry (or vice versa) without updating both. #### Required Check Categories -| Check | Required | Check Name(s) | Notes | -|-------|----------|---------------|-------| -| **SonarCloud** | All repos | `SonarCloud` | Code quality, maintainability, security hotspots | -| **CodeQL** | All repos | `CodeQL` | SAST via GitHub-managed default setup — auto-detects all supported languages (see [ci-standards.md §2](ci-standards.md#2-codeql-analysis-github-managed-default-setup)) | -| **Dev-Lead Agent** | All repos | `Dev-Lead Agent / dev-lead` | AI code review and agent automation on every PR | -| **CI Pipeline** | All repos | Repo-specific (e.g., `build-and-test`, `TypeScript`, `Go`) | Lint, format, typecheck, test | -| **Coverage** | All repos | `coverage` or embedded in CI job | Must meet repo-defined thresholds | -| **Secret Scan** | All repos | `Secret scan (gitleaks)` | Full-history gitleaks scan — see [Push Protection Standard](push-protection.md#layer-3--ci-secret-scanning-secondary-defense) | +The **four contexts codified in `code-quality.json`** are required on every repo's +default branch: + +| Check | Status | Check Name(s) | Notes | +|-------|--------|---------------|-------| +| **SonarCloud** | ✅ Required (codified) | `SonarCloud` | Code quality, maintainability, security hotspots | +| **CodeQL** | ✅ Required (codified) | `CodeQL` | SAST via GitHub-managed default setup — auto-detects all supported languages (see [ci-standards.md §2](ci-standards.md#2-codeql-analysis-github-managed-default-setup)) | +| **AgentShield** | ✅ Required (codified) | `agent-shield / AgentShield` | Deep agent-config security scan on every PR | +| **Dependency Audit** | ✅ Required (codified) | `dependency-audit / Detect ecosystems` | Only the unconditional `Detect ecosystems` job is required; per-ecosystem audit jobs are gated on lockfile presence and would fail as required-but-skipped | + +The following are **intentionally NOT** in `code-quality.json` today: + +| Check | Status | Check Name(s) | Why | +|-------|--------|---------------|-----| +| **Secret Scan** | ⏳ Template / new repos; not yet fleet-wide | `Secret scan (gitleaks)` | Produced by the template [`ci.yml`](workflows/ci.yml) (see [Push Protection Standard](push-protection.md#layer-3--ci-secret-scanning-secondary-defense)). Added to the ruleset fleet-wide only once existing repos produce it | +| **Coverage** | ⏳ Template / new repos; not yet fleet-wide | `coverage` | Produced by the template [`ci.yml`](workflows/ci.yml) (green-until-tests). Requiring it fleet-wide before repos have a coverage job would brick every PR — backfill first, then add the context | +| **Dev-Lead Agent** | ❌ Not a required context | `Dev-Lead Agent / dev-lead` | Per-PR AI review, not a merge gate: the agent's GitHub App refuses to mint a token for any PR touching workflow files, so requiring it would deadlock every workflow-modifying PR | +| **CI Pipeline** | Repo-specific (not a fixed org context) | e.g. `build-and-test`, `TypeScript`, `Go` | Lint/format/typecheck/test; the job name is repo-defined, so it is required per-repo (via branch protection), not as a codified org-wide context | + +**Sequencing (why Secret Scan + Coverage are staged).** Adding a required context +to `code-quality.json` makes it a merge gate on **every** repo the ruleset targets; +a repo that does not *produce* that check is bricked. The template `ci.yml` now +ships both jobs so new repos produce them, but existing fleet repos do not yet have +a coverage job — so `coverage` (and `Secret scan (gitleaks)`) are added to the shared +ruleset only **after** a fleet backfill. Until then, keep the stricter set scoped to +the template / new repos. See [petry-projects/.github#575](https://github.com/petry-projects/.github/issues/575). > **Check names must match exactly.** GitHub-managed CodeQL produces a check named > `CodeQL` — **not** `Analyze (actions)`, `Analyze (javascript-typescript)`, or From 89b53a56cea1913983397b54fdec54b614d4ee52 Mon Sep 17 00:00:00 2001 From: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com> Date: Thu, 2 Jul 2026 14:57:16 +0000 Subject: [PATCH 2/4] fix(reviews): address review comments [skip ci-relay] --- standards/github-settings.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/standards/github-settings.md b/standards/github-settings.md index 018def298..020342c79 100644 --- a/standards/github-settings.md +++ b/standards/github-settings.md @@ -252,19 +252,21 @@ a fresh approval under the current CODEOWNERS. ### `code-quality` — Required Checks Ruleset (All Repositories) -The **codified source of truth** for the required-check set is -[`standards/rulesets/code-quality.json`](rulesets/code-quality.json) — the table -below reconciles to it. `apply-rulesets.sh` applies that file; do not add a context -here that the file does not carry (or vice versa) without updating both. +`apply-rulesets.sh` **dynamically constructs** the `code-quality` ruleset for each +repo by probing its workflow files via `detect_required_checks()` — it does not apply +a static JSON file. The table below is the canonical reference for the required-check +set; keep it in sync with the detection logic in `scripts/apply-rulesets.sh`. Note: +`apply-rulesets.sh` manages only the `pr-quality` and `code-quality` branch rulesets — +the `release-channel-tags` ruleset is managed in `.github-private` (see +[§Source of truth & repo boundary](#source-of-truth--repo-boundary)). #### Required Check Categories -The **four contexts codified in `code-quality.json`** are required on every repo's -default branch: +The **four required check contexts** applied to every repo's default branch are: | Check | Status | Check Name(s) | Notes | |-------|--------|---------------|-------| -| **SonarCloud** | ✅ Required (codified) | `SonarCloud` | Code quality, maintainability, security hotspots | +| **SonarCloud** | ✅ Required (codified) | `SonarCloud Analysis / SonarCloud` | Code quality, maintainability, security hotspots. `apply-rulesets.sh` derives the check name as ` / SonarCloud`; the standard template sets `name: SonarCloud Analysis`, producing this context. Verify with `gh pr checks ` if the workflow name differs | | **CodeQL** | ✅ Required (codified) | `CodeQL` | SAST via GitHub-managed default setup — auto-detects all supported languages (see [ci-standards.md §2](ci-standards.md#2-codeql-analysis-github-managed-default-setup)) | | **AgentShield** | ✅ Required (codified) | `agent-shield / AgentShield` | Deep agent-config security scan on every PR | | **Dependency Audit** | ✅ Required (codified) | `dependency-audit / Detect ecosystems` | Only the unconditional `Detect ecosystems` job is required; per-ecosystem audit jobs are gated on lockfile presence and would fail as required-but-skipped | @@ -274,7 +276,7 @@ The following are **intentionally NOT** in `code-quality.json` today: | Check | Status | Check Name(s) | Why | |-------|--------|---------------|-----| | **Secret Scan** | ⏳ Template / new repos; not yet fleet-wide | `Secret scan (gitleaks)` | Produced by the template [`ci.yml`](workflows/ci.yml) (see [Push Protection Standard](push-protection.md#layer-3--ci-secret-scanning-secondary-defense)). Added to the ruleset fleet-wide only once existing repos produce it | -| **Coverage** | ⏳ Template / new repos; not yet fleet-wide | `coverage` | Produced by the template [`ci.yml`](workflows/ci.yml) (green-until-tests). Requiring it fleet-wide before repos have a coverage job would brick every PR — backfill first, then add the context | +| **Coverage** | ⏳ Template / new repos; not yet fleet-wide | `coverage` | **Not produced by the template [`ci.yml`](workflows/ci.yml) out of the box** — the template ships a single `build-and-test` job; a separate `coverage` check must be explicitly configured when adopting the template (see [ci-standards.md](ci-standards.md)). Requiring it fleet-wide before repos produce it would brick every PR — backfill first, then add the context | | **Dev-Lead Agent** | ❌ Not a required context | `Dev-Lead Agent / dev-lead` | Per-PR AI review, not a merge gate: the agent's GitHub App refuses to mint a token for any PR touching workflow files, so requiring it would deadlock every workflow-modifying PR | | **CI Pipeline** | Repo-specific (not a fixed org context) | e.g. `build-and-test`, `TypeScript`, `Go` | Lint/format/typecheck/test; the job name is repo-defined, so it is required per-repo (via branch protection), not as a codified org-wide context | From 9aa42228e69d35b64d05d7d746af49461eb26691 Mon Sep 17 00:00:00 2001 From: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com> Date: Thu, 2 Jul 2026 15:09:16 +0000 Subject: [PATCH 3/4] fix(reviews): address review comments [skip ci-relay] --- standards/github-settings.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/standards/github-settings.md b/standards/github-settings.md index 020342c79..fe1be830e 100644 --- a/standards/github-settings.md +++ b/standards/github-settings.md @@ -262,7 +262,7 @@ the `release-channel-tags` ruleset is managed in `.github-private` (see #### Required Check Categories -The **four required check contexts** applied to every repo's default branch are: +The table below covers the **unconditional fleet-wide required check contexts** — the base set `detect_required_checks()` applies when each org-standard workflow file is present. The script also conditionally adds **Dev-Lead Agent** (when `dev-lead.yml` exists) and **Secret scan** (when `ci.yml` contains the gitleaks action), but those are in staged rollout; see below. | Check | Status | Check Name(s) | Notes | |-------|--------|---------------|-------| @@ -282,11 +282,11 @@ The following are **intentionally NOT** in `code-quality.json` today: **Sequencing (why Secret Scan + Coverage are staged).** Adding a required context to `code-quality.json` makes it a merge gate on **every** repo the ruleset targets; -a repo that does not *produce* that check is bricked. The template `ci.yml` now -ships both jobs so new repos produce them, but existing fleet repos do not yet have -a coverage job — so `coverage` (and `Secret scan (gitleaks)`) are added to the shared -ruleset only **after** a fleet backfill. Until then, keep the stricter set scoped to -the template / new repos. See [petry-projects/.github#575](https://github.com/petry-projects/.github/issues/575). +a repo that does not *produce* that check is bricked. Neither context is produced by +the template `ci.yml` out of the box — repos must explicitly add a coverage job and +the gitleaks secret-scan step before those check names appear. Both are added to the +shared ruleset only **after** a fleet backfill. Until then, keep them scoped to repos +that already produce them. See [petry-projects/.github#575](https://github.com/petry-projects/.github/issues/575). > **Check names must match exactly.** GitHub-managed CodeQL produces a check named > `CodeQL` — **not** `Analyze (actions)`, `Analyze (javascript-typescript)`, or From 5b30e259a9d27efe6d1a57db6cc7c464db21c2c8 Mon Sep 17 00:00:00 2001 From: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com> Date: Thu, 2 Jul 2026 15:12:40 +0000 Subject: [PATCH 4/4] chore: dev-lead update (review-changes) [skip ci-relay] --- standards/github-settings.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/standards/github-settings.md b/standards/github-settings.md index fe1be830e..33f556541 100644 --- a/standards/github-settings.md +++ b/standards/github-settings.md @@ -262,7 +262,8 @@ the `release-channel-tags` ruleset is managed in `.github-private` (see #### Required Check Categories -The table below covers the **unconditional fleet-wide required check contexts** — the base set `detect_required_checks()` applies when each org-standard workflow file is present. The script also conditionally adds **Dev-Lead Agent** (when `dev-lead.yml` exists) and **Secret scan** (when `ci.yml` contains the gitleaks action), but those are in staged rollout; see below. +The table below covers the **unconditional fleet-wide required check contexts** — the base set `detect_required_checks()` applies when each org-standard workflow file is present. +The script also conditionally adds **Dev-Lead Agent** (when `dev-lead.yml` exists) and **Secret scan** (when `ci.yml` contains the gitleaks action), but those are in staged rollout; see below. | Check | Status | Check Name(s) | Notes | |-------|--------|---------------|-------|