Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/health-74-template-drift.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ name: Health 74 Template Drift
on:
push:
paths:
- '.github/workflows/agents-*.yml'
- '.github/workflows/*.yml'
- 'templates/consumer-repo/.github/workflows/*.yml'
- 'scripts/check_template_drift.py'
- 'config/template-drift-allowlist.txt'
pull_request:
paths:
- '.github/workflows/agents-*.yml'
- '.github/workflows/*.yml'
- 'templates/consumer-repo/.github/workflows/*.yml'
- 'scripts/check_template_drift.py'
- 'config/template-drift-allowlist.txt'
Expand Down
55 changes: 35 additions & 20 deletions config/template-drift-allowlist.txt

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/ci/WORKFLOWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ Scheduled health jobs keep the automation ecosystem aligned:
* [`health-71-sync-health-check.yml`](../../.github/workflows/health-71-sync-health-check.yml) monitors sync workflow health daily - creates issues if all recent runs failed or sync is stale (daily schedule, manual dispatch).
* [`health-72-template-sync.yml`](../../.github/workflows/health-72-template-sync.yml) validates that manifest-declared exact template-sync files are in sync with their consumer template copies (PR, push on exact-sync source/template changes).
* [`health-73-template-completeness.yml`](../../.github/workflows/health-73-template-completeness.yml) validates that consumer-intended workflows exist in the template directory and sync manifest - prevents workflows from being added to .github/workflows/ without being synced to consumer repos (PR, push on workflow/template changes).
* [`health-74-template-drift.yml`](../../.github/workflows/health-74-template-drift.yml) checks normalized content drift between main agent workflows and their consumer repo templates. It fails on unallowlisted drift and permits only fingerprinted baseline differences recorded in `config/template-drift-allowlist.txt` (PR, push on workflow/template changes).
* [`health-74-template-drift.yml`](../../.github/workflows/health-74-template-drift.yml) checks normalized content drift between main workflows that have consumer template counterparts and their `templates/consumer-repo/` copies. It fails on unallowlisted drift and permits only fingerprinted baseline differences recorded in `config/template-drift-allowlist.txt` (PR, push on any `.github/workflows/*.yml`, template workflow, checker, or allowlist change).
* [`health-75-api-rate-diagnostic.yml`](../../.github/workflows/health-75-api-rate-diagnostic.yml) monitors API rate limit utilization across PATs and GitHub Apps - alerts when usage exceeds 85% and provides load balancing analysis (scheduled every 4 hours, manual dispatch).
* [`health-76-codex-cli-freshness.yml`](../../.github/workflows/health-76-codex-cli-freshness.yml) emits a weekly machine-readable freshness contract for the verifier `@openai/codex` CLI pin and uploads the deliberate update path as an artifact (scheduled weekly, manual dispatch).
* [`health-78-backplane-contract.yml`](../../.github/workflows/health-78-backplane-contract.yml) Workflows-internal gate that runs on PRs touching the run-contract/v1 contract set (schemas, registry, validator, fixtures): asserts the three schemas load as valid draft 2020-12 JSON Schema, `config/backplane_participants.json` keeps the required shape, and the bundled valid/invalid fixtures behave (the validator self-smoke).
Expand Down
2 changes: 1 addition & 1 deletion docs/ci/WORKFLOW_SYSTEM.md
Original file line number Diff line number Diff line change
Expand Up @@ -739,7 +739,7 @@ Keep this table handy when you are triaging automation: it confirms which workfl
| **Health 71 Sync Health Check** (`health-71-sync-health-check.yml`, maintenance bucket) | `schedule` (daily), `workflow_dispatch` | Monitor sync workflow health and create issues when all recent runs failed or sync is stale. | ⚪ Scheduled/manual | [Sync health check runs](https://github.com/stranske/Workflows/actions/workflows/health-71-sync-health-check.yml) |
| **Health 72 Template Sync** (`health-72-template-sync.yml`, maintenance bucket) | `pull_request`, `push` (exact-sync sources/templates) | Validate that manifest-declared exact template-sync files match their `templates/consumer-repo/` copies. | ⚪ Required on PRs | [Template sync validation runs](https://github.com/stranske/Workflows/actions/workflows/health-72-template-sync.yml) |
| **Health 73 Template Completeness** (`health-73-template-completeness.yml`, maintenance bucket) | `pull_request`, `push` (`.github/workflows/`, `templates/`, manifest) | Validate that consumer-intended workflows exist in template and manifest. Prevents workflows added to .github/workflows/ without being synced to consumer repos. | ⚪ Required on PRs | [Template completeness runs](https://github.com/stranske/Workflows/actions/workflows/health-73-template-completeness.yml) |
| **Health 74 Template Drift** (`health-74-template-drift.yml`, maintenance bucket) | `pull_request`, `push` (agents workflows, templates, drift checker config) | Check normalized content drift between main agent workflows and consumer repo templates. Fails on unallowlisted drift; permits only fingerprinted baseline differences in `config/template-drift-allowlist.txt`. | ⚪ Required on matching PRs | [Template drift runs](https://github.com/stranske/Workflows/actions/workflows/health-74-template-drift.yml) |
| **Health 74 Template Drift** (`health-74-template-drift.yml`, maintenance bucket) | `pull_request`, `push` (any `.github/workflows/*.yml`, templates, drift checker config) | Check normalized content drift between main workflows with consumer template counterparts and their `templates/consumer-repo/` copies. Fails on unallowlisted drift; permits only fingerprinted baseline differences in `config/template-drift-allowlist.txt`. | ⚪ Required on matching PRs | [Template drift runs](https://github.com/stranske/Workflows/actions/workflows/health-74-template-drift.yml) |
| **Health 75 API Rate Diagnostic** (`health-75-api-rate-diagnostic.yml`, maintenance bucket) | `schedule` (every 4 hours), `workflow_dispatch` | Monitor API rate limit utilization across GITHUB_TOKEN, PATs, and GitHub Apps. Alerts when usage exceeds 85%, tracks consumer repo workflow activity, and provides load balancing analysis. | ⚪ Scheduled/manual | [API rate diagnostic runs](https://github.com/stranske/Workflows/actions/workflows/health-75-api-rate-diagnostic.yml) |
| **Health 76 Codex CLI Freshness** (`health-76-codex-cli-freshness.yml`, maintenance bucket) | `schedule` (weekly), `workflow_dispatch` | Compare the verifier `@openai/codex` install pin with the latest npm release and upload a machine-readable update-path contract for deliberate CLI upgrades. | ⚪ Scheduled/manual | [Codex CLI freshness runs](https://github.com/stranske/Workflows/actions/workflows/health-76-codex-cli-freshness.yml) |
| **Backplane Contract Integrity** (`health-78-backplane-contract.yml`, maintenance bucket) | `pull_request`, `push` (contract set: schemas, registry, validator, fixtures) | Workflows-internal gate over the run-contract/v1 contract set: asserts the three schemas load as valid draft 2020-12 JSON Schema, `config/backplane_participants.json` keeps the required shape, and the bundled valid/invalid fixtures behave (validator self-smoke). | ⚪ Required on contract PRs | [Backplane contract integrity runs](https://github.com/stranske/Workflows/actions/workflows/health-78-backplane-contract.yml) |
Expand Down
2 changes: 1 addition & 1 deletion scripts/check_template_drift.py
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ def discover_workflow_pairs(repo_root: Path) -> list[WorkflowPair]:

pairs: dict[tuple[str, str], WorkflowPair] = {}

for main_path in sorted(main_dir.glob("agents-*.yml")):
for main_path in sorted(main_dir.glob("*.yml")):
Comment thread
stranske marked this conversation as resolved.
template_name = WORKFLOW_ALIAS_MAPPINGS.get(main_path.name, main_path.name)
template_path = template_dir / template_name
if (
Expand Down
23 changes: 23 additions & 0 deletions tests/scripts/test_check_template_drift_coverage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
from pathlib import Path

from scripts.check_template_drift import WORKFLOW_ALIAS_MAPPINGS, discover_workflow_pairs

REPO_ROOT = Path(__file__).resolve().parents[2]


def test_pair_discovery_covers_every_shared_basename() -> None:
main_dir = REPO_ROOT / ".github" / "workflows"
template_dir = REPO_ROOT / "templates" / "consumer-repo" / ".github" / "workflows"
main_names = {p.name for p in main_dir.glob("*.yml")}
template_names = {p.name for p in template_dir.glob("*.yml")}
shared = {
name
for name in main_names & template_names
if WORKFLOW_ALIAS_MAPPINGS.get(name, name) in template_names
}
covered = {pair.main_path.name for pair in discover_workflow_pairs(REPO_ROOT)}
missing = sorted(shared - covered)
assert missing == [], (
"these basenames exist in BOTH .github/workflows/ and "
"templates/consumer-repo/.github/workflows/ but have no drift pair: " + ", ".join(missing)
)
6 changes: 6 additions & 0 deletions tests/workflows/test_template_drift_workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@
WORKFLOW = REPO_ROOT / ".github" / "workflows" / "health-74-template-drift.yml"


def test_template_drift_workflow_triggers_on_all_root_workflows() -> None:
source = WORKFLOW.read_text(encoding="utf-8")
assert ".github/workflows/*.yml" in source
assert "agents-*.yml" not in source


def test_template_drift_workflow_installs_pyyaml_before_checker() -> None:
workflow = yaml.safe_load(WORKFLOW.read_text(encoding="utf-8"))
steps = workflow["jobs"]["check-drift"]["steps"]
Expand Down
Loading