Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion docs/INTEGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -549,7 +549,9 @@ jobs:

```json
{
"coverage": 80.0,
"line": 80.0,
"warn_drop": 1.0,
"recovery_days": 3,
"updated": "2025-12-30",
"notes": "Initial baseline - adjust based on project maturity"
}
Expand All @@ -569,6 +571,13 @@ Copy from `templates/consumer-repo/.github/workflows/maint-coverage-guard.yml`
| **Baseline Issue** | Auto-created/updated issue when coverage drops below baseline |
| **Trend Artifacts** | `coverage-trend.json` and `coverage-trend-history.ndjson` for analysis |

**Guard tolerance and recovery:** `warn_drop` is a percentage-point allowance below
the baseline (default `1.0` if omitted). A run creates or updates a breach issue only
when `current < baseline - warn_drop`; a smaller drop is an intentional no-op. An open
breach issue closes only after coverage is at or above the full baseline for
`recovery_days` consecutive samples (default `3`), so the warning allowance never
closes an issue early.

**Soft vs Hard Gate:**

- **Soft gate** (`enable-soft-gate: true`): Reports coverage but doesn't fail the build
Expand Down
2 changes: 2 additions & 0 deletions docs/ci/WORKFLOWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ The gate uses the shared `.github/scripts/detect-changes.js` helper to decide wh
* [`maint-sync-env-from-pyproject.yml`](../../.github/workflows/maint-sync-env-from-pyproject.yml) keeps `pyproject.toml`, templates, and `requirements.lock` aligned to the canonical `autofix-versions.env` file.
* [`maint-coverage-guard.yml`](../../.github/workflows/maint-coverage-guard.yml) periodically verifies that the latest Gate run meets baseline coverage expectations.
* **The coverage baseline is per-repo and is NOT synced** (`config/coverage-baseline.json`; see `.github/sync-manifest.yml` exclusions). The percentage may be keyed `line` or `coverage` — both `tools/coverage_trend.py` and `tools/coverage_guard.py` accept either, and `line` wins when both are present. A repo with no baseline file reports `baseline: null` with a `baseline_status` of `absent`/`unreadable`/`no_recognised_key`, and **no delta at all**; it is never reported as a baseline of `0.00%`, because a delta measured against nothing renders as a large improvement on every run and can never fail.
* **A guard breach is deliberately less sensitive than the baseline itself.** `warn_drop` is a percentage-point allowance below the configured baseline; it defaults to `1.0` when omitted. The guard creates or updates a breach issue only when `current < baseline - warn_drop`, so a result that is below baseline but inside that allowance is a successful no-op, not a broken monitor. Set `warn_drop` explicitly in `config/coverage-baseline.json` when a repository needs a different tolerance.
* **Recovery is stricter than alerting.** An existing breach issue closes only after coverage is back at or above the full baseline for the configured `recovery_days`/`recovery_window` (default three) consecutive samples. The warning-drop allowance never authorizes early closure.
* **Coverage rows measured outside the project root are reported, not silently averaged in.** A test that copies the source tree into a tmpdir makes coverage.py record each module twice — once real, once as a barely-executed duplicate that sorts to the top of every hotspot table. `coverage-trend.json` carries `foreign_file_count`, `foreign_files` and `current_project_only` alongside the unmodified `current`, and the summary names `[tool.coverage.run] omit` as the fix. `current` is deliberately left as coverage.py computed it so the record still agrees with `coverage.xml` and the delta job.
* [`maint-metrics-retention.yml`](../../.github/workflows/maint-metrics-retention.yml) runs `scripts/metrics_retention.py` nightly (02:00 UTC) to enforce the retention policy in `config/retention-policy.json`, uploads `metrics-retention.ndjson` as an artifact, and surfaces the storage reduction percentage in the step summary. When no metrics logs are present, the run succeeds with a zero-file no-op summary.
* [`maint-46-post-ci.yml`](../../.github/workflows/maint-46-post-ci.yml) wakes up after Gate completes, validates the workflow syntax with `actionlint`, downloads the Gate artifacts, renders the consolidated CI summary (including coverage deltas), and republishes the Gate commit status while saving a markdown preview for evidence capture.
Expand Down
10 changes: 10 additions & 0 deletions tests/docs/test_workflow_source_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,16 @@ def test_workflows_doc_names_gate_autofix_dispatch_path() -> None:
assert 'autofixDispatch --> autofixLoop["Agents Autofix Loop' in doc


def test_workflows_doc_explains_coverage_guard_tolerance_and_recovery() -> None:
doc = WORKFLOWS_DOC.read_text(encoding="utf-8")

assert "warn_drop" in doc
assert "defaults to `1.0`" in doc
assert "current < baseline - warn_drop" in doc
assert "recovery_days" in doc
assert "never authorizes early closure" in doc


def test_agent_routing_doc_covers_enabled_registry_agents() -> None:
registry = yaml.safe_load(AGENT_REGISTRY.read_text(encoding="utf-8"))
routing_doc = MULTI_AGENT_ROUTING_DOC.read_text(encoding="utf-8")
Expand Down
5 changes: 5 additions & 0 deletions tests/test_coverage_guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ def test_load_baseline_enforces_minimum_recovery_days(tmp_path: Path) -> None:
assert baseline.recovery_days == 3


def test_coverage_breach_requires_drop_beyond_warn_threshold() -> None:
assert not coverage_guard.is_coverage_breach(84.0, 85.0, 1.0)
assert coverage_guard.is_coverage_breach(83.99, 85.0, 1.0)


def test_compute_top_files_prioritises_missing_lines() -> None:
coverage = {
"files": {
Expand Down
14 changes: 13 additions & 1 deletion tools/coverage_guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,11 @@ def load_baseline(path: Path) -> BaselineConfig:
)


def is_coverage_breach(current: float, baseline: float, warn_drop: float) -> bool:
"""Return whether coverage exceeds the configured drop allowance."""
return current < baseline - warn_drop


def compute_top_files(coverage_data: dict[str, Any], limit: int = 15) -> list[FileCoverage]:
"""Return the most useful file-level coverage rows for issue comments."""
files = coverage_data.get("files", {})
Expand Down Expand Up @@ -680,6 +685,7 @@ def main(args: list[str] | None = None) -> int:
)
return 0
delta = current - baseline
warn_drop = load_baseline(parsed.baseline_path).warn_drop
Comment thread
stranske marked this conversation as resolved.
configured_recovery_window = max(
1,
_to_int(
Expand Down Expand Up @@ -714,7 +720,7 @@ def main(args: list[str] | None = None) -> int:
return 0

# Create or update issue
if current < baseline:
if is_coverage_breach(current, baseline, warn_drop):
try:
_find_or_create_issue(
repo=parsed.repo,
Expand All @@ -726,6 +732,12 @@ def main(args: list[str] | None = None) -> int:
print(f"Failed to create or update coverage issue: {exc}", file=sys.stderr)
return 1
else:
if current < baseline:
print(
f"Coverage {current:.2f}% is within the configured {warn_drop:.2f}-point "
f"drop allowance below baseline {baseline:.2f}% - no new issue needed"
)
return 0
print(f"Coverage {current:.2f}% meets baseline {baseline:.2f}% - no open issue needed")
if not _recovery_window_satisfied(
trend_data,
Expand Down
Loading