Skip to content

ci: restore the original 90% changed-line coverage floor #7013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
serrrfirat merged 2 commits into from
Aug 2, 2026
Merged

ci: restore the original 90% changed-line coverage floor #7013

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
31d67a9
ci: relax changed coverage floors to 95/85
serrrfirat Aug 2, 2026
56b20a8
ci: restore original 90% changed-line floor
serrrfirat Aug 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions docs/internal/testing-playbook.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,9 +360,12 @@ The merged Reborn LCOV report enforces three complementary ratchets:
baseline (which supersedes the historical 80.81% floor);
- critical production crates have percentage and covered-line floors in
[`tests/integration/coverage-floor.toml`](../../tests/integration/coverage-floor.toml);
- added, instrumentable production lines and branch arms must be covered. The
denominator comes from the pull-request diff intersected with LLVM `DA` and
`BRDA` records, not from a hand-maintained file list.
- added, instrumentable production code must meet the original committed 90%
changed-line floor. Changed-branch coverage is still required in LCOV and
reported for review, but has no universal percentage floor. The denominator
comes from the pull-request diff intersected with LLVM `DA` and `BRDA`
records, not from a hand-maintained file list. Missing production coverage
and changed files with no measured instrumented lines remain hard failures.

Changed-code exemptions live in
[`tests/integration/changed-coverage-exemptions.toml`](../../tests/integration/changed-coverage-exemptions.toml).
Expand Down
53 changes: 53 additions & 0 deletions scripts/ci/test-reborn-changed-coverage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,59 @@ check_text "line denominator is reported" "Changed line coverage: 100.00% (3/3)"
check_text "branch denominator is reported" "Changed branch coverage: 100.00% (2/2)"
check_report_text "machine report preserves the branch denominator" '"instrumented_branches": 2'

echo "▶ restored original changed-line floor"
cat >"${work}/policy.toml" <<'TOML'
[policy]
line_percent = 90.0
branch_percent = 0.0
TOML
threshold_lines=20
: >"${case_root}/${source_path}"
: >"${work}/change.diff"
printf '%s\n' \
"diff --git a/${source_path} b/${source_path}" \
"--- /dev/null" \
"+++ b/${source_path}" \
"@@ -0,0 +1,${threshold_lines} @@" >>"${work}/change.diff"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
for line in $(seq 1 "${threshold_lines}"); do
printf 'pub fn threshold_line_%s() {}\n' "${line}" >>"${case_root}/${source_path}"
printf '+pub fn threshold_line_%s() {}\n' "${line}" >>"${work}/change.diff"
done
Comment on lines +140 to +143

@coderabbitai coderabbitai Bot Aug 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Avoid unquoted word splitting in the fixture loop.

Use a Bash arithmetic loop instead of unquoted $(seq ...). This keeps the iteration set controlled and removes word-splitting behavior from merge-gating tooling.

Proposed fix 
-for line in $(seq 1 "${threshold_lines}"); do
+for ((line = 1; line <= threshold_lines; line++)); do

As per path instructions, CI scripts under scripts/** must use set -euo pipefail and quoted expansions.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for line in $(seq 1 "${threshold_lines}"); do
printf 'pub fn threshold_line_%s() {}\n' "${line}" >>"${case_root}/${source_path}"
printf '+pub fn threshold_line_%s() {}\n' "${line}" >>"${work}/change.diff"
done
for ((line = 1; line <= threshold_lines; line++)); do
printf 'pub fn threshold_line_%s() {}\n' "${line}" >>"${case_root}/${source_path}"
printf '+pub fn threshold_line_%s() {}\n' "${line}" >>"${work}/change.diff"
done
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/ci/test-reborn-changed-coverage.sh` around lines 140 - 143, Update
the fixture loop around threshold_lines to use Bash arithmetic iteration instead
of unquoted $(seq ...), while preserving the existing generated source and diff
output. Ensure threshold_lines expansions and all path variables remain quoted,
and confirm the script uses set -euo pipefail as required for scripts under
scripts/**.

Source: Path instructions

write_threshold_lcov() {
local line_18_hits="$1"
printf 'SF:%s\n' "${case_root}/${source_path}" >"${work}/coverage.lcov"
for line in $(seq 1 17); do
printf 'DA:%s,1\n' "${line}" >>"${work}/coverage.lcov"
done
printf 'DA:18,%s\nDA:19,0\nDA:20,0\n' "${line_18_hits}" \
>>"${work}/coverage.lcov"
for line in $(seq 1 20); do
printf 'BRDA:%s,0,0,0\n' "${line}" >>"${work}/coverage.lcov"
done
printf 'LF:20\nBRF:20\nend_of_record\n' >>"${work}/coverage.lcov"
}

write_threshold_lcov 1
run_gate
check_rc "90% changed lines pass at the original floor" 0
check_text "line floor denominator is reported" "Changed line coverage: 90.00% (18/20)"
check_text "ungated branch coverage remains visible" "Changed branch coverage: 0.00% (0/20)"
check_text "uncovered branch detail remains visible" "${source_path}:1 branch 0/0"
check_report_text "machine report records the 90% line floor" '"threshold_percent": 90.0'
check_report_text "machine report records the zero branch floor" '"branch_threshold_percent": 0.0'

write_threshold_lcov 0
run_gate
check_rc "changed-line coverage below 90% fails" 1
check_text "line-floor failure names the original threshold" "line coverage 85.00% is below 90.0%"

printf '%s\n' 'pub fn classify(value: bool) -> bool {' ' value' '}' >"${case_root}/${source_path}"
cat >"${work}/policy.toml" <<'TOML'
[policy]
line_percent = 100.0
branch_percent = 100.0
TOML

echo "▶ diff markers inside hunk content are parsed by their first byte"
cat >"${work}/change.diff" <<'DIFF'
diff --git a/crates/ironclaw_demo/src/lib.rs b/crates/ironclaw_demo/src/lib.rs
Expand Down
18 changes: 14 additions & 4 deletions scripts/ci/test_reborn_changed_coverage.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
#!/usr/bin/env python3
"""Workflow-contract tests for strict changed-line and branch coverage."""
"""Workflow-contract tests for thresholded changed-line and branch coverage."""

from __future__ import annotations

import ast
import re
import subprocess
import tempfile
import tomllib
import unittest
from pathlib import Path

Expand All @@ -16,7 +17,7 @@


class ChangedCoverageWorkflowTests(unittest.TestCase):
def test_strict_gate_sabotage_harness_passes(self):
def test_gate_sabotage_harness_passes(self):
result = subprocess.run(
["bash", "scripts/ci/test-reborn-changed-coverage.sh"],
cwd=ROOT,
Expand All @@ -32,7 +33,16 @@ def test_strict_gate_sabotage_harness_passes(self):
)
self.assertIsNotNone(success, result.stdout)

def test_reborn_workflow_runs_strict_gate_and_preserves_machine_report(self):
def test_committed_policy_restores_original_line_floor(self):
with (ROOT / "tests/integration/changed-coverage-exemptions.toml").open(
"rb"
) as handle:
policy = tomllib.load(handle)["policy"]

self.assertEqual(policy["line_percent"], 90.0)
self.assertEqual(policy["branch_percent"], 0.0)

def test_reborn_workflow_runs_gate_and_preserves_machine_report(self):
workflow = (ROOT / ".github/workflows/reborn-tests.yml").read_text(
encoding="utf-8"
)
Expand All @@ -53,7 +63,7 @@ def test_reborn_workflow_runs_strict_gate_and_preserves_machine_report(self):
self.assertNotIn(
"--threshold",
gate_step,
"the strict line/branch gate has no threshold CLI and must gate every denominator",
"thresholds come from the reviewed manifest, not a workflow override",
)
self.assertRegex(
workflow,
Expand Down
15 changes: 9 additions & 6 deletions tests/integration/changed-coverage-exemptions.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# Changed-code coverage is intentionally strict: every LLVM-instrumentable
# production line and branch arm added by a PR must execute. An exemption is
# exact-line only and requires an owner, reason, tracking issue, and review
# date. Whole-file and glob exemptions are not supported.
# Changed-code coverage retains the original 90% changed-line floor. Branch
# coverage remains required in LCOV and reported for review, but it has no
# universal percentage floor: defensive async/backend arms made a blanket
# branch threshold incentive-incompatible with meaningful tests. Missing
# production coverage and changed files with an empty instrumented denominator
# still fail closed. Exemptions remain exact-line only; whole-file and glob
# exemptions are not supported.

[policy]
line_percent = 100.0
branch_percent = 100.0
line_percent = 90.0
branch_percent = 0.0

# Example (remove the comment markers only after reviewer approval):
#
Expand Down
Loading