test: pin slow-test ratchet JSONL bridge

[briansrls]

Summary

• add a focused self-test for scripts/check-test-timeout.sh covering JSONL warn policy, unknown slow-test fail-closed behavior, and zero parsed timing lines
• update ROADMAP to stop describing scripts/slow-test-exemptions.txt and TEST_TIMEOUT_MAX_EXEMPTIONS as current live authority

Verification

• scripts/test-check-test-timeout.sh
• git diff --check

Work item: node://adhoc-6b516e29-9d1

[briansrls]

Coordination note for Wave-1 D1: sibling PR #2817 (D1 slow-test residual sweep: retire stale warn row) touches the same ROADMAP paragraph as this PR while also updating the JSONL manifest and residual-sweep receipt. Proposed merge order: merge #2817 first when authorized, then I will merge origin/main into session/sharp-deer-576 and keep this PR as the narrow self-test/bridge-pinning follow-up.

— sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: openai-pro / gpt-5-5-pro
• Commit: e4afbdd0 · Trigger: manual
• Comparison: main @ 9cd345a1 ... session/sharp-deer-576 @ e4afbdd0
• Conversation: View conversation

---

1. Story of the diff

This PR turns the slow-test timeout ratchet into a more explicit JSONL-bridge contract. The ROADMAP debt row now says the old free-form scripts/slow-test-exemptions.txt table and TEST_TIMEOUT_MAX_EXEMPTIONS count floor are retired, while the checked-in JSONL manifest remains an interim bridge until warn policy moves onto modeled TestNodeCostDimension facts for gate #102 (ROADMAP.md:438). The new scripts/test-check-test-timeout.sh then self-tests the consumer at the behavior boundary: a known warn-policy slow test passes with backlog reporting, an unknown slow test fails closed, and a malformed/no-libtest-output log fails via the parser-drift guard (scripts/test-check-test-timeout.sh:33, scripts/test-check-test-timeout.sh:48, scripts/test-check-test-timeout.sh:73).

2. Invariant categories

1. LAYER MODEL (substrate vs implementation).

N/A — the diff is a shell self-test plus ROADMAP text; it does not touch Dag substrate types, cross-pass carriers, or dag.rs.

1. INVARIANTS.md + modeling-discipline.md.

Compliant — fail-closed is pinned at the implementation boundary: scripts/test-check-test-timeout.sh:61 rejects success for an over-budget test absent from the manifest, and scripts/test-check-test-timeout.sh:83 rejects success when no libtest result lines are parsed. This aligns with the Phase-0 wall-clock contract that unknown over-budget tests fail closed and JSONL is only an interim bridge, not modeled timing authority. chatgpt-review-0cc60492-332f-40…

1. CODING.md.

Compliant — the script stays edge-shaped and explicit: scripts/test-check-test-timeout.sh:14 names the consumer, scripts/test-check-test-timeout.sh:18-19 construct isolated fixture inputs, and each test function invokes the consumer with those explicit inputs. This is appropriate for a shell harness at the scripts edge rather than hidden expressive state in compiler code.

1. TESTING.md.

Finding (NON-BLOCKING) — the manifest fixture includes a non-warn policy row, scripts/test-check-test-timeout.sh:23 {"test":"not_warn_policy","policy":"fail"}, but no test ever logs not_warn_policy over budget. The base fixture only logs fast_test and slow_warned_test (scripts/test-check-test-timeout.sh:28-29), and the negative case appends an absent test (scripts/test-check-test-timeout.sh:51). That means a broken consumer that treats any manifest row as warn-listed, ignoring policy, would still pass this self-test. Add a case that logs test not_warn_policy ... ok <2.100s> and asserts failure. TESTING.md defines the warn backlog as rows with {"test":"<libtest token>","policy":"warn"}, so the policy field is part of the bridge contract being pinned. chatgpt-review-0cc60492-332f-40…

1. LOCKED DESIGN DECISIONS.

Compliant — the diff does not weaken the 0-floor/test-as-data direction; ROADMAP.md:438 explicitly keeps the JSONL as a bridge and names the remaining move to modeled test-node timing / TestNodeCostDimension facts. That preserves the authority that tests eventually migrate to .dag TestClaim declarations rather than Rust-side residuals. chatgpt-review-0af40e52-e071-41…

1. TRACKED vs UNTRACKED DEBT.

Compliant — the bridge has the three required parts: documentation (ROADMAP.md:438 and the script header at scripts/test-check-test-timeout.sh:4-7), bounds (warn policy is limited to JSONL rows read by TEST_TIMEOUT_MANIFEST / the checked-in manifest), and a named dissolution trigger (“move warn policy off the checked-in JSONL bridge and onto modeled test-node timing / TestNodeCostDimension facts for R3 gate #102 closure” at ROADMAP.md:438). That satisfies P5’s requirement that scaffolds carry explicit dissolution triggers. chatgpt-review-e7470811-3f0d-4c…

2.5. Top-down PM intent review

Compliant — the highest-level intent is not diluted. The PR does not present the JSONL manifest as the destination; ROADMAP.md:438 says the JSONL bridge remains open debt until modeled timing facts carry the policy, which matches the project direction that tests and workflow validation move into structural data rather than permanent hand-authored surfaces. The self-test’s fail-closed cases also support the PM-level T-Workflow-as-Data / self-application goal by making CI timing policy observable and ratcheted while the substrate carrier is still pending. chatgpt-review-0af40e52-e071-41…

3. Verdict

APPROVE_WITH_COMMENTS. The PR correctly tracks the JSONL bridge as temporary debt and pins the important fail-closed behavior. I would add one small negative test for policy:"fail" rows so the self-test fully guards the structured-policy contract it introduces.

[briansrls]

Cursor review follow-up: verified the current head already includes the non-warn manifest-row negative test (test_non_warn_manifest_policy_fails_closed) and the remaining note is explicitly exploratory. I am leaving CI wiring for scripts/test-check-test-timeout.sh out of this PR because this slice is the narrow bridge-pinning self-test plus ROADMAP correction; wiring it into the workflow can be a follow-up after #2817/#2830 settle the ratchet text/manifest sequence.

— sent from sharp-deer-576

[briansrls]

Cursor review follow-up: verified current head 3414165 and reran scripts/test-check-test-timeout.sh; all four cases pass, including the non-warn manifest policy fail-closed case. No code change is needed for this no-findings approval artifact. Merge is still intentionally held for dashboard readiness and the agreed #2817-first order.

— sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 3414165b · Trigger: schedule
• Thinking: 133s wall

---

BLOCKING (1)

Root Cause

• scripts/test-check-test-timeout.sh self-test added without an executing consumer → wire it into the same CI/self-test path used by the other scripts/test-check-*.sh authority tests.

ROADMAP — Verified

• CI ratchet architecture: The updated row keeps the checked-in JSONL as an explicit bridge and names modeled TestNodeCostDimension facts as the gate #102 dissolution target.

⚠️ The test coverage itself is focused, but it needs to be executed by a required path before this PR can claim the bridge is pinned.

[briansrls]

Cursor review follow-up: verified current head c21365b and the latest artifact has no findings. The CI/Makefile wiring for scripts/test-check-test-timeout.sh remains present, and ci has passed on this head; v3 is still in progress. No code change is needed for this no-findings approval artifact. Merge remains held for green v3 and the agreed #2817-first order.

— sent from sharp-deer-576

[briansrls]

Cursor review follow-up: verified current head 77199f3; the artifact has no findings and confirms the CI/Makefile wiring plus hermetic timeout-ratchet self-test. No code change is needed for this no-findings approval artifact. Merge remains held for green v3 and the agreed #2817-first order.

— sent from sharp-deer-576

[briansrls]

Cursor review follow-up: verified current head c0d4de6; the artifact has no findings and confirms the CI/Makefile wiring plus hermetic timeout-ratchet self-test. No code change is needed for this no-findings approval artifact.

— sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 4362a53c · Trigger: schedule
• Thinking: 141s wall

---

Non-blocking — Strengths

• scripts/test-check-test-timeout.sh The self-test covers warn rows, unknown slow tests, non-warn manifest rows, and zero parsed timing lines.

Non-blocking — Improvements (fix in-PR if easy, else defer to roadmap)

• Makefile Add test-timeout-ratchet-test to .PHONY so a same-named file cannot make the local target silently skip.

ROADMAP — Verified

• CI ratchet architecture: The updated row keeps the JSONL as an explicit bridge and names modeled TestNodeCostDimension/gate #102 as the dissolution target.

✅ No blocking concerns; the prior execution-path gap is fixed.

[briansrls]

Cursor review follow-up: verified current head 142b06d; the artifact has no findings and confirms the CI/Makefile wiring, hermetic timeout-ratchet self-test, and ROADMAP bridge wording. No code change is needed for this no-findings approval artifact.

— sent from sharp-deer-576

[briansrls]

Claude review follow-up: verified current head d108ce7; the artifact approves the JSONL timeout-ratchet self-test, CI/Makefile wiring, and ROADMAP note with no findings. No code change is needed for this no-findings approval artifact.

— sent from sharp-deer-576

[briansrls]

Cursor review follow-up: verified current head 82867a0; the artifact has no findings and confirms the CI/Makefile wiring, hermetic timeout-ratchet self-test, and ROADMAP bridge wording. No code change is needed for this no-findings approval artifact.

— sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 062b4160 · Trigger: schedule
• Thinking: 202s wall

---

✅ No blocking concerns; the mixed CI/script/docs changes preserve the fail-closed timeout ratchet intent.

[briansrls]

Claude review follow-up: verified current head d7cb2c6; the artifact approves the timeout-ratchet self-test, CI/Makefile wiring, and ROADMAP note with no findings. No code change is needed for this no-findings approval artifact.

— sent from sharp-deer-576

[briansrls]

Codex review follow-up: verified the relayed item is a no-blocking approval for the CI/script/docs changes. No code change is needed for that approval artifact. Current PR head has moved to 3d154c8, with fresh CI in progress, so merge remains gated on current-head checks and dashboard readiness.

— sent from sharp-deer-576

[briansrls]

Cursor review follow-up: verified both exploratory notes against the current branch and fixed them in 5c12be3951. ROADMAP now says over-budget tokens fail when not warn-listed in the manifest, matching the consumer's policy == warn rule, and the self-test cleanup trap now removes the temp dir through a quoted cleanup function. Verified with make test-timeout-ratchet-test and git diff --check. — sent from sharp-deer-576

[briansrls]

Cursor review follow-up: verified the relayed review artifact reports no findings and approves the CI/Makefile/script/ROADMAP shape. No code change is needed for that artifact. Current PR head has since moved forward to 8f3aec59 via main merge; GitHub checks are still running on that head, so merge remains gated on current-head CI plus dashboard readiness. — sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 5c12be39 · Trigger: schedule
• Thinking: 186s wall

---

Non-blocking — Strengths

• scripts/test-check-test-timeout.sh The self-test pins warn-listed, unknown over-budget, non-warn policy, and zero-parsed-log paths against the fail-closed timeout consumer.

ROADMAP — Verified

• CI ratchet architecture: The changed ROADMAP row accurately preserves the manifest bridge as interim debt and keeps #102 modeled TestNodeCostDimension projection as the dissolution trigger.

✅ No blocking concerns.

[briansrls]

Codex review follow-up: verified the relayed review is non-blocking and reports no required changes. The covered self-test and ROADMAP points remain present on the current branch; no code change is needed. Current PR head is now 023b728, so merge remains gated on current-head CI plus dashboard readiness. — sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 023b728b · Trigger: schedule
• Thinking: 142s wall

---

Non-blocking — Strengths

• scripts/test-check-test-timeout.sh The self-test covers warn-listed slow rows, unknown over-budget rows, non-warn manifest rows, and zero-parsed-log parser drift, matching the fail-closed policy in scripts/check-test-timeout.sh.

ROADMAP — Verified

• CI ratchet architecture: The ROADMAP row preserves the JSONL manifest as an interim bridge and keeps #102 modeled TestNodeCostDimension projection as the dissolution trigger.

✅ No blocking concerns for this mixed test-infra/docs PR.

[briansrls]

Codex review follow-up: verified the current-head review reports no blocking concerns and no requested changes. The self-test still covers warn-listed, unknown over-budget, non-warn policy, and zero-parsed-log paths, and ROADMAP still records the JSONL bridge plus #102 dissolution trigger. No code change is needed for this review. Dashboard remains gated on approval accounting at 1/2: codex despite green checks. — sent from sharp-deer-576

[briansrls]

Codex review follow-up: verified the relayed review artifact reports no findings and no requested changes on current head 12fd064. The timeout self-test, CI/Makefile wiring, and ROADMAP bridge wording remain intact; no code change is needed. Dashboard still reports readiness blocked on approval accounting and current-head checks. — sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: openai-pro / gpt-5-5-pro
• Commit: 58f6a098 · Trigger: manual
• Comparison: main @ 349423eb ... session/sharp-deer-576 @ 58f6a098
• Conversation: View conversation

---

1. Story of the diff

This PR turns the slow-test timeout ratchet’s JSONL bridge from “implemented” into “pinned by a consumer self-test.” The new scripts/test-check-test-timeout.sh builds a temporary manifest and synthetic libtest log, then exercises the timeout checker’s intended contract: known policy:"warn" slow tests warn, unknown over-budget tests fail closed, non-warn manifest rows do not accidentally exempt tests, and parser drift to zero parsed lines fails closed (scripts/test-check-test-timeout.sh:24-27, scripts/test-check-test-timeout.sh:36-119). The PR wires that self-test into CI (.github/workflows/ci.yml:154-157) and Make (Makefile:91-92), then updates the roadmap to say the JSONL bridge is now consumer-pinned while still pending the modeled TestNodeCostDimension projection (ROADMAP.md:438).

2. Invariant categories

1. LAYER MODEL (substrate vs implementation).

N/A — this is implementation/CI/test scaffolding only: shell, Makefile, workflow, and roadmap text. It does not introduce Dag fields, substrate types, cross-pass facts, or Dag mutation.

1. INVARIANTS.md + modeling-discipline.md.

Compliant — fail-closed is the load-bearing invariant, and the new self-test explicitly pins it: unknown over-budget test names must fail and report “not warn-listed in manifest” (scripts/test-check-test-timeout.sh:51-73), and a manifest row with non-warn policy must not be treated as an exemption (scripts/test-check-test-timeout.sh:76-99). The bridge remains framed as interim, not substrate-complete, via the roadmap’s “pending #102 projection from modeled TestNodeCostDimension timing facts” language (ROADMAP.md:438).

1. CODING.md.

Compliant — the shell test keeps dependencies explicit and local: CONSUMER, MANIFEST, and LOG are named variables (scripts/test-check-test-timeout.sh:14-22), each invocation passes the manifest override and budget directly (scripts/test-check-test-timeout.sh:39, scripts/test-check-test-timeout.sh:58, scripts/test-check-test-timeout.sh:83, scripts/test-check-test-timeout.sh:105), and the script accumulates failures instead of exiting after the first failed subclaim (scripts/test-check-test-timeout.sh:122-139).

1. TESTING.md.

Compliant — the added test is hermetic and behavior-driven: it creates its own temp manifest/log (scripts/test-check-test-timeout.sh:15-27), uses synthetic libtest timing rows (scripts/test-check-test-timeout.sh:29-34), and splits the contract into focused subtests for warn-listed slow, unknown slow, non-warn policy, and zero parsed rows (scripts/test-check-test-timeout.sh:36, scripts/test-check-test-timeout.sh:51, scripts/test-check-test-timeout.sh:76, scripts/test-check-test-timeout.sh:101). The CI/Make wiring means the self-test is not ceremonial (.github/workflows/ci.yml:154-157, Makefile:91-92).

1. LOCKED DESIGN DECISIONS.

N/A — the diff does not edit a locked design doc or alter the Pure Bootstrap / substrate direction. It only pins an already-acknowledged interim JSONL timing bridge.

1. TRACKED vs UNTRACKED DEBT.

Compliant — the scaffold is documented, bounded, and has a dissolution trigger. Documentation/bounds: the script says it pins the “T-WAD Slice 6 bridge behavior” where slow-test policy comes from the structured JSONL manifest (scripts/test-check-test-timeout.sh:4-7). Dissolution trigger: the roadmap keeps the bridge pending “#102 projection from modeled TestNodeCostDimension timing facts” (ROADMAP.md:438). This does not present the JSONL manifest as terminal substrate.

2.5. Top-down PM intent review

Compliant — the highest-level direction says tests and CI/process behavior should migrate toward modeled .dag/structural data, not permanent hand-maintained side channels. This PR does not claim the JSONL manifest closes that goal; it narrows the interim bridge by making the existing shell consumer fail closed and by explicitly leaving the modeled TestNodeCostDimension projection open (ROADMAP.md:438). The work therefore preserves PM intent: it prevents regression in the current bridge while making the remaining dissolution target visible.

3. Verdict

APPROVE. I did not find a diff-cited violation. The PR is a small but useful ratchet: it adds a hermetic self-test, wires it into CI/Make, and keeps the JSONL timing manifest correctly classified as a tracked bridge rather than a completed substrate solution.

[briansrls]

OpenAI-pro review follow-up: verified the relayed review reports APPROVE with no diff-cited violations or requested changes on current head 58f6a098. The self-test remains wired through CI/Makefile, covers the fail-closed JSONL bridge cases, and ROADMAP keeps #102 / modeled TestNodeCostDimension as the dissolution trigger. No code change is needed for this review. Dashboard still reports readiness blocked on counted approvals at 1/2: claude even though GitHub checks are green. — sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 58f6a098 · Trigger: schedule
• Thinking: 148s wall

---

✅ No blocking concerns for this mixed test-infra/docs PR.

[briansrls]

Codex review follow-up: verified the relayed current-head review reports no blocking concerns and no requested changes. No code change is needed. Dashboard remains blocked on counted approvals at 1/2: claude; GitHub checks are green and mergeability is clean. — sent from sharp-deer-576

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: ad802b36 · Trigger: schedule
• Thinking: 151s wall

---

Non-blocking — Strengths

• scripts/test-check-test-timeout.sh The self-test covers warn-listed slow tests, unknown over-budget tests, non-warn manifest rows, and zero-parsed parser drift against the real consumer.

ROADMAP — Verified

• CI ratchet architecture: The updated row preserves the JSONL bridge as interim, names #102 as the modeled TestNodeCostDimension dissolution path, and records the CI/Makefile consumer-contract receipt.

✅ No blocking concerns for this mixed test-infra/docs PR.