Repository navigation
T-38: harden generated corpus eval receipt - #4159
Conversation
briansrls
left a comment
There was a problem hiding this comment.
Review metadata
- Provider / model:
codex/unknown - Commit:
1d0b0bc9· Trigger:schedule - Thinking:
253s wall
BLOCKING (1)
Root Cause
src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rsThe new T-38 receipt is carried by an existing SG-0 Rust harness, but the PR diff does not update a planning artifact and the provided PR context does not state the exact INVARIANTS.md P5 per-PR receipt → add that exact receipt in the PR body or a changed planning artifact, or move this check to .dag/generated TestClaim coverage.
|
Violations (could not place on specific lines):
|
|
Addressed the P5 process gate by adding an explicit P5 Disposition section to the PR body: it records that this expands the existing |
|
Addressed the fail-closed receipt gap by adding a behavior-level regression test for the post- |
briansrls
left a comment
There was a problem hiding this comment.
Review metadata
- Provider / model:
codex/unknown - Commit:
e5d810db· Trigger:schedule - Thinking:
162s wall
BLOCKING (1)
Root Cause
scripts/v4-testclaim-corpus-eval.shregex text receipt treats punctuation around the equality as harmless trivia → parse the boolean expression structurally or make the receipt reject unary negation around either matched zero comparison, with regression cases for both sides.
| fail_deferred_conjunction = re.compile( | ||
| r"tally\.fail={2}[^&|;=!]*(?:Nat::)?[Zz]ero\b[^&|;]*&&" | ||
| r"[^&|;]*tally\.deferred={2}[^&|;=!]*(?:Nat::)?[Zz]ero\b" | ||
| r"tally\.fail={2}[^&|;=!A-Za-z0-9_:]*(?:Nat::)?[Zz]ero\b[^&|;=]*&&" |
There was a problem hiding this comment.
BLOCKING: The hardened receipt still matches inverted predicates written as !(tally.fail == Zero) or !(tally.deferred == Zero), so T-38 can pass on a generated corpus gate that is logically false instead of failing closed under INVARIANTS P3.
|
Verified valid against the merged #4159 code: the receipt still accepted unary-negated zero comparisons such as |
Summary
Tightens the generated
manual_corpus_evalreceipt added in #4142 so it rejects operator drift in the modeled corpus gate. The probe now requires equality to the allowed zero token for bothtally.failandtally.deferred, rejects inverted/non-zero variants, and pins the unnegated inlineis_empty(report.entries) { false } else { manual_corpus_all_pass(...) }branch shape.This is a narrow follow-up to blocking review feedback on #4142 after that PR merged. It does not claim P5 green or add runtime TestClaim execution authority.
P5 Disposition
Explicit deferral receipt: this PR expands the existing hand-written Rust integration test
src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rswithout adding a new SG-0 census path. It is a bounded T-38/T-22 receipt hardening patch forscripts/v4-testclaim-corpus-eval.sh.Concrete dissolution row:
src/v4/TASKS.md§ T-38 — TestClaim execution harness (T-38 closes whenitems 1-3: CI runs T-22 eval oversrc/v4/test/claim/manual/*.dag, results surface asTestClaimRunverdicts/workflow data, and the shell corpus gate is deleted). This also aligns withROADMAP.mdMilestone shape item 3, where structuralTestClaimpredicates are evaluated by generated or substrate runners. When that T-38 row lands, this Rust parse/string ratchet should be deleted in favor of direct.dagTestClaimRunverdict execution.Test plan
bash -n scripts/v4-testclaim-corpus-eval.shrustfmt --check src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs!= Zero,NotZero,Foo::Zero,!is_empty(...), and post-Zero(field==Zero)==falseinversionsctrl-build -- /opt/cargo/bin/cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_t38_script -- --quiet