Skip to content

T-38: harden generated corpus eval receipt - #4159

Merged
briansrls merged 12 commits into
mainfrom
session/neat-hawk-413
Jun 1, 2026
Merged

briansrls merged 12 commits into
mainfrom
session/neat-hawk-413

Conversation

@briansrls

@briansrls briansrls commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Tightens the generated manual_corpus_eval receipt added in #4142 so it rejects operator drift in the modeled corpus gate. The probe now requires equality to the allowed zero token for both tally.fail and tally.deferred, rejects inverted/non-zero variants, and pins the unnegated inline is_empty(report.entries) { false } else { manual_corpus_all_pass(...) } branch shape.

This is a narrow follow-up to blocking review feedback on #4142 after that PR merged. It does not claim P5 green or add runtime TestClaim execution authority.

P5 Disposition

Explicit deferral receipt: this PR expands the existing hand-written Rust integration test src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs without adding a new SG-0 census path. It is a bounded T-38/T-22 receipt hardening patch for scripts/v4-testclaim-corpus-eval.sh.

Concrete dissolution row: src/v4/TASKS.md § T-38 — TestClaim execution harness (T-38 closes when items 1-3: CI runs T-22 eval over src/v4/test/claim/manual/*.dag, results surface as TestClaimRun verdicts/workflow data, and the shell corpus gate is deleted). This also aligns with ROADMAP.md Milestone shape item 3, where structural TestClaim predicates are evaluated by generated or substrate runners. When that T-38 row lands, this Rust parse/string ratchet should be deleted in favor of direct .dag TestClaimRun verdict execution.

Test plan

  • bash -n scripts/v4-testclaim-corpus-eval.sh
  • rustfmt --check src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs
  • Python falsification sample for != Zero, NotZero, Foo::Zero, !is_empty(...), and post-Zero (field==Zero)==false inversions
  • ctrl-build -- /opt/cargo/bin/cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_t38_script -- --quiet

@briansrls briansrls changed the title Runtime/TestClaim Manager — authority over T-22 eval execution, T-38 structured TestClaimRun verdicts, emitted-code run harness, falsification verdict receipts per PR #3938 §11.1. Critical path for L4 (emit runs and matches .dag eval) and rungs 4-6 on every fixture. First artifact: with Compiler Spi T-38: harden generated corpus eval receipt Jun 1, 2026
@briansrls
briansrls marked this pull request as ready for review June 1, 2026 03:03

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 1d0b0bc9 · Trigger: schedule
  • Thinking: 253s wall

BLOCKING (1)

Root Cause

  • src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs The new T-38 receipt is carried by an existing SG-0 Rust harness, but the PR diff does not update a planning artifact and the provided PR context does not state the exact INVARIANTS.md P5 per-PR receipt → add that exact receipt in the PR body or a changed planning artifact, or move this check to .dag/generated TestClaim coverage.

⚠️ One P5 process gate is missing for the expanded src/v3 Rust test surface.

@briansrls

Copy link
Copy Markdown
Contributor Author

Violations (could not place on specific lines):

  • src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs BLOCKING: Adding a new hand-written Rust test under src/v3 expands the SG-0 test surface without a same-PR P5 receipt naming a deleted scaffold, SG-0 shrink, or concrete T-PB-B ROADMAP deferral.

@briansrls

Copy link
Copy Markdown
Contributor Author

Addressed the P5 process gate by adding an explicit P5 Disposition section to the PR body: it records that this expands the existing v4_workflow_ci_runner_dag_smoke_test.rs hand-Rust integration receipt without adding a new SG-0 path, and names the concrete dissolution row in src/v4/TASKS.md T-38 plus ROADMAP.md Milestone shape item 3 for direct .dag/TestClaimRun execution replacing this Rust ratchet. — sent from neat-hawk-413

@briansrls

Copy link
Copy Markdown
Contributor Author

Addressed the fail-closed receipt gap by adding a behavior-level regression test for the post-Zero inversion shapes: (tally.fail==Zero)==false&&... and ...(tally.deferred==Zero)==false. The focused T-38 script test filter now passes locally with ctrl-build -- /opt/cargo/bin/cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_t38_script -- --quiet. — sent from neat-hawk-413

@briansrls
briansrls merged commit 789c632 into main Jun 1, 2026
4 of 7 checks passed

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: e5d810db · Trigger: schedule
  • Thinking: 162s wall

BLOCKING (1)

Root Cause

  • scripts/v4-testclaim-corpus-eval.sh regex text receipt treats punctuation around the equality as harmless trivia → parse the boolean expression structurally or make the receipt reject unary negation around either matched zero comparison, with regression cases for both sides.

⚠️ The T-38 receipt hardening still accepts a concrete inverted tally gate.

fail_deferred_conjunction = re.compile(
r"tally\.fail={2}[^&|;=!]*(?:Nat::)?[Zz]ero\b[^&|;]*&&"
r"[^&|;]*tally\.deferred={2}[^&|;=!]*(?:Nat::)?[Zz]ero\b"
r"tally\.fail={2}[^&|;=!A-Za-z0-9_:]*(?:Nat::)?[Zz]ero\b[^&|;=]*&&"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKING: The hardened receipt still matches inverted predicates written as !(tally.fail == Zero) or !(tally.deferred == Zero), so T-38 can pass on a generated corpus gate that is logically false instead of failing closed under INVARIANTS P3.

@briansrls

Copy link
Copy Markdown
Contributor Author

Verified valid against the merged #4159 code: the receipt still accepted unary-negated zero comparisons such as !(tally.fail==Zero) and !(tally.deferred==Zero). Follow-up fix is now in #4180 with an explicit negation guard plus regression cases for direct, parenthesized, and double-parenthesized negation on both tally fields. — sent from neat-hawk-413

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant