Skip to content

T-38: reject negated corpus tally predicates - #4180

Merged
briansrls merged 22 commits into
mainfrom
session/neat-hawk-413
Jun 1, 2026
Merged

briansrls merged 22 commits into
mainfrom
session/neat-hawk-413

Conversation

@briansrls

@briansrls briansrls commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #4159: the generated corpus-eval receipt now rejects unary-negated zero comparisons such as !(tally.fail==Zero) and !(tally.deferred==Zero). The existing conjunction probe still requires zero Fail and zero Deferred, but an explicit inverted_zero_comparison guard now fails closed before accepting the receipt.

The T-38 smoke regression now exercises direct, parenthesized, and double-parenthesized negation on both tally fields.

P5 Disposition

Explicit deferral receipt: this PR extends the existing hand-written Rust integration test src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs without adding a new SG-0 census path. It is a bounded T-38/T-22 receipt hardening patch for scripts/v4-testclaim-corpus-eval.sh.

Concrete dissolution row: src/v4/TASKS.md § T-38 — TestClaim execution harness (T-38 closes when items 1-3: CI runs T-22 eval over src/v4/test/claim/manual/*.dag, results surface as TestClaimRun verdicts/workflow data, and the shell corpus gate is deleted). This also aligns with ROADMAP.md Milestone shape item 3, where structural TestClaim predicates are evaluated by generated or substrate runners. When that T-38 row lands, this Rust parse/string ratchet should be deleted in favor of direct .dag TestClaimRun verdict execution.

Test plan

  • bash -n scripts/v4-testclaim-corpus-eval.sh
  • rustfmt --check src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs
  • Python falsification sample for unary negation around tally.fail == Zero and tally.deferred == Zero
  • ctrl-build -- /opt/cargo/bin/cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_t38_script -- --quiet

@briansrls briansrls changed the title Runtime/TestClaim Manager — authority over T-22 eval execution, T-38 structured TestClaimRun verdicts, emitted-code run harness, falsification verdict receipts per PR #3938 §11.1. Critical path for L4 (emit runs and matches .dag eval) and rungs 4-6 on every fixture. First artifact: with Compiler Spi T-38: reject negated corpus tally predicates Jun 1, 2026
@briansrls
briansrls marked this pull request as ready for review June 1, 2026 04:48
@briansrls

Copy link
Copy Markdown
Contributor Author

Verified against the current PR body on #4180: this is already addressed without a code/planning-file change. The body has exactly one P5 Disposition section stating that this extends the existing src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs hand-written Rust integration receipt without adding a new SG-0 path, and it names the concrete dissolution row: src/v4/TASKS.md T-38 plus ROADMAP.md Milestone shape item 3 for direct .dag / TestClaimRun execution replacing this Rust ratchet. — sent from neat-hawk-413

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 898dd16a · Trigger: schedule
  • Thinking: 280s wall

BLOCKING (1)

Root Cause

  • scripts/v4-testclaim-corpus-eval.sh string-regex receipt treats a positive zero-comparison substring as sufficient evidence instead of validating the enclosing Bool expression → parse the generated manual_corpus_all_pass body or reject symmetric false-equality wrappers before accepting the tally conjunction

⚠️ One blocking receipt bypass remains.

"generated tally source for zero Fail/Deferred predicate",
)
normalized_all_pass = "".join(all_pass_body.split())
inverted_zero_comparison = re.compile(

This comment was marked as resolved.

@briansrls

Copy link
Copy Markdown
Contributor Author

Investigated the CI failure on head 93f55d4. fmt and affected pass; ci fails only because ci_floor fails closed. The actual failure is in ci_floor at M1 v4 full-tree rust emit probe: v2 compile succeeds with compiled: 351 files emitted, 0 diagnostics, then emitted Rust cargo check exits 101 on the known full-tree generated-Rust debt class (TestClaimRun missing generics, duplicate target model items, FreeMonoid dropck, nat_cata, etc.). This PR diff only changes scripts/v4-testclaim-corpus-eval.sh and src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs; it does not touch src/v4 or the emitter surface, so there is no scoped fix to push here. — sent from neat-hawk-413

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 93f55d42 · Trigger: schedule
  • Thinking: 111s wall

BLOCKING (1)

Root Cause

  • scripts/v4-testclaim-corpus-eval.sh string-level blacklist of selected inverted forms → validate the enclosing Bool expression or require an exact top-level tally.fail == Zero && tally.deferred == Zero shape before accepting the receipt

⚠️ One receipt bypass remains.

Comment thread scripts/v4-testclaim-corpus-eval.sh Outdated
)
normalized_all_pass = "".join(all_pass_body.split())
inverted_zero_comparison = re.compile(
r"(?<![A-Za-z0-9_:])(?:!\(*|\(*false\)*={2}\(*)"

This comment was marked as resolved.

@briansrls

briansrls commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor Author

Verified against current #4180 head 503cfbc and the current PR body: this P5 receipt already exists without needing a code/planning-file change. The body has exactly one P5 Disposition section stating that this extends the existing hand-written Rust integration test src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs without adding a new SG-0 census path, and it names the concrete dissolution row: src/v4/TASKS.md T-38 plus ROADMAP.md Milestone shape item 3 for direct .dag / TestClaimRun execution replacing this Rust ratchet. — sent from neat-hawk-413

@briansrls

Copy link
Copy Markdown
Contributor Author

Investigated the CI failure on head 503cfbc. fmt and affected pass; ci fails only because ci_floor fails closed. The failing ci_floor step is M1 v4 full-tree rust emit probe: v2 compile succeeds with compiled: 351 files emitted, 0 diagnostics, then emitted Rust cargo check exits 101 on the known full-tree generated-Rust debt class (TestClaimRun missing generics, FreeMonoid dropck, nat_cata move, etc.).

I compared current PR run 26740397675 against recent main ci run 26734801058. Both show the same 26 rustc error classes; the only count drift is in existing E0432/E0433 import-resolution lines. This PR diff only changes scripts/v4-testclaim-corpus-eval.sh and src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs, with no src/v4 or emitter changes, so there is no scoped fix to push here. — sent from neat-hawk-413

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 503cfbcb · Trigger: schedule
  • Thinking: 207s wall

BLOCKING (1)

Root Cause

  • scripts/v4-testclaim-corpus-eval.sh string-level receipt accepts an occurrence of the predicate instead of the returned Bool expression → parse/validate the function body AST or require the exact final returned expression to be the zero Fail && zero Deferred conjunction

⚠️ One receipt bypass remains in the generated corpus tally checker.

Comment thread scripts/v4-testclaim-corpus-eval.sh Outdated
r"tally\.fail={2}[^&|;=!A-Za-z0-9_:]*(?:Nat::)?[Zz]ero\b[^&|;=]*&&"
r"[^&|;]*tally\.deferred={2}[^&|;=!A-Za-z0-9_:]*(?:Nat::)?[Zz]ero\b[^&|;=]*(?:;|\})"
r"(?:^|;)\(*tally\.fail={2}[^&|;=!A-Za-z0-9_:]*(?:Nat::)?[Zz]ero\b\)*&&"
r"\(*tally\.deferred={2}[^&|;=!A-Za-z0-9_:]*(?:Nat::)?[Zz]ero\b\)*(?:;|\})"

This comment was marked as resolved.

@briansrls

Copy link
Copy Markdown
Contributor Author

Investigated the CI failure on head 0d5a3b8. fmt and affected pass; ci fails only because ci_floor fails closed. The failing ci_floor step is still M1 v4 full-tree rust emit probe: v2 compile succeeds with compiled: 351 files emitted, 0 diagnostics, then emitted Rust cargo check exits 101 on the existing full-tree generated-Rust debt class (TestClaimRun missing generics, FreeMonoid dropck, nat_cata move, etc.).

I compared current PR run 26744865142 against recent main ci run 26734801058. Both show the same 26 rustc error classes; this PR run has fewer E0432/E0433 import-resolution lines than main and no new error classes. This PR diff only changes scripts/v4-testclaim-corpus-eval.sh and src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs, with no src/v4 or emitter changes, so there is no scoped fix to push here. — sent from neat-hawk-413

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: openai-pro / gpt-5-5-pro
  • Commit: 0d5a3b88 · Trigger: manual
  • Comparison: main @ 113372a6 ... session/neat-hawk-413 @ 0d5a3b88
  • Conversation: View conversation

1. Story of the diff

This PR tightens the T-38 generated corpus-eval receipt so manual_corpus_all_pass can no longer satisfy the receipt by negating the zero Fail / Deferred tally predicates. The script now normalizes the generated function body, rejects explicit inversions such as !(tally.fail == Zero), false == (...), and true != (...), then requires a direct tally.fail == Zero && tally.deferred == Zero conjunction near a closing brace. The integration smoke test mirrors the new regex needles, and the negative test expands from a few local regex assertions into a small receipt_accepts harness covering positive, inverted, not-zero, and simple “computed but not returned” cases. chatgpt-review-6ca7239c-9fbf-4a…

2. Invariant categories

  1. LAYER MODEL (substrate vs implementation).

N/A — the diff is implementation/test receipt logic in scripts/v4-testclaim-corpus-eval.sh and a Rust integration test; it introduces no Dag substrate type, std/ primitive, Dag mutation path, or new modeled variant.

  1. INVARIANTS.md + modeling-discipline.md.

Finding — scripts/v4-testclaim-corpus-eval.sh:424 / fail-closed receipt boundary. The changed receipt regex ends the accepted conjunction with only \}:

r"\(*tally\.deferred={2}[^&|;=!A-Za-z0-9_:]*(?:Nat::)?[Zz]ero\b\)*\}"

Because that } is not anchored to the end of manual_corpus_all_pass’s body, the receipt can still pass when the required conjunction is merely the final expression of an inner block and the actual function result is later false. For example, after the same whitespace normalization used by the script, a body shaped like lettally=x;{letinner=1;tally.fail==Zero&&tally.deferred==Zero};false} is accepted by fail_deferred_conjunction.search(...) even though manual_corpus_all_pass returns false. This violates P3 fail-closed / verification-as-receipt discipline: the receipt reports the generated all-pass gate as structurally present when the actual returned predicate is not the required gate. The fix is to make the conjunction match the returned tail of the function body, not any inner block tail — for example by anchoring the pattern to the end of the normalized body and covering the intended final-expression forms, plus adding the nested-block negative case to the test. chatgpt-review-6ca7239c-9fbf-4a…

  1. CODING.md.

Compliant — aside from the blocking regex boundary above, the added script code keeps the existing free-function style, uses a named local regex plus a typed ReceiptError instead of a silent boolean, and the test-side helper receipt_accepts makes the expected predicate composition readable rather than repeating two regex calls inline. chatgpt-review-6ca7239c-9fbf-4a…

  1. TESTING.md.

Finding — src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs:1396 / regression test misses the bypass shape. The new non_returned cases only cover direct tail forms like tally.fail==Zero&&tally.deferred==Zero;false} and letok=...;false}; they do not cover a conjunction returned from an inner block followed by outer false, which the production regex still accepts because of scripts/v4-testclaim-corpus-eval.sh:424. The test is correctly at the receipt-regression level, but it needs the nested-block false-positive case to prove the receipt actually checks the returned manual_corpus_all_pass predicate. chatgpt-review-6ca7239c-9fbf-4a…

  1. LOCKED DESIGN DECISIONS.

N/A — the diff does not alter a locked design document or semantic design decision; it stays within the existing T-38 receipt-checking surface.

  1. TRACKED vs UNTRACKED DEBT.

N/A — no new TODO, scaffold, bridge, or temporary lane is introduced. The issue above is not tracked debt; it is a fail-open receipt bug in the changed code.

2.5. Top-down PM intent review

Finding — scripts/v4-testclaim-corpus-eval.sh:424 dilutes the T-38 intent. The high-level intent of this PR is to reject negated or non-authoritative corpus tally predicates, and the thesis/test direction is that tests and receipts are structural data that validate declared behavior rather than hand-maintained behavior assertions. But the changed receipt still accepts a non-returned inner-block conjunction because the required tally.fail == Zero && tally.deferred == Zero pattern is not tied to the returned function tail. A worker following this receipt faithfully could ship a generated manual_corpus_all_pass that returns false while still passing the T-38 receipt, which is exactly the semantic dilution this PR is supposed to prevent. chatgpt-review-6ca7239c-9fbf-4a…

3. Verdict

REQUEST_CHANGES. The PR is pointed in the right direction and the explicit inversion guard is useful, but the tightened conjunction regex still fails open on an inner-block/non-returned conjunction. Please anchor the receipt to the actual returned tail and add the nested-block negative regression before merge.

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 61e83ebf · Trigger: schedule
  • Thinking: 278s wall

✅ The changed receipt closes the previously flagged bypasses and I found no new blocking concerns.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: openai-pro / gpt-5-5-pro
  • Commit: 61e83ebf · Trigger: manual
  • Comparison: main @ 113372a6 ... session/neat-hawk-413 @ 61e83ebf
  • Conversation: View conversation

1. Story of the diff

This PR tightens the T-38 generated corpus-eval receipt around manual_corpus_all_pass. The script now normalizes the generated Rust body, rejects several textual forms that negate the zero Fail / Deferred tally checks, and narrows the accepted shape so the body must end with tally.fail == Zero && tally.deferred == Zero. The integration smoke test mirrors those regex expectations and expands the rejection corpus for false/true inversion, direct !, and some “good check is not returned” layouts. chatgpt-review-731abc7d-f9f9-46…

2. Invariant categories

  1. LAYER MODEL (substrate vs implementation).

N/A — diff is implementation/test receipt logic in a shell/Python script and Rust integration test; it does not alter Dag, substrate types, std/ primitives, reusable algebraic helpers, or modeled variants.

  1. INVARIANTS.md + modeling-discipline.md.

Finding — fail-closed / early detection: scripts/v4-testclaim-corpus-eval.sh:423 adds r"(?:^|;)\(*tally\.fail... and applies the conjunction regex with .search over the whole normalized body. Because that accepts any semicolon-delimited positive tail, a body like return (tally.fail == Zero) == false && tally.deferred == Zero; tally.fail == Zero && tally.deferred == Zero would still pass: the new inversion regex at scripts/v4-testclaim-corpus-eval.sh:414 only catches prefix inversions (!, false ==, true !=), while the conjunction regex matches the unreachable good tail. That leaves a route where manual_corpus_all_pass actually returns a negated fail predicate but the receipt accepts it, violating the fail-closed receipt purpose. The check needs to reject early return/early-exit before the final expression, or parse/assert the actual returned expression rather than searching for a good tail. chatgpt-review-731abc7d-f9f9-46…

  1. CODING.md.

Compliant — the changed implementation stays file-scoped and explicit: local compiled regexes, ReceiptError on failure, no new object state, no hidden global dependency, and no explanatory comment added where naming/control flow should carry the meaning.

  1. TESTING.md.

Finding — regression misses the bypass shape: src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs:1396 adds “non-returned” cases, but only for a good conjunction followed by a false tail. It does not cover the inverse layout that this PR still accepts: an early returned inverted predicate followed by a dead positive tail. Since the PR’s behavior contract is “reject negated corpus tally predicates,” the regression should include that case, preferably by exercising the actual script/check path rather than only a copied regex snippet.

  1. LOCKED DESIGN DECISIONS.

N/A — no locked design document or locked substrate/design decision is changed; the diff is a receipt-gate implementation change.

  1. TRACKED vs UNTRACKED DEBT.

N/A — no new TODO, scaffold, temporary bridge, or deferred marker is introduced.

2.5. Top-down PM intent review

Compliant with the intended direction, but blocked by the implementation gap above. The PR is semantically aimed at preserving the T-38 gate by making generated corpus evaluation fail closed when manual_corpus_all_pass is inverted, and that aligns with the project’s broader requirement that verification/test surfaces reject invalid behavior rather than hand-wave it. The problem is not PM-level scope dilution or a conflicting design direction; it is that the specific regex receipt still has an early-return bypass, so the implementation does not yet fully deliver the stated intent.

3. Verdict

REQUEST_CHANGES

The PR is focused and mostly shaped correctly, but the receipt can still accept a function that returns an inverted tally predicate before an unreachable positive tail. That is exactly the class of failure this PR is supposed to close, so it needs a fail-closed fix plus a regression for the early-return bypass before merge.

@briansrls

Copy link
Copy Markdown
Contributor Author

Investigated the CI failure on #4180 head 4605e93. The completed failed jobs are ci_floor plus the aggregate ci job that fails closed because ci_floor failed. fmt and affected pass.

The failing ci_floor step is still M1 v4 full-tree rust emit probe: v2 compile succeeds with compiled: 351 files emitted, 0 diagnostics, then emitted Rust cargo check exits 101. The failure is the existing generated-Rust baseline class: duplicate target-model items, missing generics for Outcome/FreeMonoid/TestClaimRun, recursive alias/dropck issues, and nat_cata move errors. Comparing the completed PR attempt against main run 26734801058 shows the same 26 rustc error classes and no new error class. The only count drift is in existing E0432/E0433 import-resolution lines. This PR diff only changes scripts/v4-testclaim-corpus-eval.sh and src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs, with no src/v4 or emitter changes, so there is no scoped fix to push here. A ci_floor rerun is currently in progress on the same head. — sent from neat-hawk-413

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 4605e939 · Trigger: schedule
  • Thinking: 188s wall

Non-blocking — Strengths

  • scripts/v4-testclaim-corpus-eval.sh The PR is implementation-only and keeps the receipt fail-closed against the previously demonstrated negation and tail-expression bypasses.

✅ No blocking concerns found.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: openai-pro / gpt-5-5-pro
  • Commit: 4605e939 · Trigger: manual
  • Comparison: main @ 113372a6 ... session/neat-hawk-413 @ 4605e939
  • Conversation: View conversation

1. Story of the diff

This PR tightens the T-38 generated corpus-eval receipt so manual_corpus_all_pass is only accepted when its tail expression is the positive conjunction that both Fail and Deferred tallies are zero. The script now rejects two bypass classes before checking the final conjunction: any explicit return inside the body, and negated/inverted zero checks such as !…, false == (…), or true != (…) against tally.fail / tally.deferred; then it anchors the accepted conjunction at the body tail instead of accepting any earlier occurrence. That same contract is reflected in the Rust integration smoke: the script needle test now requires the new regex probes, and the regression test mirrors the receipt function with accepted canonical cases plus rejected non-final, inverted, and non-zero variants. scripts/v4-testclaim-corpus-eval.sh:412-430; src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs:1342-1351,1372-1429. chatgpt-review-be5d4169-01d2-43…

chatgpt-review-be5d4169-01d2-43…

2. Invariant categories

  1. LAYER MODEL (substrate vs implementation). N/A — the diff is implementation/test receipt logic in a shell/Python script and a Rust integration test; it does not change substrate .dag types, Dag storage, cross-pass substrate facts, or new variants.
  2. INVARIANTS.md + modeling-discipline.md. Compliant — fail-closed / early detection is tightened: scripts/v4-testclaim-corpus-eval.sh:412-430 rejects explicit returns and inverted zero predicates before accepting only a final positive fail == Zero && deferred == Zero conjunction. This moves the receipt away from plausible-but-wrong acceptance and toward the P3/C-8 shape where malformed support rejects rather than fabricates a pass. chatgpt-review-be5d4169-01d2-43…

No predicate/walker dissolution trigger applies because this is not a substrate/std/ Bool helper over a modeled coproduct; it is a bounded receipt parser for generated Rust text.

  1. CODING.md. Compliant — the new implementation is local, named by the specific contract it checks (explicit_return, inverted_zero_comparison, fail_deferred_conjunction), and fails through the existing ReceiptError path rather than comments or silent defaults. scripts/v4-testclaim-corpus-eval.sh:412-430 keeps the control flow linear and readable without adding explanatory prose. chatgpt-review-be5d4169-01d2-43…
  2. TESTING.md. Compliant — the diff adds regression coverage at the right seam: src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs:1342-1351 verifies the script contains the new receipt probes, and :1372-1429 exercises the checker shape with accepted positive cases plus rejected inverted/non-final cases. chatgpt-review-be5d4169-01d2-43…

chatgpt-review-be5d4169-01d2-43…

Although the long-term direction is .dag TestClaim data, the current testing guidance still allows Rust integration tests during the zero-floor migration when that shape is cleanest. chatgpt-review-eef43eb7-cfac-43…

  1. LOCKED DESIGN DECISIONS. N/A — the PR does not edit or reinterpret a locked design doc, substrate shape, or workflow authority. The changed behavior is a stricter receipt for an existing T-38 CI/testclaim gate.
  2. TRACKED vs UNTRACKED DEBT. N/A — no TODOs, scaffolds, temporary bridges, or new deferred shapes are introduced. The added regexes and test cases are operative receipt logic, not parked debt.

2.5. Top-down PM intent review

Compliant — the PM-level intent is preserved. The highest-level direction says tests and CI/build orchestration should be structurally modeled, with Rust surfaces dissolving over the zero-floor trajectory, while current migration guidance still permits Rust integration tests until the hand-authored surface reaches zero. chatgpt-review-a70b79fe-9a33-42…

chatgpt-review-eef43eb7-cfac-43…

This PR does not dilute that plan into permanent Rust test authority; it tightens a T-38 receipt so generated corpus evaluation cannot pass through an earlier positive expression, explicit return bypass, or negated tally predicate. scripts/v4-testclaim-corpus-eval.sh:412-430; src/v3/compiler/tests/integration/v4_workflow_ci_runner_dag_smoke_test.rs:1372-1429. chatgpt-review-be5d4169-01d2-43…

chatgpt-review-be5d4169-01d2-43…

3. Verdict

APPROVE

No blocking findings. The PR narrows the receipt in the fail-closed direction, adds focused regression cases for the bypasses it closes, and does not introduce substrate changes, design drift, or untracked debt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant