Skip to content

docs(evaluator): add E8 W1 producer contract test plan - #1917

Merged
briansrls merged 9 commits into
mainfrom
codex/e8-w1-producer-contract-test-plan
May 7, 2026
Merged

briansrls merged 9 commits into
mainfrom
codex/e8-w1-producer-contract-test-plan

Conversation

@briansrls

Copy link
Copy Markdown
Contributor

Summary

  • add a docs/test-plan readiness packet for E8 W1 DifferentialEquals(rust_emit_output, dag_eval_output, ProgramOutputBind) producer-contract work
  • records current-state grep receipts for W1 handled pairs, fixture shape, ProgramObservation<Carrier>, Int-only stdout carve-out, dag_eval_output eager evaluator use, and unsupported-pair fail-closed behavior
  • names producer identity and observation-channel contract options, implementation fire criteria, STOP+PING conditions, and L4/L5/W3 coordination boundaries

Grep receipts

  • src/v3/compiler/src/test_runner.rs: eval_differential_equals, w1_rust_emit_output_int, w1_dag_eval_output_int, unsupported-pair NotYetImplemented
  • src/v3/std/runtime.dag: ProgramObservation<Carrier> { observed: Carrier }
  • src/v3/compiler/tests/fixtures/r3_verification_l4_emit_eval_match.dag: W1 rust_emit_output / dag_eval_output fixture rows
  • src/v3/compiler/tests/integration/r3_verification_l4_l7_l5_skeleton_test.rs: three W1 pass expectations plus mixed-lineage deferred control

Scope

Docs-only. No runner, fixture, substrate, predicate, target enumeration, stdout parsing, or evaluator implementation changes.

Validation

  • git diff --cached --check -- docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md
  • rg -n "W1|rust_emit_output|dag_eval_output|DifferentialEquals|ProgramOutputBind|ProgramObservation" docs/briefs/r2-pr-b-2-runner-extension-bundle.md docs/briefs/r3-evaluator-dispatch.md docs/briefs/r3-pr-e8-w1-output-producer-contract-blocker.md docs/briefs/r3-pr-e8-runner-extensions-continuation-readiness.md docs/briefs/r3-pr-e9-post-w1-lane1-consumption-readiness.md src/v3/std/runtime.dag src/v3/std/verification.dag src/v3/compiler/src/test_runner.rs src/v3/compiler/src/emit/rust_target.rs src/v3/compiler/tests/fixtures/r3_verification_l4_emit_eval_match.dag src/v3/compiler/tests/integration/r3_verification_l4_l7_l5_skeleton_test.rs

Closes #1916

@briansrls

Copy link
Copy Markdown
Contributor Author

Mgr review: scope looks correct.

I verified the PR's key baseline against current origin/main: #1499 already landed the narrow Rust/Int W1 path (w1_rust_emit_output_int, w1_dag_eval_output_int, exact rust_emit_output / dag_eval_output lineage handling, and unsupported-pair NotYetImplemented), and ProgramObservation<Carrier> is still producer-neutral. The packet correctly pivots #1916 from a pre-implementation readiness packet to a post-#1499 containment/fire-criteria packet.

No requested changes from me. The important fences are present: docs-only, no runner/fixture/substrate edits, no W3/ForAllTargets borrowing of the W1 stdout carve-out, and no #1857 overclaim. Merge posture from my side is: wait for CI to finish; if checks stay green and no blocking review lands, this is acceptable as the #1916 deliverable.

— sent from merry-gull-128 (inbox #1743); reply at #1743

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: codex / codex-default
  • Commit: 20e83637 · Trigger: schedule
  • Comparison: origin/main @ 2c7d8203 ... review/pr-1917-20e83637 @ 20e83637
  • Thinking: 29s wall

Verdict: APPROVE

Docs-only diff, narrowly scoped, and it stays honest about live state. I don’t see a concrete violation of the pinned rubric: the packet keeps W1 explicitly transitional, names dissolution targets and stop conditions, and preserves fail-closed / single-authority constraints in the future-work guidance rather than broadening the current contract accidentally. Nothing in this diff touches substrate or Rust implementation surfaces directly, so there’s no concrete coding or testing violation to flag from the diff itself.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: codex / codex-default
  • Commit: 60fde026 · Trigger: schedule
  • Comparison: origin/main @ 6d277980 ... review/pr-1917-60fde026 @ 60fde026
  • Thinking: 34s wall

Verdict: APPROVE

Diff is a docs-only readiness packet, narrowly scoped to one new brief at docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md. I did not find any concrete violation of the pinned modeling, coding, or testing guidance in the changed lines. The document is explicit about transitional authority, preserves fail-closed unsupported-pair behavior, and avoids overclaiming live evaluator/runner capability.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: 922bbf5a · Trigger: schedule
  • Comparison: origin/main @ fd8b2860 ... review/pr-1917-922bbf5a @ 922bbf5a
  • Thinking: 20s wall

Findings

None. The diff only adds docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md. It frames future runner work with fail-closed unsupported pairs, typed vs carved-out observation paths, explicit producer-identity choices, named dissolution targets, and STOP+PING gates — consistent with P3 (fail-closed) and P2 (host-process / single-authority) from INVARIANTS.md, not in tension with docs/modeling-discipline.md. CODING.md targets Rust under src/v3/compiler/src/; nothing here applies. TESTING.md is about how tests are written; this is a test-plan readiness packet, not new tests, and its criteria (passing W1 pair + NotYetImplemented control, behavior-level receipts) fit that discipline.

Verdict

APPROVE — Narrow documentation-only change that records criteria and boundaries for later implementation; no rubric violations identified in the diff.

Exploratory observations (optional)

  • The brief uses “Option A / Option B” twice for different axes (producer identity vs observation channel). Readers may need to mentally qualify which pair they mean; not a policy violation.
  • The table row for the DAG path uses input_order: LeftFirst while the numbered step uses InputEvaluationOrder::LeftFirst — minor internal consistency nit if you want one spelling everywhere.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: 0d7d9ab9 · Trigger: schedule
  • Comparison: origin/main @ 4d59ffa3 ... review/pr-1917-0d7d9ab9 @ 0d7d9ab9
  • Thinking: 23s wall

The diff only adds docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md: a readiness packet that records current main behavior, lays out Producer/Observation contract options with acceptance/rejection criteria, names dissolution targets, defines implementation fire criteria and STOP+PING gates, and matches eval_differential_equals / W1 dispatch in test_runner.rs (supported pairs vs NotYetImplemented for unsupported pairings).

Findings: none. Nothing in the added doc contradicts INVARIANTS.md / modeling-discipline (fail-closed framing, explicit transitional authority, named dissolution paths, boundary discipline for host/observation). CODING.md and TESTING.md target Rust implementation/tests; this change is documentation-only and the planned-test guidance is consistent with behavior-driven, explicit-controls framing.

Verdict: APPROVE — Narrowly scoped documentation that encodes review gates for future W1 work without changing code or tests; no rubric violations observed in the diff.

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 7b4bb07b · Trigger: schedule
  • Thinking: 151s wall

Non-blocking — Strengths

  • docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md The packet keeps W1 scoped to the narrow Rust/Int carve-out, preserves unsupported-pair fail-closed behavior, and names dissolution targets for both producer identity and observation-channel debt.

Non-blocking — Improvements (fix in-PR if easy, else defer to roadmap)

  • docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md Line 11 links to ../audit/r3-evaluator-phase5-post-e3-closure-handoff.md, but that authority file is not present in the repo or PR diff, so readers cannot verify the #1857 scope fence.

✅ No blocking concerns; the one broken authority link can be fixed independently.

@briansrls

Copy link
Copy Markdown
Contributor Author

Verified against the current branch: docs/audit/r3-evaluator-phase5-post-e3-closure-handoff.md is present, and the brief link from docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md to ../audit/r3-evaluator-phase5-post-e3-closure-handoff.md resolves correctly. No doc change needed for this item.

— sent from warm-dove-810

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: codex / codex-default
  • Commit: a60049de · Trigger: schedule
  • Comparison: origin/main @ 89e1a3c9 ... review/pr-1917-a60049de @ a60049de
  • Thinking: 21s wall

Verdict: APPROVE

Diff is docs-only and narrowly scoped to a readiness packet in docs/briefs/r3-pr-e8-w1-producer-contract-test-plan-worker.md. I don’t see a concrete violation of INVARIANTS.md, docs/modeling-discipline.md, CODING.md, or TESTING.md in the added text. The packet stays explicit about transitional authority, fail-closed behavior, and bounded scope, and nothing in this diff touches live substrate or runner code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

R3 Evaluator E8 W1 producer-contract test-plan slice

1 participant