Skip to content

feat(evaluator): PR-E E1 value behavior execution - #1387

Merged
briansrls merged 9 commits into
mainfrom
feat/r3-evaluator-e1-value-behavior
May 1, 2026
Merged

briansrls merged 9 commits into
mainfrom
feat/r3-evaluator-e1-value-behavior

Conversation

@briansrls

@briansrls briansrls commented May 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • add the first body-evaluator execution path for Behavior::Value -> evaluator::Value::LiteralValue
  • preserve the E0 public boundary: evaluate_body(dag, entry, &mut stack, strategy)
  • expose the E1 evaluator shell downstream slices need: eval_value, eval_port, eval_node, and evaluate_body delegation
  • keep Transform, Branch, Loop, Bind, missing entry nodes, unbound ports, and unsupported strategies fail-closed through typed EvalError variants
  • add focused unit coverage for literal Value evaluation, strategy gating, frame-backed port lookup, producer fallback, unbound ports, and unsupported behavior boundaries

Scope

  • E1 only: no Transform/Branch/Loop/Bind expansion, memoization, witness construction, runner work, test_runner.rs changes, or .dag substrate carrier edits
  • Rust runtime Value mirror matches the existing five-variant substrate Value shape and only constructs LiteralValue in this slice
  • EvalStrategy/InputEvaluationOrder mirrors are minimal E1 strategy-boundary mirrors; only ApplicativeOrder/LeftFirst executes

Validation

  • cargo fmt --all --check
  • RUSTC_WRAPPER= cargo test -p v3-compiler --lib evaluator::tests
  • RUSTC_WRAPPER= cargo clippy -p v3-compiler --lib -- -D warnings

@briansrls
briansrls marked this pull request as ready for review May 1, 2026 07:12
@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: 43589fe9 · Trigger: schedule
  • Comparison: origin/main @ 569b753b ... review/pr-1387-43589fe9 @ 43589fe9
  • Thinking: 22s wall

Findings

None. The diff stays in implementation-layer evaluator code (not new substrate on the DAG). Value and EvalError are documented as TERMINAL mirrors / fail-closed miss modes (src/v3/compiler/src/lib.rs — new blocks ~56–94, 101–135), which matches modeling-discipline.md (explicit classification for new enums). Failure paths return EvalError (MissingNode, UnboundPort, UnsupportedBehavior) rather than inventing values (P3 / fail-closed). eval_port wires frame lookup (lookup → clone) before producer resolution (101–112), matching facts flow forward for bindings vs graph producers. Tests build minimal Dag shapes and assert structured outcomes (261–470 region), consistent with TESTING.md unit-test guidance for code where a full builder surface is still evolving.

Verdict

APPROVE — Scoped E1 shell: literal Value evaluation, explicit unsupported arms for other behaviors, and regression tests; nothing in the diff clearly violates INVARIANTS, modeling discipline, CODING, or TESTING.

Exploratory observations (optional)

UnsupportedBehavior carries behavior: &'static str (91–94) produced only by the exhaustive behavior_label match (137–145), so this is not C-5-style string probing for detection. If the error surface is meant to stay stable for callers, consider an enum later; not required for this slice.

@briansrls

Copy link
Copy Markdown
Contributor Author

Manager review on draft #1387: good direction, but please keep this draft until two E0 contract gaps are fixed.\n\nBlocking before ready:\n\n1. evaluate_body must keep the E0 public API boundary. E0 locked evaluate_body(dag, body, initial_stack, strategy) -> Result<Value, EvalDiagnostic>; this draft has evaluate_body(dag, entry, _state) -> Result<Value, EvalError> and drops EvalStrategy. Even while only ApplicativeOrder/LeftFirst is executable, the strategy argument is the fail-closed boundary downstream E3/E5 depend on. Add minimal EvalStrategy / InputEvaluationOrder Rust mirrors and return UnsupportedStrategy for anything not the eager baseline.\n\n2. E1 needs the shared shell E4/E3 can call. Add an internal eval_node(dag, node, &mut stack, &strategy) shell and an eval_port helper that resolves producer ports before dispatch. Current evaluate_body directly matches the entry node and leaves no reusable dispatch/port boundary; Worker C's E4 STOP was specifically blocked on missing eval_node / eval_port.\n\nKeep the rest of the scope as-is: no Branch/Transform/Loop/Bind implementation, no runner/test_runner work, no substrate changes. The Value mirror shape itself matches runtime.dag closely enough for this slice.\n\nLocal check from a temp worktree: cargo fmt --all --check and git diff --check passed; RUSTC_WRAPPER= cargo test -p v3-compiler --lib evaluator::tests passed (11 tests).\n\n— sent from snappy-moth-795

@briansrls

Copy link
Copy Markdown
Contributor Author

Verified against current head 30c0a93: both E0 contract gaps are addressed. The evaluator module now has minimal EvalStrategy / InputEvaluationOrder mirrors, EvalError::UnsupportedStrategy, eval_node(dag, node, &mut stack, &strategy), eval_port(dag, port, &mut stack, &strategy), and evaluate_body(dag, entry, &mut stack, strategy) delegating through the shared shell. No Branch/Transform/Loop/Bind implementation, runner work, or substrate edits were added.\n\nLocal validation for the fix: cargo fmt --all --check, RUSTC_WRAPPER= cargo test -p v3-compiler --lib evaluator::tests, and RUSTC_WRAPPER= cargo clippy -p v3-compiler --lib -- -D warnings.\n\n— sent from nimble-tern-266

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 30c0a93f · Trigger: schedule
  • Thinking: 290s wall

BLOCKING (1)

Root Cause

  • src/v3/compiler/src/lib.rs EvalStateStack is keyed by arbitrary PortId and eval_port has no parameter-port gate → validate the port against the Dag, evaluate producer-owned ports through resolve_producer_opt first, and use frame lookup only for producerless parameter ports.

Non-blocking — Strengths

  • src/v3/compiler/src/lib.rs Classification: code (.rs/stage0); the E1 literal-value path and unsupported Behavior arms are scoped and fail closed with typed EvalError.

⚠️ One blocking evaluator authority issue needs to be fixed before this API becomes the E1 behavior boundary.

Comment thread src/v3/compiler/src/lib.rs Outdated
strategy: &EvalStrategy,
) -> Result<Value, EvalError> {
ensure_supported_strategy(strategy)?;
if let Ok(value) = state.lookup(port) {

This comment was marked as resolved.

@briansrls

Copy link
Copy Markdown
Contributor Author

Fixed in f83bb10. Verified the finding was valid: eval_port previously consulted EvalStateStack before checking whether the port had a DAG producer. It now resolves dag.resolve_producer_opt(&port) first and dispatches through eval_node when a producer exists; frame lookup is used only for producerless ports. Added focused tests for producer authority over a shadowing frame binding and producerless innermost-frame lookup.\n\nLocal validation: cargo fmt --all --check, RUSTC_WRAPPER= cargo test -p v3-compiler --lib evaluator::tests, and RUSTC_WRAPPER= cargo clippy -p v3-compiler --lib -- -D warnings.\n\n— sent from nimble-tern-266

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: codex / codex-default
  • Commit: f83bb105 · Trigger: schedule
  • Comparison: origin/main @ 569b753b ... review/pr-1387-f83bb105 @ f83bb105
  • Thinking: 61s wall

Findings

  • BLOCKING: src/v3/compiler/src/lib.rs:90 and src/v3/compiler/src/lib.rs:96 add Unsupported inhabitants to types documented as Rust mirrors of src/v3/std/runtime.dag’s PR-A.3 strategy carriers. The substrate carrier has exactly EvalStrategy = ApplicativeOrder { input_order: InputEvaluationOrder } and InputEvaluationOrder = LeftFirst, with the comment that additional strategy/order variants must land with executable behavior. This admits states the substrate authority does not, violating P2 single-authority / illegal-states-unrepresentable. Keep unsupported strategy as an evaluator input/error test shape outside the mirrored carrier, or land the substrate variant and executable rule together.

Verdict

REQUEST_CHANGES. The value execution shell is otherwise narrow and fail-closed, but the public strategy carrier currently drifts from the locked substrate authority. I did not run builds or tests, per instruction.

@briansrls

Copy link
Copy Markdown
Contributor Author

Fixed in f06cf0f. Verified the finding was valid: the previous minimal strategy mirrors admitted non-substrate Unsupported inhabitants. EvalStrategy and InputEvaluationOrder now mirror src/v3/std/runtime.dag exactly: ApplicativeOrder { input_order: InputEvaluationOrder } and LeftFirst only. The public strategy boundary remains on evaluate_body / eval_node / eval_port, but unsupported strategies are now unrepresentable until a future substrate carrier and executable rule land together.\n\nLocal validation: cargo fmt --all --check, RUSTC_WRAPPER= cargo test -p v3-compiler --lib evaluator::tests, and RUSTC_WRAPPER= cargo clippy -p v3-compiler --lib -- -D warnings.\n\n— sent from nimble-tern-266

@briansrls

Copy link
Copy Markdown
Contributor Author

Verified the 07:32 REQUEST_CHANGES against current head f06cf0fb5: the blocking strategy-carrier issue is resolved on the branch. EvalStrategy and InputEvaluationOrder now mirror the closed PR-A.3 substrate carriers exactly (only ApplicativeOrder { input_order: LeftFirst } / LeftFirst), with no unsupported Rust-only inhabitants.\n\nLocal verification on current head: cargo fmt --all --check, RUSTC_WRAPPER= cargo test -p v3-compiler --lib evaluator::tests, and RUSTC_WRAPPER= cargo clippy -p v3-compiler --lib -- -D warnings all passed. CI is still running on GitHub; no additional commit needed from the manager side.\n\n— sent from snappy-moth-795

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: f06cf0fb · Trigger: schedule
  • Comparison: origin/main @ 7c750541 ... review/pr-1387-f06cf0fb @ f06cf0fb
  • Thinking: 39s wall

Findings

None. The new Value / EvalStrategy / EvalError types include explicit dissolution receipts (terminal / scoped terminal), eval_* surfaces return Result<_, EvalError> instead of fabricating values on unsupported paths, and tests assert typed outcomes (UnsupportedBehavior, UnboundPort, MissingNode) rather than string probing on diagnostics.

Verdict

APPROVE — The change is small, implementation-layer (Rust evaluator mirror, not new Dag substrate), matches CODING’s data + free functions and structured error shape, and aligns with TESTING’s practical path for #[cfg(test)] code that builds minimal Dag shapes. Nothing in the diff clearly violates INVARIANTS or docs/modeling-discipline.md in a way that needs a fix before merge.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: codex / codex-default
  • Commit: 818af9eb · Trigger: schedule
  • Comparison: origin/main @ d8ddc112 ... review/pr-1387-818af9eb @ 818af9eb
  • Thinking: 55s wall

Verdict: APPROVE — the diff is narrowly scoped to the evaluator shell, keeps missing/unsupported execution paths typed and fail-closed, and the new Rust mirrors are documented against the existing runtime.dag carriers. I don’t see a concrete violation of the pinned modeling, coding, or testing discipline in the changed lines.

@briansrls

Copy link
Copy Markdown
Contributor Author

Addressed the active codex blocker by f488d55af. eval_port now first proves the PortId exists in the Dag via dag.port_opt(&port); producer-owned ports still evaluate through the DAG producer, and frame lookup is only reached for producerless ports that are members of the DAG. Added a regression test proving a stale PortId present in an EvalFrame still fails closed as UnboundPort when absent from the DAG.\n\nLocal checks: cargo fmt --all --check, RUSTC_WRAPPER= cargo test -p v3-compiler --lib evaluator::tests, and RUSTC_WRAPPER= cargo clippy -p v3-compiler --lib -- -D warnings. Push hook also ran cargo fmt --all --check.\n\n— sent from snappy-moth-795

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: openai-pro / gpt-5-5-pro
  • Commit: f488d55a · Trigger: manual
  • Comparison: main @ d8ddc112 ... feat/r3-evaluator-e1-value-behavior @ f488d55a
  • Conversation: View conversation

1. Story of the diff

This PR adds the first executable shell for the evaluator’s E1 “value behavior” path inside src/v3/compiler/src/lib.rs. It introduces typed runtime-facing carriers (Value, EvalStrategy, EvalError) as Rust mirrors of the substrate/runtime concepts, then wires a narrow evaluator: evaluate_body delegates to eval_node, eval_node currently accepts only Behavior::Value, and unsupported core behaviors fail closed through EvalError::UnsupportedBehavior rather than being collapsed or guessed. The port boundary is also made explicit: eval_port first validates the port against the DAG, prefers a DAG producer when one exists, and only consults the frame stack for producerless DAG ports. The tests exercise the intended E1 surface directly with small DAG shapes: literal value execution, producer authority over frame bindings, innermost frame lookup for producerless ports, stale-port rejection, missing-entry rejection, and fail-closed behavior for Transform/Branch/Loop/Bind.

2. Invariant categories

  1. LAYER MODEL (substrate vs implementation).

Compliant — this diff does not mutate dag.rs or substrate .dag declarations; it adds implementation-side evaluator carriers and explicitly labels them as mirrors rather than new substrate authorities. The Value mirror is bounded to the runtime carrier’s five inhabitants at src/v3/compiler/src/lib.rs:56-72, and the strategy mirror is bounded to ApplicativeOrder / LeftFirst at src/v3/compiler/src/lib.rs:81-96.

  1. INVARIANTS.md + modeling-discipline.md.

Compliant — fail-closed and single-authority behavior are handled at the evaluator boundary: eval_port rejects ports absent from the DAG at src/v3/compiler/src/lib.rs:124-125, prefers the DAG producer over frame state at src/v3/compiler/src/lib.rs:127-128, and maps missing frame bindings to EvalError::UnboundPort at src/v3/compiler/src/lib.rs:130-133. eval_node likewise returns MissingNode for absent nodes and UnsupportedBehavior for non-Value behaviors at src/v3/compiler/src/lib.rs:147-152, so unsupported E1 behavior does not fabricate a runtime value.

  1. CODING.md.

Compliant — the new surface follows data + free functions rather than object behavior: Value, EvalStrategy, and EvalError are plain carriers at src/v3/compiler/src/lib.rs:64-112, while the executable operations are explicit free functions with precise inputs and Result<Value, EvalError> outputs at src/v3/compiler/src/lib.rs:114-162. The strategy dependency is also explicit in the signatures for eval_port, eval_node, and evaluate_body, rather than hidden as global evaluator state.

  1. TESTING.md.

Compliant — the diff adds behavior-focused unit tests at the evaluator surface. The positive path is covered by eval_value_constructs_literal_runtime_value and eval_node_dispatches_value_behavior_to_literal_runtime_value at src/v3/compiler/src/lib.rs:306-337; producer/frame authority is covered at src/v3/compiler/src/lib.rs:339-390; stale/missing inputs are covered at src/v3/compiler/src/lib.rs:393-450; and all four currently unsupported behavior arms are covered fail-closed at src/v3/compiler/src/lib.rs:453-560. These are direct unit-level checks, not broad pipeline fixtures.

  1. LOCKED DESIGN DECISIONS.

N/A — the diff does not modify locked design documents or substrate declarations. The closest touched surface is the strategy mirror, and the comment explicitly says additional inhabitants must land with substrate carriers and executable evaluator rules at src/v3/compiler/src/lib.rs:83-87.

  1. TRACKED vs UNTRACKED DEBT.

Compliant — the new mirror/scaffold surfaces are documented and bounded. Value states its mirror source, current host-helper role, and replacement trigger via generated substrate-backed calls at src/v3/compiler/src/lib.rs:56-62; EvalStrategy states the exact current scope and extension gate at src/v3/compiler/src/lib.rs:81-87; and the E1 unsupported behavior modes are explicitly typed as current miss modes at src/v3/compiler/src/lib.rs:98-111. I do not see a new TODO, unbounded temporary shape, or silent bridge in the changed lines.

3. Verdict

APPROVE

The PR keeps the E1 evaluator intentionally narrow, typed, and fail-closed. The producer/frame authority boundary is explicit, unsupported behavior is represented as a diagnostic result rather than a fabricated value, and the added tests cover the load-bearing behavior without reaching beyond this slice.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: f488d55a · Trigger: schedule
  • Comparison: origin/main @ d8ddc112 ... review/pr-1387-f488d55a @ f488d55a
  • Thinking: 20s wall

Findings: None. The diff aligns the Rust evaluator surface with runtime.dag (Value, NamedField, EvalStrategy, InputEvaluationOrder), documents TERMINAL/dissolution receipts (consistent with docs/modeling-discipline.md coproduct checkpoints), uses structured EvalError / Result instead of silent failures (INVARIANTS.md P3 / fail-closed), keeps producer-before-frame precedence in eval_port (single authority for produced ports), and follows CODING-style free functions with explicit inputs. Tests are behavior-named, mostly minimal-Dag construction with one targeted std_fixture_bootstrap_snapshot use (TESTING.md practical guidance).

Verdict: APPROVE — Scoped E1 value-path execution with explicit unsupported-behavior errors; mirrors substrate and does not show rubric violations in the changed lines.

@briansrls

Copy link
Copy Markdown
Contributor Author

Current head f488d55a has fresh approvals after the eval_port authority fix. Confirmed check state: fmt, ci, and v3 are passing; self_host_ratchet is still pending. No additional code change is indicated by the latest review comments.\n\nThe older codex blocker is addressed by f488d55af: frame lookup is gated by DAG port membership and producer-owned ports evaluate through the DAG producer first.\n\n— sent from snappy-moth-795

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: claude / claude-opus-4-7
  • Commit: a7f90c47 · Trigger: schedule
  • Comparison: origin/main @ f7d5ff3d ... review/pr-1387-a7f90c47 @ a7f90c47
  • Thinking: 21s wall

Verdict: APPROVE — small, narrowly scoped PR-E E1 slice. Adds eager eval_value/eval_port/eval_node/evaluate_body plus a Value/EvalStrategy/EvalError triad, all carrying TERMINAL dissolution receipts pointing back to the substrate carriers in runtime.dag. Only Behavior::Value evaluates; Transform/Branch/Loop/Bind all fail closed via UnsupportedBehavior and have explicit regression tests. eval_port checks the dag producer first and refuses frame bindings for ports absent from the dag (eval_port_rejects_frame_binding_for_port_absent_from_dag) — fail-closed as INVARIANTS demands. Tests are hermetic and behavior-driven per TESTING.md.

Exploratory observations (non-blocking):

  • eval_node matches on strategy solely to consume it — fine as a forward-compat shape lock for PR-A.3, but worth a // strategy threaded through; arms land with later PR-E slices comment so future readers don't think it's dead.
  • Value mirrors the five substrate inhabitants but only LiteralValue is constructed in this slice. The receipt notes this is a host helper until generated substrate-backed calls replace it; tracked debt looks bounded.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: openai-pro / gpt-5-5-pro
  • Commit: a7f90c47 · Trigger: manual
  • Comparison: main @ f7d5ff3d ... feat/r3-evaluator-e1-value-behavior @ a7f90c47
  • Conversation: View conversation

1. Story of the diff

This PR adds the first executable slice of the evaluator: an E1 value-only runtime shell inside src/v3/compiler/src/lib.rs. It introduces Rust runtime carriers for Value, EvalStrategy, and EvalError, then wires eval_value, eval_port, eval_node, and evaluate_body so a Behavior::Value node can execute into a typed Value::LiteralValue while all missing ports/nodes and non-Value behaviors fail closed. The load-bearing authority decision is in eval_port: it checks that the requested port exists in the Dag, prefers the Dag producer when present, and only consults the frame stack for producerless ports. The tests exercise the happy path, producer-vs-frame precedence, frame lookup for producerless ports, missing-node/missing-port failures, and fail-closed rejection of Transform/Branch/Loop/Bind.

2. Invariant categories

  1. LAYER MODEL (substrate vs implementation).

Compliant — this is implementation-only Rust evaluator code, not a Dag/substrate mutation: the diff reads existing substrate-shaped Behavior through dag.node_opt(&node) at src/v3/compiler/src/lib.rs:147 and dispatches only Behavior::Value(value) at src/v3/compiler/src/lib.rs:148; it does not add Dag fields, ports, nodes, or behavior variants.

  1. INVARIANTS.md + modeling-discipline.md.

Compliant — fail-closed and single-authority behavior are explicit: eval_port rejects ports absent from the Dag at src/v3/compiler/src/lib.rs:124-125, preserves producer authority before frame lookup at src/v3/compiler/src/lib.rs:127-129, and maps unresolved frame lookup to EvalError::UnboundPort at src/v3/compiler/src/lib.rs:130-133. eval_node likewise rejects missing nodes through EvalError::MissingNode at src/v3/compiler/src/lib.rs:147 and unsupported behavior arms through EvalError::UnsupportedBehavior at src/v3/compiler/src/lib.rs:149-152, rather than fabricating a runtime value.

  1. CODING.md.

Compliant — the evaluator follows the data + free-functions style: eval_port declares all dependencies in its signature at src/v3/compiler/src/lib.rs:118-123, eval_node does the same at src/v3/compiler/src/lib.rs:136-141, and the public shell returns Result<Value, EvalError> rather than a sentinel value at src/v3/compiler/src/lib.rs:156-162.

  1. TESTING.md.

Compliant — tests were added at the right level for this slice: small unit-style tests construct minimal Dag shapes and assert on evaluator behavior, e.g. literal execution at src/v3/compiler/src/lib.rs:306-331, producer-over-frame authority at src/v3/compiler/src/lib.rs:339-353, producerless frame lookup at src/v3/compiler/src/lib.rs:356-378, and fail-closed unsupported behavior cases at src/v3/compiler/src/lib.rs:453-559.

  1. LOCKED DESIGN DECISIONS.

N/A — the diff does not alter files or declarations marked locked in the provided reference material; it only adds a Rust evaluator shell and tests around existing Dag/runtime concepts.

  1. TRACKED vs UNTRACKED DEBT.

Compliant — the new Rust mirrors are documented and bounded rather than silent scaffolds: Value names itself as a mirror of src/v3/std/runtime.dag and says it exists until generated substrate-backed calls replace the host helper at src/v3/compiler/src/lib.rs:56-62; EvalStrategy bounds the current executable strategy to ApplicativeOrder / LeftFirst and requires future inhabitants to land with substrate carriers and evaluator rules at src/v3/compiler/src/lib.rs:81-87; unsupported non-Value behavior remains a typed miss mode through EvalError at src/v3/compiler/src/lib.rs:98-111.

3. Verdict

APPROVE. The diff is narrow, fail-closed, and well-tested for the E1 value-only evaluator boundary; I did not find a diff-citable invariant or coding violation.

@briansrls
briansrls merged commit 88dc434 into main May 1, 2026
4 checks passed

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: a7f90c47 · Trigger: schedule
  • Thinking: 178s wall

Non-blocking — Strengths

  • src/v3/compiler/src/lib.rs Classification: code (.rs/stage0); the E1 Value path is narrow, strategy is preserved at the public boundary, unsupported behavior arms fail closed, and the prior producer-authority bug is covered by regression tests.

✅ No blocking concerns in the changed lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant