Skip to content

P5 bootstrap-evaluator corpus runtime implementation per worksheet on main (post #4143) — runtime verdict authority, not authoring-time const witnesses - #4150

Closed
briansrls wants to merge 7 commits into
mainfrom
session/witty-crab-534
Closed

briansrls wants to merge 7 commits into
mainfrom
session/witty-crab-534

Conversation

@briansrls

@briansrls briansrls commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implements the P5 bootstrap-evaluator corpus runtime slice from docs/planning/v4-p5-bootstrap-evaluator-corpus-runtime-worksheet-2026-06-01.md. The manual corpus runner rebuilds each rostered TestClaimEvalSubject with the bootstrap v4_evaluator runtime pin, checks that pin on the execution path, and invokes run_test_claim before folding the existing CorpusEvalReport.

The host transport now fails closed by adding an emitted Rust integration test to the generated crate and running witness_manual_corpus_gate_closed(). The JSON receipt reports runtime_gate_bool: true and runtime_witness: PASS_EXECUTED only after that emitted test passes.

Validation

  • bash -n scripts/v4-testclaim-corpus-eval.sh — passed.
  • git diff --check — passed.
  • CTRL_BUILD_WRAP_CARGO=0 cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_m1_rust_emit_probe_modeled_and_bound_to_ci_yml -- --exact --quiet — passed.
  • CTRL_BUILD_WRAP_CARGO=0 cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_bankruptcy_tier0_ -- --quiet — passed.
  • CTRL_BUILD_WRAP_CARGO=0 cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_wave1_ -- --quiet — passed.
  • CTRL_BUILD_WRAP_CARGO=0 cargo test -p v3-compiler --test integration v4_workflow_ci_runner_dag_smoke_test::v4_workflow_ci_t38_script_checks_generated_manual_corpus_eval_receipt -- --exact --quiet — passed.

Worksheet F1-F6

  • F1: script receipt emits execution_status: runtime_verdicts after the emitted Rust runtime gate test passes.
  • F2: run_manual_testclaim_corpus_eval executes the modeled runner path via run_test_claim instead of repackaging roster rows.
  • F3: runtime subject construction pins bootstrap_projection_inputs.runtime_model.runtime_identity; the runner checks it against v4_evaluator_runtime_id before executing entries.
  • F4: manual_corpus_gate(report) reads the runtime-produced CorpusEvalReport; the host transport runs witness_manual_corpus_gate_closed() and fails closed unless it evaluates true.
  • F5: the script no longer treats source presence as sufficient; source checks are preflight only, followed by emitted runtime execution.
  • F6: dissolution comments are amended in the runner, manual witness, and CI model surfaces.

@briansrls
briansrls marked this pull request as ready for review June 1, 2026 02:53
@briansrls
briansrls force-pushed the session/witty-crab-534 branch from 7f25ffa to 6c4fdf3 Compare June 1, 2026 03:06

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: abd66b9b · Trigger: schedule
  • Thinking: 271s wall

BLOCKING (1)

Root Cause

  • src/v4/test/claim/workflow/testclaim_corpus_runner.dag BootstrapEvaluatorCorpusRuntimeEval allows any Symbol as evaluator_pin → make the v4 pin invariant unrepresentable or return a report/outcome variant that preserves the mismatch reason and roster facts.

⚠️ The runtime path is mostly clean, but the pin-mismatch branch needs a typed failure shape instead of collapsing to an empty report.

@briansrls

Copy link
Copy Markdown
Contributor Author

Violations (could not place on specific lines):

  • src/v4/test/claim/workflow/testclaim_corpus_runner.dag:154 BLOCKING: Returning an empty CorpusEvalReport on evaluator-pin mismatch fabricates a plausible empty corpus and drops the roster/pin-mismatch facts instead of carrying a typed failure, violating INVARIANTS.md P2 boundary discipline and facts-flow-forward.

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 6c4fdf34 · Trigger: schedule
  • Thinking: 238s wall

BLOCKING (1)

Root Cause

  • src/v4/test/claim/manual/manual_corpus_roster.dag manual_corpus_node_runtime_value_rows remains List<TestClaimRun<Node, RuntimeValue>> → introduce a claim/subject roster carrier that cannot contain authoring-time verdicts, then execute run_test_claim from that carrier.

⚠️ The runtime gate still depends on verdict-bearing roster rows, so the PR has not fully closed the authoring-time const-witness authority split.


fn runtime_subject_from_roster_run(
runtime: BootstrapEvaluatorCorpusRuntimeEval,
run: TestClaimRun<Node, RuntimeValue>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKING: runtime_subject_from_roster_run still accepts TestClaimRun input, so the runtime path carries and silently drops a pre-authored verdict instead of making the worksheet §4 F2 "claims, not pre-built runs" authority unrepresentable (P2 facts-flow-forward/P5).

briansrls added a commit that referenced this pull request Jun 2, 2026
briansrls added a commit that referenced this pull request Jun 2, 2026
briansrls added a commit that referenced this pull request Jun 2, 2026
… eval in testclaim_corpus_runner; address cursor RC; coordinate vivid-lynx on rust emit (#4259)

* WIP: T-38 bootstrap-evaluator corpus runtime (#4150 restart): per-row T-22 ev

* WIP: T-38 bootstrap-evaluator corpus runtime (#4150 restart): per-row T-22 ev

* T-38: align corpus eval receipt with per-row subjects

* T-38: clarify corpus receipt is source-shape only
@briansrls

Copy link
Copy Markdown
Contributor Author

Closing — superseded by #4259 (T-38A.3 manual corpus runner via run_test_claim, MERGED 2026-06-02T01:00:54Z). Original cursor RC addressed at #4259 spawn per operator 2026-06-01 policy. witty-crab-534 was archived; nimble-crab-195 caretaker → neat-wolf adhoc-168ccc84-887 (eager-cat-510) completed the work.

— sent from nimble-dove-733

@briansrls briansrls closed this Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant