arch(ws-17): prove product live planned-runtime cutover

[henrypark133]

Context

Product live cutover/readiness branch. It proves the product inbound path can select the planned runtime for a normal no-profile user message and persist the final assistant reply in product-visible thread history.

Master spec: docs/reborn/agent-loop-skeleton.md
Workstream brief: docs/reborn/agent-loop-briefs/product-live-cutover.md
Stack base: arch/ws-16

Latest stack maintenance on 2026-05-14:

• Rebased this branch onto its current stack base after the WS0 prompt-authority fix and the follow-up WS8/WS14-parent/WS16/WS17 conflict resolutions.
• Pushed the updated branch with force-with-lease where the remote already existed, or published it as a new branch where it did not.
• Verified the final ancestry chain from origin/reborn-integration through WS17 before publishing the PR descriptions.

What landed

• Host-runtime production coordinator wiring requires an explicit RunProfileResolver; it no longer silently relies on DefaultTurnCoordinator's text-only resolver.
• Product workflow caller-level no-profile test proves inbound user messages resolve to reborn-planned-default with the planned resolver.
• Product-live caller test builds the WS16/17 composition, submits a normal inbound user message through DefaultInboundTurnService, runs the Reborn worker, and verifies the final assistant reply is persisted.
• ThreadCheckpointLoopExitEvidencePort can verify completion evidence against the configured product/user ThreadScope, fixing owner-scoped thread verification.
• Arc<C> implements TurnCoordinator for shared live coordinators.
• build_product_live_planned_runtime(...) fails closed unless model-route resolver, input queue, cancellation factory, and identity context source are present.
• Additional cleanup from preserving the staged WS17 follow-up before the final rebase.

Reviewer focus

• This branch is product-facing: review the inbound service, coordinator resolver, worker, and thread-history evidence path together.
• No-profile product turns must use the planned resolver only when the live-required services are present.
• Cancellation and completion evidence should remain host-verified and owner/thread scoped.
• The stricter tool-result completion evidence caveat remains intentionally deferred; this PR does not claim full tool-using product-live completion.

Non-goals / deferred work

• It does not remove text-only rollback/profile routing.
• It does not claim product-visible completed tool-result evidence; that needs a dedicated follow-up.
• It does not make unsupported/missing live services best-effort; the builder should fail closed.

Validation

• git diff --check
• cargo fmt --check
• cargo test -p ironclaw_reborn --test loop_driver_host product_live_runtime
• cargo test -p ironclaw_product_workflow --test inbound_turn_contract user_message_no_profile_can_cancel_product_live_run_from_product_path
• cargo test -p ironclaw_reborn loop_exit_applier::tests::cancelled_exit_requires_observed_cancel_input
• cargo test -p ironclaw_reborn loop_exit_applier::tests::observed_host_cancellation_still_requires_final_checkpoint_when_configured
• cargo test -p ironclaw_host_runtime --test host_runtime_services_contract production_wiring_validation_rejects_noop_turn_wake_notifier
• cargo test -p ironclaw_reborn_cli --test smoke run_reports_runtime_readiness_snapshot_without_touching_v1_state

Stack position

[#3550 ws-0] state/checkpoint foundation -> reborn-integration
   |-- #3551 ws-1 strategy alpha -> ws-0
   |-- #3552 ws-2 strategy beta -> ws-0
   |-- #3553 ws-3 strategy gamma -> ws-0
   |-- #3643 ws-3.5 loop family registry -> ws-0
   '-- #3554 level1-merged -> ws-0
         |-- #3555 ws-4 planner facade -> level1
         |-- #3556 ws-5 default strategies -> level1
         '-- #3557 level2-merged -> level1
               '-- #3596 ws-6a canonical executor -> level2
                     '-- #3597 ws-7 PlannedDriver adapter -> ws-6a
                           '-- #3598 ws-8 integration/test support -> ws-7
                                 |-- #3644 ws-9 capability host wiring -> ws-8
                                 |-- #3645 ws-10 checkpoint load/resume -> ws-8
                                 |-- #3646 ws-11 input port -> ws-8
                                 |-- #3647 ws-12 progress port -> ws-8
                                 |-- #3648 ws-13 cancellation accessor -> ws-8
                                 |-- #3649 ws-15 prompt/identity context -> ws-8
                                 '-- #3650 ws-14-parent integrated host ports -> ws-8
                                       '-- #3651 ws-14 planned default registration -> ws-14-parent
                                             '-- #3652 ws-16 live runtime wiring -> ws-14
                                                   '-- #3653 ws-17 product live cutover -> ws-16

[gemini-code-assist]

Code Review

This pull request implements the 'Product-Live Readiness Evidence' workstream (WS-17), focusing on ensuring the Reborn agent loop can be safely composed with the product path. Key changes include the introduction of a RunProfileResolver in the host runtime wiring, the addition of a ProductLiveCancellationProbe to verify that cancellation factories are externally controllable, and the implementation of build_product_live_planned_runtime with fail-closed readiness checks for critical adapters. Additionally, ThreadCheckpointLoopExitEvidencePort was updated to support scoped verification and product-path cancellation observation. Extensive contract tests were added to validate that no-profile turns correctly exercise the planned runtime and support cancellation. Documentation and CLI tools were also updated to reflect these readiness snapshots. I have no feedback to provide as there were no review comments to assess.

[zmanian]

Review notes — WS17 product live cutover evidence

To answer the obvious question first: this is not the flip. build_product_live_planned_runtime exists and is well-tested, but no production entry point calls it over build_default_planned_runtime. The PR is instrumentation + readiness gates + caller-level evidence; the actual composition-root swap is the follow-up.

Reading this as a "live cutover" PR would be wrong; reading it as "everything that needs to be true before the cutover" is correct.

Fail-closed builder is the right shape

build_product_live_planned_runtime rejects each missing component (ModelRouteResolver, InputQueue, CancellationFactory, IdentityContextSource) and rejects inert probes, with a typed error enum per component. ProductLiveCancellationProbe + verify_product_live_cancellation_probe actually exercise the request/observe path before building the runtime — meaningful liveness check, not a presence check.

This closes the gate that WS16 left open (issue #3602). The remaining work is purely calling this from a product entry point.

Caller-level tests are the cutover evidence

Three tests in inbound_turn_contract.rs drive DefaultInboundTurnService → build_product_live_planned_runtime → real worker, not mocks: happy-path persistence (MessageStatus::Finalized), mid-flight cancellation through the product TurnCoordinator, and rejection of an unretained cancellation factory. Genuinely caller-driven.

The new owner-scoping fix on ThreadCheckpointLoopExitEvidencePort::new_with_thread_scope (binding completion evidence to configured ThreadScope, rejecting turn-scope mismatch) is a real correctness fix that rides along — worth calling out separately from cutover work.

Items to track

• Tool-result completion evidence is deferred per the PR description. The live path is text-only reply persistence; agents that need tool-loop completion will still hit the deferred caveat.
• is_cancellation_observed silent Ok(false) for the unwired-factory case is intentional but worth a debug! log — operationally, "did anyone observe the cancel?" is the question on-call will ask.
• No metrics/replay-trace evidence. For a true cutover I'd expect telemetry on resolved_run_profile_id == "reborn-planned-default" rates and cancel observation latency. Plausibly in the live ops stack — confirm before flipping.
• The actual flip (composition-root swap to call build_product_live_planned_runtime) needs its own PR. That PR closes issue Wire Reborn loop production readiness gate into startup composition #3602.

Minor

• RuntimeShellReport → RuntimeReadinessSnapshot rename is cosmetic but cascades through smoke tests; fine.
• default_planned_run_profile_resolver().unwrap() in tests — acceptable per CLAUDE.md, but a typed expect reads better.
• Unrelated reorder of model_route_error_to_host_error / capability_resolve_error_to_host_error in loop_driver_host.rs could be split out; trivial.

Rollback posture is solid: text-only path preserved, no schema/state changes, reverting the cutover is "stop calling the new builder at the composition root."

[serrrfirat]

Summary

Reviewed WS17 PR #3653 only.
Base fb72b490bcb5056087cf5b6df2d55cfd34fd8d03 → head 5ddd72b77641d91eb2cb6a380d51d2ea1d83af85.

Readiness evidence improved, but current tests still do not prove production default-path live cutover.

Validation:

• cargo check -p ironclaw_product_workflow ✅

Findings

#	Sev	Category	File:Line	Issue	Fix suggestion
1	Medium	Evidence validity / Correctness	crates/ironclaw_product_workflow/tests/inbound_turn_contract.rs:775-790, crates/ironclaw_turns/src/coordinator.rs:176-181	Product cancellation evidence bypasses actual product path. Test calls coordinator.cancel_run(...), which only requests cancel in turn store, then manually flips the test cancellation factory via request_product_cancellation(...). Test would pass even if real product cancellation never reaches live run handle.	Add product-path cancel API/test where one external cancel action drives both turn-state cancel request and runtime cancellation observation. Remove manual factory backdoor from proof.
2	Medium	Canary / Test coverage	crates/ironclaw_reborn/src/runtime.rs:173-203, crates/ironclaw_product_workflow/tests/inbound_turn_contract.rs:716-730	Product-live readiness gate checks resolver/input/cancellation/identity, but current contract tests use EmptyCapabilityFactory and do not prove tool/capability execution plus product-visible result/evidence path. This leaves cutover evidence short of title/claim.	Add product-facing tool-use canary covering capability execution, result refs/evidence, checkpoint/progress plumbing, and product-visible outcome.

Security/data-flow notes

• No new auth/secret leak found.
• Main risk is false readiness signal, not direct exploit.

Correctness/invariant notes

• Cancellation and tool-result evidence must be caller-boundary, not helper-driven.

Missing tests

• Single product cancel action reaches retained runtime cancellation handle.
• Product-facing tool-use planned-loop contract with visible result/evidence.

[henrypark133]

Deferred from WS-16 review (PR #3652)

The WS-16 review surfaced production-readiness items that were explicitly out of scope for WS-16 and tagged for this PR. build_product_live_planned_runtime already fail-closes on model_route_resolver, input_queue, cancellation_factory (with probe), and identity_context_source, but the following remain unhandled here:

1. model_policy_guard fail-closed check — serrrfirat arch(ws-16): wire live planned runtime composition #3652 High. Host factory currently defaults to NoOpPolicyGuard. DefaultPlannedRuntimeParts has no field for it; live composition silently uses no-op. Either add a required field to DefaultPlannedRuntimeParts (or a product-live-only wrapper) and reject None in build_product_live_planned_runtime, or wire it through RebornLoopProductionInputs so the readiness gate catches it.
2. model_budget_accountant fail-closed check — serrrfirat arch(ws-16): wire live planned runtime composition #3652 High. Same shape as policy guard: defaults to NoOpBudgetAccountant. Needs an entry on the parts struct and a fail-closed branch.
3. safety_context fail-closed check — serrrfirat arch(ws-16): wire live planned runtime composition #3652 High. RebornLoopDriverHostFactory::safety_context defaults to None. Live composition must supply an InstructionSafetyContext; reject None in the product-live builder.
4. Production-readiness gate invocation from a startup entry point — zmanian arch(ws-16): wire live planned runtime composition #3652, issue Wire Reborn loop production readiness gate into startup composition #3602. The production_readiness module exists and RebornLoopProductionInputs / validate_reborn_loop_production_readiness are the intended contract, but no production entry point invokes them today. WS-17 owns this hookup per the brief; Wire Reborn loop production readiness gate into startup composition #3602 should close on the PR that wires it.

Suggested follow-up: extend ProductLiveRuntimeReadinessComponent with ModelPolicyGuard, ModelBudgetAccountant, SafetyContext variants and route those checks through verify_product_live_*_probe helpers analogous to the cancellation probe, then have the startup composition call validate_reborn_loop_production_readiness and gate on report.is_ready().

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b26455e8d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Wire a production run-profile resolver before coordinator build

turn_coordinator_for_production now hard-fails when run_profile_resolver is unset, but the production service assembly paths still never set one (crates/ironclaw_reborn_composition/src/factory.rs libsql path at lines 265-287 and postgres path at lines 327-349 only call .with_turn_run_wake_notifier(...) before turn_coordinator_for_production()). In production/migration profiles this now returns a missing-component wiring report and prevents coordinator construction, so bootstrapping the Reborn production facades fails even with valid DB/runtime wiring.

Useful? React with 👍 / 👎.