Skip to content

feat(server): make session messages queue safely - #5

Merged
roughcoder merged 6 commits into
mainfrom
feat/session-send-modes
Aug 12, 2026
Merged

feat(server): make session messages queue safely#5
roughcoder merged 6 commits into
mainfrom
feat/session-send-modes

Conversation

@roughcoder

@roughcoder roughcoder commented Aug 12, 2026

Copy link
Copy Markdown

What

  • make send_to_session default to queue at the decider boundary
  • persist busy-child messages as FIFO queued turn starts and release one at safe provider session boundaries
  • recover stale persisted running sessions only after a decision-time provider liveness check inside the serialized worker
  • make release idempotent through queued IDs in the authoritative command read model
  • cancel queued work on stopped/error sessions instead of resurrecting children
  • bound interrupt fallback: cancel the replacement, stop the still-running session, and notify the parent when no boundary arrives
  • return delivery: immediate | queued | unknown; acknowledgement uncertainty after a committed send no longer reports failure
  • deduplicate interrupt timeout fibers by queued message and fail safely on invalid persisted deadlines
  • preserve graceful-stop notices as immediate in-turn provider steers while ordinary busy-session messages queue
  • document notify as a follow-up because provider steering behavior is not uniform

Why

Parent sessions frequently message children that are already mid-turn. Delivery must remain durable and live across provider failure, server restart, concurrent release signals, terminal session states, ignored interrupts, provider rebind windows, and graceful-stop deadlines.

Validation

Deferred

  • notify requires an explicit cross-provider steering capability/fallback contract
  • queued-row cleanup for deleted/archived threads and replacing the global recovery scan with a thread-indexed query remain persistence follow-ups
  • the web client currently ignores the queued event; a queued-delivery affordance is a separate web-app follow-up documented in the internals guide
  • no full-suite or browser validation was run

Generated by GPT-5.6-sol via the Phoenix Codex harness.

@roughcoder roughcoder left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: feat(server): make session messages queue safely

Verdict: REQUEST-CHANGES — the core design is better than I expected and several races I went looking for are genuinely closed. What blocks it is a set of liveness gaps around the release boundary: a queued message can be stranded indefinitely or delivered twice, and none of the concurrency behaviour is tested.

Read + reason only — per resource limits I ran no tests, no typecheck, no installs.


Verified correct (I tried to break these and couldn't)

Worth stating up front, because it changes how the rest should be read:

  • No lost-wakeup on the "message arrives as the turn ends" race. projectionPipeline.projectEvent runs inside the same SQL transaction as eventStore.append (OrchestrationEngine.ts:170-205), and dispatch is serialized through one queue against the same read model the decider reads. So if the decider sees running, the thread.session-set that flips it was not yet processed, and the queued row is committed before that event is published. The boundary cannot fire "between" the decision and the insert.
  • No duplicate release via the event worker. makeDrainableWorker is a single TxQueue.take + Effect.forever fiber (DrainableWorker.ts:47-56), and releaseNextQueuedTurn's dispatch returns only after the transaction that deletes the queued row commits. Sequential boundary events are safe.
  • Persistence is well-separated. state TEXT NOT NULL has no CHECK constraint (005_Projections.ts:81) so 'queued' inserts fine; UNIQUE (thread_id, turn_id) doesn't collide because SQLite treats NULLs as distinct, so >1 queued row per thread works; getPendingProjectionTurn and clearPendingProjectionTurnsByThread both filter state = 'pending', so queued rows are neither started early nor wiped by replacePendingTurnStart. listByThreadId gained state <> 'queued' so they stay out of the UI.
  • FIFO across restart holdsORDER BY requested_at ASC, row_id ASC (ProjectionTurns.ts:221), and row_id INTEGER PRIMARY KEY AUTOINCREMENT does exist (005_Projections.ts:76), so the tiebreak is stable.
  • delivery is accurate. readEvents is fromSequenceExclusive and dispatch returns lastSequence (OrchestrationEngine.ts:231), so readEvents(sequence - 1, 1) really does return the last event of the command. Queue → turn-start-queued, interrupt → turn-start-queued, idle → turn-start-requested. Interrupt against an idle child correctly skips the interrupt entirely and reports immediate.
  • Schema safety ✅ — mode is optional and typed; the new command is in InternalOrchestrationCommand so it isn't externally dispatchable; decider.ts:1513's command satisfies never is satisfied by the new case. The new event type doesn't break any exhaustive switch: applyThreadDetailEvent ends with a forward-compatible fallback (threadReducer.ts:618), and AgentAwarenessRelay.ts:91 / projector.ts:828 have default:.

The unverified post-typecheck fix — reads correct

SessionSpawnReactor.ts:196-203. Cause and Option are imported (lines 8, 13) and nowIso is defined at line 30, so nothing dangles. Effect.interrupt | Effect.logWarning collapses the channel to never, which matches the shape start needs. Layer.provide(ProjectionTurnRepositoryLive) at line 213 is the same pattern already used at ProviderRuntimeIngestion.ts:2071, so the SqlClient requirement bubbling to server.ts:247 is idiomatic here. I could not run typecheck, but I found no reason it would fail.

One inconsistency: this boundary uses Effect.interrupt while its sibling processEventSafely uses Effect.failCause(cause) (line 155). Effect.interrupt discards the original cause and re-interrupts with the current fiber id. Prefer failCause for consistency.


Blocking

1. [major] A session that dies without a terminal status strands its queue forever — SessionSpawnReactor.ts:131 + 186-191

Release is driven purely by thread.session-set transitions. If a provider crashes, or the server is killed mid-turn, the last persisted status stays running. No further boundary event ever arrives, so the queued message is never released — and the restart sweep is no help, because it explicitly skips threads whose status is starting/running (line 188-190). The parent received delivery: "queued" and waits indefinitely with nothing surfacing the stall. The recovery sweep is the natural place to fix this: a thread with queued rows and no live provider session is exactly the case it exists to rescue.

2. [major] interrupt mode has no fallback if the interrupt doesn't take — decider.ts:1028-1069

The decider emits thread.turn-interrupt-requested optimistically and queues the replacement behind it. Nothing verifies the provider honoured it, and there's no timeout. On any adapter where interrupt is unsupported, silently dropped, or fails, the session stays running and the queued replacement strands exactly as in #1 — while the tool has already told the parent the message is queued. Given the PR explicitly defers notify because "provider steering behavior is not uniform", the same non-uniformity applies to interrupt and deserves an explicit fallback contract.

3. [major] stopped and error are treated as safe release boundaries — SessionSpawnReactor.ts:131

The guard is "not starting and not running", so idle, ready, interrupted, stopped and error all release. Two consequences worth an explicit decision rather than falling out of the predicate:

  • A parent calls stop_session on a child with a queued message → the stop drives a terminal status → the queued message immediately starts a new turn, resurrecting the child the parent just stopped.
  • A child crashes → the reactor notifies the parent of the error (line 133-142) and restarts the child with the queued message.

4. [major] Startup TOCTOU between the recovery sweep and the event worker — SessionSpawnReactor.ts:168-195

forkParked starts the stream consumer, then the sweep runs inline on a different fiber. Both do read-listQueuedTurnStarts-then-dispatch, and that pair is not atomic across fibers (the intra-worker case is safe, as noted above). If a boundary event for a queued thread arrives while the sweep is walking, both can dispatch thread.turn.start.queued for the same messageId.

The duplicate is never harmlessly absorbed, because there is no idempotency guard: the decider's queued case (decider.ts:1070-1093) only checks that the message exists and is a user message — not whether it already started a turn. So the second command either emits a second turn-start-requested (same message, two turns) or, if the session is already starting, re-emits turn-start-queued, whose projection re-inserts the queued row for another round. A message.turnId === null check in the decider, or draining the sweep through the worker, would close this.

Non-blocking

5. [minor] One bad thread aborts the whole recovery sweep — SessionSpawnReactor.ts:180-195. Effect.forEach with concurrency: 1 and the catch outside it: a failure on the first thread skips every remaining thread's queued messages until the next restart. Catch per-thread instead.

6. [minor] Queued rows leak for dead threads — ProjectionTurns.ts:100-110. clearPendingProjectionTurnsByThread filters state = 'pending', so queued rows survive thread teardown, and listQueuedTurnStarts is an unfiltered global scan that returns them forever. Tolerated at runtime (getThreadShellById → none → void) but it grows unbounded.

7. [minor] Readback miss reports optimistic success — handlers.ts:59, 407-417. Stream.runHead returning Option.none maps to "immediate". A failed lookup should not resolve to the happy answer; prefer surfacing an error.

8. [minor] Queueing is gated on deliveryMode !== undefineddecider.ts:1028. Only the MCP handler sets it, so the human UI path keeps the old racy mid-turn behaviour. Reasonable scoping, but the safety property now lives in the caller rather than the contract; any future dispatcher silently opts out.

9. [nit] handlers.test.ts collides with PR #4, which adds the same new file (with a vite-plus/test import vs. @effect/vitest here). Guaranteed add/add conflict — worth sequencing with whoever merges first.

Test quality

decider.settled.test.ts:402-479 is genuine behavioural testing — it asserts exact event sequences for queue, interrupt (including turnId propagation), and release. Good.

The gap is that every finding above lives outside what's tested. The two helpers in handlers.test.ts (normalizeSendToSessionMode, deliveryFromAcknowledgedEventType) are near-tautological — one is ?? and the other a single ternary. Nothing covers the reactor release path, double-release, stranding on a stuck running session, the enqueue/delete SQL, FIFO with more than one queued message, or restart recovery. For a change whose entire value proposition is concurrency safety, the concurrency behaviour is the part with no coverage.

Not verified

No test run, no typecheck, no lint, no live multi-session exercise. In particular the typecheck status of the post-fix reactor is reasoned from imports and an existing layer precedent, not confirmed; and findings #1-#4 are derived from reading the fiber/transaction structure rather than reproduced.

@roughcoder

Copy link
Copy Markdown
Author

Fixed in fbe88cc8b.

Fixed:

  • stale persisted starting/running queues now recover on startup and periodic sweeps only after ProviderService.listSessions() confirms no live binding (with a staleness grace)
  • interrupt delivery has a durable 30s deadline; timeout cancels the replacement, requests session stop, and notifies the parent
  • stopped/error cancel all queued rows and notify instead of releasing/restarting
  • queued IDs/modes are hydrated into the authoritative command read model; release requires the queued ID, busy release preserves FIFO, and all boundary/recovery work is serialized through one worker
  • recovery failures are isolated per worker input
  • missing/unexpected delivery acknowledgement now fails instead of reporting immediate
  • omitted deliveryMode defaults to queue in the decider
  • helper-only tests were replaced and reactor coverage now exercises release-on-boundary, terminal cancellation, double-release serialization, stale-running recovery, and interrupt timeout

Deferred as requested:

  • deleted/archived queued-row teardown and replacing the global recovery scan with a thread-indexed query
  • provider-neutral notify capability

Validation: targeted command covering five touched suites passes 50/50. The one permitted Node 24 typecheck found four localized test/recovery typing issues; those exact diagnostics were fixed, and the 50 targeted tests pass afterward. Per the shared-machine rule, typecheck was not run twice.

@codex review

@roughcoder roughcoder left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta re-review — fbe88cc8b

Verdict: APPROVE-WITH-NITS. All four of my findings are resolved, and #4 in particular is fixed properly rather than papered over. One residual race is worth closing before this carries production traffic; everything else is nits.

Delta only (f8cd73bc7..fbe88cc8b). Read + reason, no runs.

Findings status

1. [major] Stranded queue — RESOLVED (with a caveat, below). Recovery now runs at startup and every 30s, and force-recovers a persisted starting/running session only when it has no live provider binding and its updatedAt is stale.

2. [major] Interrupt fallback — RESOLVED. Bounded at 30s from the queue entry's requestedAt, then cancel + stop + notify. Old entries compute remaining = 0 and fire on the first sweep, so a stuck interrupt that predates a restart is handled rather than waiting a fresh 30s.

3. [major] Terminal states — RESOLVED. Release is now restricted to idle/ready/interrupted; stopped/error cancel instead. No resurrection.

On your notification question: the parent can distinguish all three outcomes. Interrupted-and-replaced is silent (the message simply arrives), terminal cancellation says "N queued messages were cancelled because the spawned session entered stopped/error state", and timeout says "Interrupt delivery timed out; the queued replacement was cancelled and the spawned session was stopped." The ordering is right, too: the timeout handler cancels before stopping, so the resulting stopped event finds an empty queue and cancelTerminalQueue returns 0 without emitting a second, contradictory notification.

4. [major] TOCTOU / idempotency — RESOLVED, and this is the strongest part of the delta. Three independent mechanisms now back each other:

  • Serialization covers both paths. The sweep no longer releases directly; it enqueues {type:"recover"} into the same single-fiber worker as the stream consumer, and the interrupt timer enqueues {type:"interrupt-timeout"} there too. All three entry points are serialized. This is exactly the pairing I flagged.
  • Real idempotency at the decider. queuedTurnStarts is hydrated into the authoritative command read model and maintained by projectEvent (added on queued, removed on requested/cancelled), so a duplicate release finds no entry and fails as an invariant error instead of starting a second turn.
  • No more re-queue loop. The busy branch of thread.turn.start.queued now errors rather than re-emitting turn-start-queued, so a duplicate can't reinsert the row for another round — the failure mode I was most worried about.

5. [minor] Sweep error isolation — RESOLVED. Per-thread work moved inside the worker, where processInputSafely catches each input independently. The remaining catch wraps only the two fetches, where aborting is correct.

7. [minor] Optimistic delivery ack — RESOLVED, with a trade-off worth naming (below).

8. [minor] Contract-level queue default — RESOLVED. command.deliveryMode ?? "queue" in the decider, so the safety property no longer depends on the caller.

6. [minor] Queued-row teardown — DEFERRED, acknowledged in the reply.

Residual

[major] The liveness check is a snapshot, and staleness doesn't protect long turns. live is computed in enqueueRecovery (SessionSpawnReactor.ts:362) and consumed later inside the worker (:370), so it can be arbitrarily old by the time it's acted on — the queue may hold other work, and each recover dispatches commands synchronously through commit.

That matters more than it looks because the second guard doesn't carry much weight: stale (:261) compares session.updatedAt against 30s, but updatedAt only moves on session status transitions, so any turn longer than 30 seconds — i.e. most real agent turns — is permanently "stale". Liveness is therefore load-bearing on its own. If listSessions() runs during a provider rebind (crash-restart, adapter reconnect) the session is briefly absent, live: false sticks, and the sweep force-sets interrupted with activeTurnId: null and releases a queued turn while the original turn is still running — the double-running-turn this PR exists to prevent.

Two cheap fixes, either sufficient: re-check providerService.listSessions() inside the worker at decision time (it's in-memory and Effect<…, never>, so it costs nothing and closes the snapshot window entirely), or require two consecutive not-live observations before recovering, which is the standard failure-detector shape and makes the rebind window a non-event.

[minor] Strict ack failure re-introduces "fail after the side effect". resolveSendToSessionDelivery now fails on a missing/unexpected ack — correct in that it no longer reports the optimistic answer — but the dispatch has already happened. A transient event-store read hiccup surfaces as an error for a message that really is queued, and a parent that retries creates a duplicate (the retry carries a fresh messageId, so command-receipt dedup won't catch it). Consider carrying the threadId in that error, or a third delivery value, so the parent knows the send landed.

[minor] The contract-level default now changes the human UI path. Any thread.turn.start to a busy thread queues, including sends from the app. That's the safer behaviour and it's what I asked for, but no client affordance shows "queued" — the client reducer ignores thread.turn-start-queued via its forward-compatible fallback. A user typing into a busy thread now sees their message appear and nothing happen, with no explanation, until the boundary. Worth a follow-up on the client.

[nit] Timeout fibers are re-forked every sweep. recover forks scheduleInterruptTimeout per queued entry (:256) on every 30s pass, with no dedupe by (threadId, messageId). Self-limiting in practice — the first firing cancels the entry — but an entry that survives an early-return path accumulates a sleeper per sweep.

[nit] Date.parse NaN at :224 flows into Duration.millis(NaN). Unreachable while requestedAt comes from IsoDateTime, and the sibling staleness check fails safe on NaN; noting it only because the two behave differently.

[nit] Benign release races now log warnings. If the session goes busy between the reactor's projection read and the decider's read model, the release fails as an invariant error and logs at warning level each time. Correct outcome, noisy signal.

On the four unverified typing fixes

I checked the semantics rather than assuming the gate will catch them, and found no problems:

  • listSessions() exists with no arguments and no error channel, and ProviderSession.threadId is a required ThreadId, so .map(entry => entry.threadId) is sound.
  • The new ProviderService requirement bubbles out of SessionSpawnReactorLive (only ProjectionTurnRepositoryLive is provided), but it joins ReactorLayerLive alongside ProviderCommandReactorLive and ProviderRuntimeIngestionLive, which already require it — so the composition at server.ts:247 shouldn't shift.
  • mode is threaded consistently end-to-end: persisted as a distinct interrupting state, and every query (enqueue, delete, listQueued, and the listByThreadId exclusion) was updated to IN ('queued','interrupting'). No half-updated predicate.
  • ThreadTurnStartQueuedPayload.mode carries withDecodingDefault("queue"), so already-persisted events decode.

The one I'd still watch at the gate: Effect.forkScoped inside processInput requires Scope in the worker's R. makeDrainableWorker propagates R and the layer build provides a scope, so it should hold — but it's the least locally obvious of the four.

Tests

TestClock, not real sleeps — TestClock.setTime drives both the stale-recovery and interrupt-timeout cases, so the suite stays fast and deterministic. The assertions are real: exactly one thread.turn.start.queued when a boundary event races startup recovery (a direct guard on finding 4), the FIFO head asserted by messageId, zero releases and two cancels for both stopped and error, and the timeout case pinned to the timeout path by setting live: true so it can't be satisfied by the stale-recovery branch. The double-release test is a structural regression guard rather than a concurrency stress test, which is the right bar given the design now makes the property structural.

@roughcoder

Copy link
Copy Markdown
Author

Fixed in 9496f002e.

Fixed:

  • recovery now re-checks ProviderService.listSessions() inside the serialized worker immediately before deciding whether a persisted running session is stale, eliminating the queued liveness snapshot/rebind race
  • post-dispatch acknowledgement uncertainty returns delivery: "unknown" with the committed threadId instead of failing and inviting a duplicate retry
  • interrupt timeout scheduling is deduplicated by (threadId, messageId)
  • invalid persisted interrupt deadlines fail safely and log instead of reaching Duration.millis(NaN)
  • a reactor regression test proves a stale persisted session with a live provider binding is not force-recovered

Deferred and documented in the PR description/internals guide:

  • queued human-message UI affordance, which is a web-app change outside this PR
  • queued-row teardown/thread-indexed recovery scan and provider-neutral notify

Validation:

  • targeted tests: 3 files, 21 tests passed, exit code 0
  • the single permitted npm run typecheck exited 1 with two exact localized diagnostics (implicit test parameter type; recovery worker error channel); both sites were fixed afterward
  • typecheck was not rerun due the explicit once-per-session resource constraint
  • git diff --check passed

@codex review

@roughcoder roughcoder left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta re-review — 9496f002e

Verdict: APPROVE — merge-ready from my side. Both residuals are closed properly, and I judge the remaining liveness risk acceptable. Only nits left.

Delta only (fbe88cc8b..9496f002e). Read + reason, no runs.

[major] Liveness snapshot — RESOLVED, and the race is genuinely closed

live is now computed inside processInput's recover branch, on the serialized worker, immediately before the decision — and the live field is gone from WorkerInput entirely, so there's no vestigial path still carrying a stale value. The window between observing !live and dispatching the recovery is now the dispatch itself rather than however long the input sat in the queue.

On your specific question — yes, the recheck hits the same structure the rebind window affects. ProviderService.listSessions() aggregates each subscribed adapter's own session list, which is exactly what a rebind mutates. So this is a true recheck, not a different source that would paper over the problem.

Two-phase observation does not exist; one recheck is deemed sufficient. I think that's the right call here, for three reasons:

  1. Two independent conditions must both hold — no live binding and no session status write for 30s. If a rebind writes a starting status, updatedAt refreshes, stale goes false, and the session is protected regardless of the directory.
  2. For a healthy session to be falsely recovered, the adapter directory would have to transiently lose an entry for a live session with no accompanying status transition. That's a much narrower window than the one you had before.
  3. If the process genuinely died silently, recovery is the correct action — and two-phase would delay every genuine recovery by a full 30s interval, which is the primary case this exists for.

The residual therefore rests on one assumption: absence from the directory means genuinely no binding, not a transient gap. That's worth a one-line comment at the recheck, because if that property ever stops holding, two-phase observation is the fix and nothing else in the code would signal it.

[minor] Ambiguous ack — RESOLVED, and "unknown" is a real enum value

SendToSessionResult.delivery is Schema.Literals(["immediate", "queued", "unknown"]) — declared in the contract, not a stringly escape, so it encodes into the tool's JSON Schema as a third valid variant. The readback failure is caught (Effect.catchOption.none()), so a transient event-store hiccup no longer reports failure for a message that actually committed. The tool description documents when unknown appears. This is the right resolution of the trade-off I raised.

[nit] fixes — both RESOLVED

Timeout fibers are deduped by ${threadId}:${messageId} with check-and-add on the serialized worker (so no check-then-act race) and Effect.ensuring cleanup that also covers interruption. Date.parse NaN now fails safe at both sites — the timeout path logs and declines to schedule instead of sleeping on Duration.millis(NaN), and the staleness path treats an unparseable updatedAt as not-stale, which is the safe direction (it declines to force-interrupt).

The two post-typecheck fixes

Per the amended process rule, re-running typecheck to verify a fix is now allowed and expected — please do that rather than leaving the final state unverified. My read of the semantics found no problems:

  • Effect.failCause(cause)Effect.interrupt in processInputSafely. Sound: the branch only runs under Cause.hasInterruptsOnly, so the discarded cause contained nothing but interrupts and no diagnostic value is lost. It also makes this consistent with the sweep's catch, which already used Effect.interrupt. Presumably done to pin the error channel to never for DrainableWorker<WorkerInput>. Behaviourally unchanged: either form kills the worker loop, and only shutdown reaches it.
  • Parameters<typeof scheduleInterruptTimeout>[0] in forkInterruptTimeout is the one I'd eyeball first — that's an exotic type query over an Effect.fn callable, and it's the kind of expression that resolves differently than you'd expect. The shape it needs to accept (listQueuedTurnStarts entries: threadId, messageId, mode, requestedAt) does match. Second place to check is Effect.catch(() => Effect.succeed(Option.none())) in handlers.ts, where a bare Option.none() can infer Option<never>.

Nits

  • resolveSendToSessionDelivery now returns Effect.succeed on every path — it can be a plain function, dropping the yield* at the call site.
  • The dedupe key is added in forkInterruptTimeout but removed in two other places (scheduleInterruptTimeout's NaN branch and the ensuring). Correct, but the add/remove responsibility is spread across three sites; centralising the removal in ensuring alone would be easier to keep right.
  • mode !== "interrupt" is checked in both forkInterruptTimeout and scheduleInterruptTimeout. Harmless.

Tests

The new case is the right one: stale + running + live: true asserts no thread.session.set, no release, and the queued row retained — the exact false-recovery guard, and the inverse of the existing stale-recovery test, so both directions are now pinned. Still TestClock throughout.

Remaining from earlier, unchanged and non-blocking: queued-row teardown is deferred, and the contract-level queue default still has no client affordance surfacing "queued" to a user typing into a busy thread.

@roughcoder

Copy link
Copy Markdown
Author

Closing items fixed in e8c7555f9:

  • added the requested recovery invariant comment: provider-directory absence is assumed to mean no binding, not a transient rebind gap
  • changed the recovery sweep's interrupt-only catch branch to Effect.interrupt, which keeps the start effect's error channel at never

Verification:

  • npm run typecheck with Node v24.13.1: exit code 0
  • pnpm exec vp test run apps/server/src/orchestration/Layers/SessionSpawnReactor.test.ts: 6 passed, exit code 0
  • git diff --check: passed

The reviewer-flagged Parameters<typeof scheduleInterruptTimeout>[0] and Option.none() inference sites both compile successfully.

Persist busy-child deliveries as FIFO queued turn starts and release them at provider session boundaries. Interrupt mode requests the existing provider interrupt before the queued replacement is released.

Constraint: send_to_session must remain event-sourced and provider-neutral.

Rejected: provider-level notify steering | adapter semantics are inconsistent and need an explicit capability contract.

Confidence: high

Scope-risk: moderate

Directive: add notify only after every provider has an explicit steering or fallback contract.

Tested: pnpm exec vp test run apps/server/src/mcp/toolkits/sessions/handlers.test.ts apps/server/src/orchestration/decider.settled.test.ts (15 passed); npm run typecheck under Node 24 found one startup error-channel leak, fixed afterward.

Not-tested: full typecheck was not rerun due the session's one-run resource constraint; no browser or full-suite validation.
Make queued delivery release idempotent, recover stale persisted sessions only after checking live provider bindings, cancel terminal queues, and bound interrupt fallback with parent-visible cancellation.

Constraint: queued delivery must remain event-sourced and safe across restart, provider failure, and concurrent release signals.

Rejected: unconditional stop-then-send fallback | provider exit can race terminal queue cancellation and resurrect stopped work.

Confidence: high

Scope-risk: moderate

Directive: preserve queued IDs in the command read model when changing turn projections.

Tested: pnpm exec vp test run apps/server/src/orchestration/Layers/SessionSpawnReactor.test.ts apps/server/src/orchestration/decider.settled.test.ts apps/server/src/mcp/toolkits/sessions/handlers.test.ts apps/server/src/orchestration/projector.test.ts apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.test.ts (50 passed); npm run typecheck under Node 24 found four localized typing issues, fixed afterward.

Not-tested: typecheck was not rerun due the one-run resource constraint; no full suite or browser validation.
Move provider liveness observation into the serialized recovery decision and make post-dispatch acknowledgement uncertainty explicit.

Constraint: Shared-machine validation permits one typecheck run and targeted tests only.
Rejected: Two-observation failure detection | a decision-time in-memory provider check closes the stale snapshot window directly.
Confidence: high
Scope-risk: narrow
Directive: Keep provider liveness checks inside serialized recovery decisions.
Tested: pnpm exec vp test run apps/server/src/orchestration/Layers/SessionSpawnReactor.test.ts apps/server/src/mcp/toolkits/sessions/handlers.test.ts apps/server/src/orchestration/decider.settled.test.ts (21 passed, exit 0).
Not-tested: npm run typecheck was not rerun after fixing its two exact diagnostics; its single permitted run exited 1.
Record the provider-directory assumption at the recovery decision and keep recovery sweep interruption out of the error channel.

Constraint: Recovery may force-release only when directory absence is authoritative.
Rejected: Optional cleanup refactors | closing the reviewed invariant and compiler issue keeps this commit narrow.
Confidence: high
Scope-risk: narrow
Directive: Revisit the recovery failure detector if provider directory absence can become transient during rebind.
Tested: Node 24 npm run typecheck (exit 0); targeted SessionSpawnReactor test (6 passed, exit 0); git diff --check.
Not-tested: Full test suite and browser validation were not run.
Keep grace-stop notices immediate while ordinary busy-session messages queue, and make provider reactor fixtures model a completed turn before follow-up commands.

Constraint: Rebase must preserve graceful-stop deadlines and queued session delivery semantics.
Rejected: Queueing grace notices | it can hide the stop deadline until the turn it must stop has already ended.
Confidence: high
Scope-risk: narrow
Directive: Internal in-turn steering commands must opt out explicitly if future delivery defaults queue busy sessions.
Tested: Node 24 npm run typecheck (exit 0); targeted provider/handlers tests (53 passed, exit 0); targeted decider/session reactor/snapshot tests (39 passed, exit 0); git diff --check.
Not-tested: Full repository test suite and browser validation were not run.
@roughcoder
roughcoder force-pushed the feat/session-send-modes branch from e8c7555 to f78ddd6 Compare August 12, 2026 05:48
@roughcoder

Copy link
Copy Markdown
Author

Rebased onto current origin/main and force-pushed with lease at f78ddd602.

Conflict resolution preserved the landed structured reports, checkout support, and graceful-stop session fields/flows. This branch adds no migration, so no 045 renumbering was needed.

Grace-stop interaction:

  • graceStopNotice now explicitly bypasses the busy-session queue and emits thread.turn-start-requested immediately. This preserves feat: let spawned sessions stop gracefully #6's intended provider steer: the child learns the stop deadline during its current turn instead of only after the safe boundary.
  • ordinary user/parent messages still default to FIFO queueing while a session is busy.
  • when feat: let spawned sessions stop gracefully #6's deadline hard-stops the session, its thread.session-set with status: "stopped" reaches SessionSpawnReactor, whose terminal path cancels queued rows rather than releasing or resurrecting them.
  • landed provider-reactor follow-up fixtures now project a real ready boundary before issuing a second normal turn, matching the new queue contract.

Verification:

  • Node v24.13.1 npm run typecheck: exit code 0
  • provider reactor + MCP handlers: 53 passed, exit code 0
  • decider + session reactor + ProjectionSnapshotQuery: 39 passed, exit code 0
  • git diff --check: passed

Exercise conflicting provider turn acceptance through the intentional immediate grace-notice steer now that ordinary busy-session messages queue by default.

Constraint: The ingestion test must create a pending turn start before asserting conflicting turn acceptance.
Rejected: Restoring implicit busy-session steering | it would violate send_to_session's queue-by-default contract.
Confidence: high
Scope-risk: narrow
Directive: Pending-turn ingestion tests must use an explicit immediate-start path when the thread is already running.
Tested: Node 24 npm run typecheck (exit 0); targeted ProviderRuntimeIngestion, ProviderCommandReactor, SessionSpawnReactor, and decider suites (112 passed, exit 0); single regression test (exit 0); git diff --check.
Not-tested: Full repository suite and browser validation were not rerun locally.
@roughcoder

Copy link
Copy Markdown
Author

Gate failure diagnosed and resolved in 58dba81fc.

Root cause/event flow:

  1. The test projected an existing running session with activeTurnId = turn-steered-over.
  2. It then dispatched an ordinary thread.turn.start with no delivery mode.
  3. Queue-by-default correctly emitted thread.message-sent → thread.turn-start-queued; ProjectionTurns therefore contained a queued row, not a pending turn-start row.
  4. When the provider emitted conflicting turn.started(turn-from-steer), ProviderRuntimeIngestion read getPendingTurnStartByThreadId = none. Its stale-turn guard correctly rejected that conflicting lifecycle event, so the test's poll for activeTurnId = turn-from-steer timed out.

The production behavior is intentional and unchanged: ordinary messages must not implicitly steer a busy provider anymore. The obsolete test setup now uses graceStopNotice: true, the explicit internal immediate-steer path introduced/preserved for graceful stop. That path emits thread.turn-start-requested, creates the pending row, and still proves the ingestion invariant: a provider-expected conflicting turn.started replaces the superseded active turn.

Verification:

  • single reproduced regression after correction: 1 passed, exit 0
  • full targeted ProviderRuntimeIngestion.test.ts, ProviderCommandReactor.test.ts, SessionSpawnReactor.test.ts, and decider.settled.test.ts: 112 passed, exit 0
  • Node v24.13.1 npm run typecheck: exit 0
  • git diff --check: passed

@roughcoder
roughcoder merged commit 46579a2 into main Aug 12, 2026
6 of 8 checks passed
roughcoder added a commit that referenced this pull request Aug 12, 2026
#5/#6/#9/#10

Rebased onto main after #4/#5/#6/#9/#10 merged; carries the reconciliation:

- Envelope delivery applies to agent AND system-origin synthetic reports
  (same report-posted path); formatReportMessage keeps #10's origin-aware
  lead and delivers >1KB summaries as an envelope with a read_report hint.
- SessionReportEnvelope carries #9's structured data compactly:
  recommendation and completionPercent whole, findings/validation as
  counts; and #10's origin so a synthesized epitaph is visible at a
  glance. read_report returns the full findings/validation arrays with
  every page (bounded by the 32KB structured cap) plus origin.
- findByReportId now selects abstract, structured_json, and origin and
  maps through the shared mapReportRow, so no column can be silently
  dropped by one read path; the ProjectionSnapshotQuery Struct.pick
  allowlist gained abstract.
- Migration renumbered 043 -> 046 (043 structured, 044 stop audit, 045
  origin landed first).
- read_report docs record UTF-16 code-unit paging (pages can run one unit
  short or long at surrogate boundaries) and the unguessable-UUID timing
  assumption behind the reportId-only lookup path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
roughcoder added a commit that referenced this pull request Aug 12, 2026
… sibling access (#8)

Rebased onto main after #4/#5/#6/#9/#10 merged; carries the reconciliation:

- Envelope delivery applies to agent AND system-origin synthetic reports
  (same report-posted path); formatReportMessage keeps #10's origin-aware
  lead and delivers >1KB summaries as an envelope with a read_report hint.
- SessionReportEnvelope carries #9's structured data compactly:
  recommendation and completionPercent whole, findings/validation as
  counts; and #10's origin so a synthesized epitaph is visible at a
  glance. read_report returns the full findings/validation arrays with
  every page (bounded by the 32KB structured cap) plus origin.
- findByReportId now selects abstract, structured_json, and origin and
  maps through the shared mapReportRow, so no column can be silently
  dropped by one read path; the ProjectionSnapshotQuery Struct.pick
  allowlist gained abstract.
- Migration renumbered 043 -> 046 (043 structured, 044 stop audit, 045
  origin landed first).
- read_report docs record UTF-16 code-unit paging (pages can run one unit
  short or long at surrogate boundaries) and the unguessable-UUID timing
  assumption behind the reportId-only lookup path.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@roughcoder
roughcoder deleted the feat/session-send-modes branch August 12, 2026 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant