back all client modes with the daemon

[kevinjosethomas]

• routes interactive, print, json, rpc, piped input, and no-session clients through daemon-owned workers.
• preserves existing output protocols, commands, lifecycle behavior, and repeated rpc prompting.
• shares autonomous loops, heartbeats, refinements, subagents, rlm, and ipython across every client form.

---

Note

High Risk
All standard client modes now depend on daemon availability and supervisor/worker correctness; regressions affect session isolation, RPC wire compatibility, and headless lifecycle across a large surface area.

Overview
Interactive, print, JSON, RPC, piped stdin, and --no-session now run on the same supervisor/worker runtime instead of separate in-process or legacy owned-worker frontends, while keeping existing CLI commands, output shapes, and exit behavior.

Headless paths use client-owned daemon workers (createDaemonClientConnection + DaemonAgentConnection with ownedSession): print/JSON go through runPrintModeWithConnection, RPC through runRpcModeWithConnection, and completion uses shared waitForHeadlessCompletion / new daemon commands (prompt_and_wait, execute_bash_and_wait, wait_for_headless_completion, get_session_header, set_auto_retry). Workers can be completed, reconnected with launch env, promoted to resident when cron/heartbeat is added, and cleaned up after owner disconnect grace.

The daemon protocol gains client_owned_sessions, lifecycle on create, and supervisor routing that hides client-owned workers from global list/agents view unless owner-scoped. RPC docs add daemon-parity commands (schedules, heartbeats, agent messaging, observe/unobserve).

Startup: maybeStartDaemonEarly replaces interactive-only early launch (includes -p/print, longer 30s timeout); legacy owned-worker frontend is off by default with rollback when a stale busy daemon is detected. Shutdown confirmation counts busy client-owned sessions. Subprocess spawn uses createCliSubprocessEnv for tsx/tsconfig propagation.

^{Reviewed by Cursor Bugbot for commit a7ffbe7. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Route all CLI client modes (interactive, print, JSON, RPC) through the daemon

• All client modes now run via a daemon-backed AgentConnection abstraction instead of an in-process runtime, enabling session sharing, scheduling, and messaging across clients.
• Adds client_owned_sessions capability to the daemon protocol: clients can create private owned sessions that are cleaned up on disconnect (with a grace period) or explicitly promoted to resident sessions.
• Extends DaemonAgentConnection with new methods: promptAndWait, waitForHeadlessCompletion, executeBashAndWait, sendAgentMessage, heartbeat/cron management, and session observation.
• Extends the RPC wire protocol with commands for agent messaging, cron schedules, heartbeats, and observe/unobserve for live session events.
• runPrintModeWithConnection and runRpcModeWithConnection are introduced as new public entrypoints so any AgentConnection implementation can drive those modes.
• Adds createCliSubprocessEnv to propagate TSX_TSCONFIG_PATH into child processes when running under tsx.
• Risk: the RPC mode prompt handling now buffers connection events until the response is emitted, changing the ordering of events relative to the prompt response compared to prior behavior.

^{Macroscope summarized a7ffbe7.}

[linear]

ENG-4685 Daemon-back all Prime Agent client modes

Outcome

Make the daemon/supervisor the execution infrastructure for every Prime Agent CLI client form while preserving all existing public behavior and wire contracts.

Interactive, print, JSON, and RPC are client modes. The daemon is not a peer mode; it owns every root AgentSessionRuntime, kernel, subagent runtime, heartbeat, schedule, and live-session lease.

Interactive TUI ─┐
Print client ─────┤
JSON event client ├── AgentConnection / daemon protocol ── supervisor ── session worker
RPC client ───────┘                                                ├─ root runtime + kernel
                                                                  └─ subagent runtimes + kernels

Existing evals, scripts, subprocess clients, and SDK consumers must continue to work without changes. The migration must preserve CLI parsing, stdout/stderr, exit codes, JSONL/RPC framing and schemas, event ordering, prompt semantics, session selection, extension behavior, signal handling, and model-visible context.

Current state

• Normal persisted interactive startup creates or attaches a daemon session through DaemonAgentConnection.
• --print and --mode json create an invocation-local AgentSessionRuntime and run runPrintMode() directly.
• --mode rpc creates an invocation-local runtime and keeps it alive behind the stdin/stdout RPC dispatcher.
• Headless RLM children are inline children. They are not daemon-addressable and are disposed with the client process.
• Heartbeats, cron jobs, agent messaging, active-session observation, and the RLM heartbeat controller only exist on daemon-owned sessions.
• The CLI type model conflates client presentation with infrastructure: AppMode includes daemon, and --mode daemon is also used to launch the internal service.

Relevant foundations already exist:

• AgentConnection, DaemonAgentConnection, and the daemon JSONL protocol
• per-session daemon workers and supervisor recovery from ENG-4576 / PR isolate daemon sessions in worker processes #383
• active-session attach and resume handling from ENG-4656 / PR attach resume clients to active sessions #443
• daemon event sequencing, snapshots, extension UI relay, subagent ownership, heartbeats, and client capabilities

Design principles

1. One execution architecture. Normal CLI execution never constructs a root runtime in the client process after migration.
2. Clients only present and translate. Client modes parse input, render output, translate commands, and manage a lease; workers own execution.
3. Compatibility before parity. Existing invocations retain their exact observable contracts. New capabilities are additive.
4. No prompt drift. Moving execution must not silently add model-visible skills, system-prompt sections, or resource ordering to existing print/JSON/RPC invocations.
5. Explicit lifecycle. Resident interactive agents and client-owned headless sessions use different daemon leases rather than relying on process lifetime accidentally.
6. Transport envelopes stay private. Daemon hello/snapshot/replay/ack messages never leak into JSON or RPC stdout.
7. Preserve public APIs. Existing exported TypeScript entry points and accepted CLI values remain source-compatible.

Compatibility contract

Before changing routing, add characterization coverage for all externally observable behavior.

CLI and process behavior

• Preserve prime-agent, -p/--print, --mode text, --mode json, and --mode rpc semantics.
• Preserve automatic print selection for piped stdin and the current stdin + @file + first-message concatenation order.
• Preserve ordered handling of multiple positional prompts.
• Preserve --continue, --resume, --fork, --session-dir, --no-session, provider/model/thinking/tool/resource flags, extension flags, --offline, and autonomous options.
• Preserve current active-session conflict/attach behavior per mode; encode it as an explicit creation policy instead of letting daemon reuse change it accidentally.
• Preserve stdout cleanliness, stderr diagnostics, exit codes, SIGHUP/SIGTERM handling, abort behavior, and bounded shutdown.
• Preserve cold-start and warm-start behavior within an agreed regression budget.

Print contract

• Emit only the final assistant text to stdout, exactly as today.
• Preserve error/aborted handling and exit status.
• Preserve autonomous gate retries and their stderr diagnostics.
• Close the client-owned daemon session after output is flushed so ordinary print invocations do not leave resident agents behind.

JSON contract

• Emit the same session header first.
• Emit only the existing AgentSessionEvent JSON objects, one LF-delimited record per line.
• Preserve event schemas, ordering, multiplicity, and final flush behavior.
• Do not emit daemon protocol messages, attach snapshots, replay metadata, responses, or logs.
• Where daemon-owned subagents add internal identifiers, keep the legacy JSON projection stable unless a new explicitly versioned output field is additive and proven safe.

RPC contract

• Preserve strict LF-delimited stdin/stdout framing.
• Preserve every existing command and response schema.
• Preserve request ID correlation and prompt preflight timing: success means accepted/queued/handled, while post-acceptance failures remain events.
• Preserve event schemas and ordering, extension UI request/response behavior, stdin EOF shutdown, and concurrent prompt/steer/follow-up semantics.
• Add parity commands only additively; existing clients must not need to negotiate them.

Model/session contract

• Preserve the effective system prompt, skill list/order, extension/resource resolution, tool set/order, environment, cwd, model selection, thinking level, service tier, and transport for existing commands.
• Preserve session JSONL contents apart from unavoidable non-semantic IDs/timestamps that are normalized in tests.
• Preserve IPython namespace snapshot and restore behavior.
• Preserve current harness scope and refinement storage behavior.

Implementation plan

Phase 0 — Freeze behavior with a dual-backend compatibility harness

1. Add end-to-end fixtures using the faux provider for print, JSON, and RPC.
2. Capture normalized golden transcripts for stdout, stderr, exit status, session JSONL, JSON events, RPC responses/events, prompt ordering, and signal shutdown.
3. Run the same fixtures against an injectable legacy in-process backend and the new daemon backend during development.
4. Cover cold daemon, warm daemon, daemon restart/reconnect, version mismatch, worker crash, client disconnect, busy session, persisted session, and --no-session.
5. Add system-prompt/resource snapshots so daemon-only skills cannot appear accidentally.
6. Establish startup latency and memory baselines for eval-sensitive headless invocations.

Phase 1 — Separate client mode from service bootstrap

1. Replace internal AppMode = interactive | print | json | rpc | daemon reasoning with:
   • ClientMode = interactive | print | json | rpc
   • a separate internal service/runtime command for supervisor and worker startup
   • a separate output format where needed (text | json)
2. Move daemon supervisor/worker startup behind the existing private runtime command marker or a dedicated internal command path.
3. Keep --mode daemon accepted as a hidden compatibility shim because removing an accepted value or the exported Mode union would be breaking. Route it to the internal service command and mark it deprecated in code; do not describe it as a user mode.
4. Preserve the current --mode text behavior even though it is historically overloaded.
5. Update early daemon startup detection so every normal client mode can overlap daemon readiness with client preparation.

Phase 2 — Introduce shared daemon-backed client session bootstrap

Create one composition-root helper used by all four client modes, conceptually:

createDaemonClientSession({
  clientMode,
  sessionSelection,
  runtimeConfig,
  compatibilityProfile,
  lifecycle,
  capabilities,
})

It must:

1. Ensure/recover the correct supervisor socket.
2. Resolve cwd and session selection in the same order as today.
3. forward the complete runtime configuration and client environment to the worker without leaking secrets into logs.
4. Atomically create-or-select a session and attach the client before prompts can emit events.
5. Return an AgentConnection, initial snapshot/header, event cursor, and lease handle.
6. Support persisted and truly ephemeral (--no-session) daemon sessions.
7. Encode existing-owner policy explicitly: create, attach, or error according to the current mode/flag contract.
8. Clean up partially created workers when attach or client initialization fails.

Phase 3 — Add explicit daemon session leases

Daemon residency must not be inferred solely from socket disconnects.

• Resident lease: normal interactive agents remain daemon-owned after the UI disconnects, matching current behavior.
• Client-owned lease: print, JSON, and RPC sessions close when their owning client exits or stdin closes, matching current headless process lifetime.
• Ephemeral lease: --no-session gets no transcript file and is fully removed on close.
• Promotion: an explicitly new durable operation such as creating a heartbeat may promote a client-owned session to resident. Promotion is additive and must never happen for an unchanged legacy command by accident.
• Client death, supervisor restart, update restart, and reconnect need deterministic lease recovery/expiry rules.
• Client-owned headless sessions should not appear as abandoned agents after normal completion; any visibility while active must be intentional and tested.

Add protocol operations for graceful close/release and an idempotent cleanup path for disconnected clients.

Phase 4 — Provide a headless AgentConnection execution surface

Extend the connection/protocol with the minimum generic operations that headless clients currently obtain by reaching into AgentSessionRuntime:

• atomic event subscription at a known sequence boundary
• session header retrieval
• wait-for-idle / wait-for-terminal barrier
• last assistant message/text and stop reason
• autonomous status, gate refresh, and host continuation support
• session stats/state needed by existing RPC responses
• graceful abort, close, and lease release

Prefer moving autonomous terminal/gate coordination into the worker behind a generic terminal barrier so print does not reconstruct worker policy client-side. The worker remains the authority for queues, retries, compaction, subagents, and terminal evidence.

Phase 5 — Migrate print and JSON without changing their runners

1. Split the current print implementation into a transport-independent client runner plus compatibility wrappers.
2. Make the CLI instantiate the runner with the daemon-backed headless connection.
3. Keep the exported runPrintMode(runtimeHost, options) entry point working by adapting an in-process runtime for SDK/tests; do not break downstream TypeScript consumers.
4. Subscribe before the first prompt and suppress attach/snapshot transport noise.
5. Preserve ordered multiple-prompt execution.
6. Preserve text final-output extraction and JSON event projection exactly.
7. Preserve autonomous gate behavior via the worker terminal barrier.
8. On completion or failure, abort outstanding work as appropriate, close the client-owned lease, flush output, and return the legacy exit code.

Phase 6 — Migrate RPC behind AgentConnection

1. Extract the existing stdin/stdout dispatcher so it depends on an RPC session-client interface rather than AgentSessionRuntime.
2. Implement that interface with DaemonAgentConnection and the new headless operations.
3. Keep runRpcMode(runtimeHost) source-compatible as an in-process adapter for SDK consumers and unit tests; route the CLI through a new internal daemon-backed entry point.
4. Relay daemon extension UI requests through the unchanged RPC extension_ui_request / extension_ui_response protocol.
5. Preserve prompt preflight acknowledgements and asynchronous event delivery.
6. Make stdin EOF, shutdown commands, and signals release the client-owned session deterministically.
7. Ensure reconnect/replay does not duplicate RPC-visible events or responses.

Phase 7 — Add host-level feature parity additively

Once all clients use the daemon, extend RPC with optional commands mapping to existing AgentConnection capabilities:

• list/get/set/pause/resume/clear heartbeats
• list/add/cancel scheduled prompts
• agent messaging status/send/pause/resume/clear
• active-session and subagent observation/watch operations
• queue inspection/clear/abort-and-clear
• scoped models, service tier, transport, and missing session-tree operations
• any missing goal, refinement, compaction, retry, and resource snapshot controls

Do not change existing RPC commands or require a new handshake. Advertise optional capabilities through additive state/capability metadata if needed.

For print/JSON, parity means the same daemon-owned execution semantics, kernels, subagents, refinement store, and failure recovery—not an interactive management channel. Do not reinterpret existing prompt text such as /refine, because print currently treats unsupported built-in TUI commands as prompt content. New headless controls must use additive flags/subcommands or a separately versioned protocol.

Phase 8 — Move subagents to daemon ownership with compatibility projection

1. Root sessions created for print/JSON/RPC receive the daemon subagent runtime host.
2. Children run as supervised worker sessions and gain daemon observation/messaging internally.
3. Parent close cascades to active and retained children according to the client lease.
4. Preserve existing parent event and result semantics, usage attribution, child transcript paths, cancellation, and recursion limits.
5. Keep legacy JSON/RPC event payloads stable even if internal children now have an activeSessionId.
6. Do not add daemon-only model-facing orchestration skills to legacy headless prompts unless their previous absence is preserved by a compatibility profile or the change is separately versioned and eval-approved.

Phase 9 — Consolidate startup and remove the normal in-process CLI path

After parity and compatibility suites pass:

1. Route interactive—including --no-session—print, JSON, and RPC through shared daemon bootstrap.
2. Keep in-process construction only for SDK use, focused tests, and an internal temporary rollback switch.
3. Canary the daemon backend, switch it to the default, and retain a non-public kill switch for one release.
4. Remove the kill switch and all normal CLI in-process routing after production/eval validation.
5. Update README/docs to describe four client modes and the daemon execution architecture.

Verification plan

Existing suites to preserve

• test/print-mode.test.ts
• test/rpc.test.ts
• test/rpc-prompt-response-semantics.test.ts
• test/rpc-jsonl.test.ts
• test/rpc-client-refine.test.ts
• test/stdout-cleanliness.test.ts
• test/main-interactive-routing.test.ts
• test/agent-connection-daemon.test.ts
• daemon client/protocol/mode/supervisor/process/recovery suites

New integration matrix

Run each applicable case for print, JSON, and RPC:

• cold/warm daemon
• fresh/persisted/ephemeral session
• continue/resume/fork and active-owner conflict policy
• one and multiple prompts
• piped stdin and @file inputs
• text, image, tool calls, compaction, retry, refinement, goals, autonomous gates, and subagents
• extension commands and extension UI behavior
• model/thinking/tool/resource configuration
• client disconnect, SIGTERM, SIGHUP, abort, worker crash, supervisor replacement, and version mismatch
• daemon replay/reconnect without duplicate output
• stdout/stderr byte or normalized-golden comparison against the legacy backend
• session JSONL and system-prompt/resource snapshot comparison

Use the faux provider and test harness only; no real provider APIs or paid tokens.

After implementation, run every modified specific test file and npm run check. Run the established Prime Agent headless eval/smoke suite against both backends before switching the default.

Acceptance criteria

• Interactive, print, JSON, and RPC are the only conceptual client modes.
• Every normal CLI mode, including --no-session, is backed by the daemon/supervisor and a session worker.
• --mode daemon is no longer an internal client-mode branch, while its accepted legacy CLI/API surface remains compatible.
• Existing print stdout/stderr/exit behavior is unchanged.
• Existing JSON header, event schemas, ordering, and LF framing are unchanged.
• Existing RPC commands, responses, timing semantics, events, extension UI, and LF framing are unchanged.
• Existing evals and headless integrations require no configuration or code changes.
• Existing system prompts, model-visible skills/resources, tools, and session transcripts remain compatible.
• Headless root and subagent runtimes are daemon-supervised.
• Headless lifecycle cleanup leaves no orphan workers, sockets, leases, kernels, or temporary directories.
• RPC gains additive access to daemon heartbeats, schedules, messaging, observation, and other missing connection capabilities.
• Print/JSON retain finite-client semantics and do not silently reinterpret existing prompts.
• Characterization, daemon integration, failure injection, and parity tests pass.
• npm run check passes with no errors, warnings, or infos.
• README and architecture docs describe daemon as infrastructure, not a mode.

Non-goals

• Redesigning the public JSON or RPC protocols.
• Changing default session persistence or making ordinary print/JSON invocations resident.
• Reinterpreting existing slash-command-looking prompt text in print/JSON.
• Removing in-process SDK support.
• Changing provider/model/tool behavior unrelated to transport ownership.

Delivery

Implement as phased PRs under this issue. Land characterization and protocol primitives first, then print/JSON, then RPC/parity, and only then switch the default and remove normal CLI in-process routing.

Review in Linear

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4ad36b2. Configure here.}