Skip to content

fix(coding-agent): recover from interactive-engine death and stop Escape from killing the engine - #2076

Merged
flora131 merged 15 commits into
mainfrom
fix/interactive-engine-recovery
Jul 31, 2026
Merged

fix(coding-agent): recover from interactive-engine death and stop Escape from killing the engine#2076
flora131 merged 15 commits into
mainfrom
fix/interactive-engine-recovery

Conversation

@flora131

@flora131 flora131 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Review did not converge. This automated run exhausted its budget after 10 review iterations without unanimous reviewer approval. One reviewer approved; the other rejected the patch with three blocking findings that remain open (reproduced verbatim below). A human must decide whether to continue repair on this branch or restart with a narrower scope. This PR is a draft, is not approved, and no reviewers are requested.

Unresolved review findings

Reproduced from the final review-round artifact (review-round-latest.json, convergence_decision.approved: false). Nothing here has been resolved.

1. [P2] Satisfy the literal 500-line cap for every touched file — BLOCKING

  • Priority: 2
  • Objective alignment: required_by_objective
  • Confidence: 1.0
  • Reviewer: reviewer-b
  • Location: packages/coding-agent/CHANGELOG.md:44

The required changelog edit leaves this touched file at 6,164 physical lines. Independent wc -l also found packages/coding-agent/docs/extensions.md at 2,853, packages/workflows/CHANGELOG.md at 1,422, and packages/mcp/CHANGELOG.md at 809. bun run check:file-length passes because it checks source-like files only, but the objective says every touched file. This needs either a compliant split or a user amendment.

This is the finding the implementer restated as unresolved in six consecutive iterations (5 through 10). Their position, recorded in the notes: the same launch contract requires editing CHANGELOG.md and docs/extensions.md, both of which were already over 500 lines before this work started, and released changelog sections are immutable — so "every touched file ≤ 500 lines" and "update these two files" cannot both hold literally. They implemented the narrowest reading (the repository's bun run check:file-length gate, source-like files only, green at 2,554 files) and flagged that the literal wording needs a user decision. No supervisor or user was reachable via intercom during the run.

2. [P2] Do not restore admitted work based on plain error text — BLOCKING

  • Priority: 2
  • Objective alignment: consistent_with_objective
  • Confidence: 0.96
  • Reviewer: reviewer-b
  • Location: packages/coding-agent/src/modes/interactive/interactive-prompt-restore.ts:30-32

The legacy substring fallback classifies any untyped provider, RPC-response, or extension error containing Agent process stopped or another marker as an unsent transport failure. Such errors can occur after the engine admitted and ran the request, but restoreUnsentPromptDraft then restores the draft and suppresses the error, inviting duplicate side effects. The direct Bun probe produced {classifiedAsSendFailure:true, restored:true} for a plain provider error with this text. Production transport failures already carry the typed marker, so admission must remain decisive instead of falling back to message text for production responses.

3. [P2] Clear the dead generation's watchdog latch before recovery — BLOCKING

  • Priority: 2
  • Objective alignment: consistent_with_objective
  • Confidence: 0.94
  • Reviewer: reviewer-b
  • Location: packages/coding-agent/src/modes/interactive-engine/engine-health.ts:143-145

A watchdog diagnostic latches unresponsive, but an unexpected generation end clears activity and starts recovery without clearing that latch. During the fresh replacement's pre-ready window, needsExplicitTermination() therefore remains true and the first Ctrl+C stops the new child immediately, despite the documented fresh-replacement fence. The direct controller probe printed {"restartCalls":1,"stopCalls":0,"armedDuringFreshReplacement":true}. Clear generation-owned health state at the death boundary while retaining the separate failed-replacement and overdue-attempt gates.

The approving reviewer (reviewer-a) evaluated this claim independently and declined to treat it as blocking, arguing the stale latch leaves needsExplicitTermination() true only during the restart window and the worst case is one extra user-requested restart. The two reviewers disagree; the finding is recorded as unresolved.

4. [P3] Comment claims an inherited legacy engine variable cannot select engine mode, but it still does — non-blocking

  • Priority: 3
  • Objective alignment: consistent_with_objective
  • Confidence: 0.9
  • Reviewer: reviewer-a
  • Location: packages/coding-agent/src/utils/interactive-engine-env.ts:9-10

The new module header states that the legacy interactive-engine variables "are still recognized and scrubbed as defense in depth, so an inherited or caller-supplied value can neither select engine mode nor reach a child." The second half holds, but the first does not: captureInteractiveEngineStartupEnv falls back to process.env.ATOMIC_INTERACTIVE_ENGINE_CHILD (line 63) and isInteractiveEngineChild() returns true from that fallback, so an inherited ATOMIC_INTERACTIVE_ENGINE_CHILD=1 still flips the process into RPC engine-child mode. I reproduced this against the built dist: launching bun packages/coding-agent/dist/cli.js -e ./freeze-test.ts from a shell that had inherited ATOMIC_INTERACTIVE_ENGINE_CHILD=1 printed a stream of {"type":"engine_heartbeat"...} instead of starting the TUI, and the same launch with env -u ATOMIC_INTERACTIVE_ENGINE_CHILD ... started normally. The runtime behavior is identical to the pre-patch baseline so this is not a regression, but the new comment describes an invariant on a security-adjacent boundary that the code does not enforce and could mislead a maintainer into treating the legacy fallback as inert.

Reviewer split

Reviewer Verdict Findings
reviewer-a patch is correct (confidence 0.9) 1 non-blocking P3
reviewer-b patch is incorrect (confidence 0.98) 3 blocking P2

reviewer-b marked three requirements contradicted in traceability: verbatim draft restoration on send failure, the every-touched-file 500-line cap, and state invariants across the recovering/replacement-ready transitions.

Follow-up: the three unresolved findings are addressed (435f68f)

All three findings above are resolved. bun run test:unit 4569 pass / 1 skip / 0 fail,
bun run typecheck, bun run lint, and bun run check:file-length (2,557 files) all clean.

  1. [P2] 500-line cap on every touched file — resolved as a contract defect, no code change.
    The instruction that produced this finding was mine and it was wrong: it demanded a changelog
    and docs update while also requiring every touched file to stay under 500 physical lines. The
    repo's own gate covers source-like files by design (.ts, .tsx, .js, .jsx, .mjs,
    .cjs, .rs), and check:file-length passes. Markdown was never in scope. The reviewer was
    correctly enforcing bad wording for six iterations.

  2. [P2] Text-based classification restores admitted work — fixed.
    isEngineSendFailure now decides on admission alone. Every rejection RpcClient raises is
    built through rpcTransportError, and an accepted request is re-marked per request, so the
    legacy marker list was dead weight in production and a live hazard: any provider, extension, or
    command error quoting a phrase like Agent process stopped after the engine had already run the
    submission would restore the draft and suppress the real error. Tests that relied on the
    fallback now inject typed transport errors, which is what production raises. Removing it
    surfaced 12 tests in interactive-submit-send-failure.test.ts that passed only because of it.
    A new case pins that identical wording resolves differently based only on admission.

  3. [P2] Stale watchdog latch across generation death — fixed.
    recover() clears the unresponsive latch when a replacement attempt starts. The verdict
    belonged to the generation being replaced, and a fresh child emits no heartbeat before
    engine_ready, so the latch kept needsExplicitTermination() true through the entire pre-ready
    window and the first Ctrl+C would stop a replacement that never misbehaved. The fence stays
    time-bounded: an overdue replacement arms Ctrl+C again on its own account. The regression test
    was confirmed to fail without the fix.

  4. Leaked test child — fixed while reproducing the above.
    interactive-engine-abort-lifetime.test.ts SIGSTOPs its engine child and had no finally, so a
    failed assertion left a frozen child holding the test process's stdio pipes. That is the same
    leak class that hung the Windows job in fix(ci): restore green Windows CI and make timed-out suites diagnosable #2066, and one such orphan was found still running
    locally. Teardown is now unconditional.

Live verification against this branch

Run from the branch source with the engine-child environment scrubbed:

  • Escape on a wedged engine (kill -STOP the engine mid-bash, then Escape): the engine is
    not terminated, Engine terminated; appears zero times in the full scrollback, typing still
    works, and Ctrl+C then terminates explicitly with the typed draft preserved in the editor.
  • Engine death with an inline remote UI mounted (extension opens a non-overlay ctx.ui.custom
    that never resolves, then kill -9 the engine child): the stale UI is torn down within 5 s, a
    replacement engine spawns, the editor returns, and typing works. On released 0.9.11-alpha.7 the
    same sequence wedges the TUI until kill -9 from another terminal.

Contract amendments received

None. No mid-run user steering amended the objective. Every iteration's notes record "Contract amendments received: None"; the two entries that quote inherited text (iterations 3, 5, 6) quote upstream state reports about commit SHAs, not behavior changes. The six-item contract, the Bun-only rule, the 500-line wording, and the verification requirements were never amended.

Two conflicts were resolved by the implementer under the narrowest-reading rule because no user or supervisor was reachable through intercom:

  1. The literal 500-line cap versus the required Markdown edits (finding 1 above) — still needs a user decision.
  2. Whether the first Ctrl+C in a trapped remote UI should reach the host or the component — resolved as: unresponsive engine wins first; a mount that declares the new handlesCtrlC option gets press 1 and is closed on press 2; an undeclared mount closes on press 1.

What the change does

Fixes interactive-engine interrupt and engine-death handling in packages/coding-agent, against the reproduced evidence in /tmp/atomic-repro/engine-death-findings.md.

  1. Escape no longer kills the engine. abortAndRecover in isolated-runtime.ts dropped the 250 ms race against client.abort(); Escape now requests the cooperative abort and waits, unbounded. abort and the other cooperative-cancel commands joined LONG_LIVED_COMMANDS so the generic 30 s RPC deadline does not silently replace the removed one.
  2. The Engine terminated; <label> result unknown; inspect side effects before retrying diagnostic is gone, along with the startsWith("Engine terminated;") branch in interactive-mode-base.ts and the old transport-test assertion. Ripgrep finds it only in changelog prose and in two tests that assert it as forbidden text.
  3. Ctrl+C is the explicit escape hatch and now reaches the host through remote custom UI and overlays. Safety keys route by physical identity (interactive-key-identity.ts, pi-tui matchesKey), so rebinding app.clear cannot break either key.
  4. Engine death no longer locks the TUI. RemoteComponentController subscribes to a new host-local InteractiveEngineGenerationEnded event and tears down newest-first: settles ui.custom promises, releases widget keys, resets terminal modes, remounts the editor, restores focus, unwinds blockingInlineCustomUiDepth to 0. Teardown does not wait for a later engine_ready.
  5. Unexpected engine exit recovers via EngineHealthController — one automatic replacement, calm status notices rather than red chat errors, a restart permit fencing shutdown, and a bounded terminal-drain phase so a dying generation's admission frames are not lost.
  6. The four ATOMIC_INTERACTIVE_ENGINE_* variables no longer leak into any child. Deleting them from process.env is not sufficient under Bun 1.3.14 (verified: Bun snapshots the environment at startup for spawns that omit env), so the engine is now spawned with a scrubbed environment and receives host PID, guard path, and API key through a 0600 bootstrap file passed as a private argv flag and unlinked by the child. Subagent foreground, streaming-background, and detached spawns scrub after all merges.
  7. Typed input is not dropped. A send that fails before the engine admitted the request restores the raw draft (plus any queued submissions, FIFO) instead of showing Error: Agent process stopped. Ownership is decided by a new engine_request_accepted protocol frame (protocol version 1 → 2), not by output heuristics — this is exactly the boundary finding 2 above says still has a text-matching fallback.

Breaking change: third-party ctx.ui.custom() components that consume Ctrl+C must now declare handlesCtrlC: true; undeclared components close on the first press. All bundled surfaces (MCP panels, workflow graph) were migrated.

Verification claimed by the implementer and confirmed by both reviewers

Both reviewers independently ran the gates at HEAD 6ce8125f5:

  • bun run typecheck — exit 0
  • bun run lint — exit 0
  • bun run check:file-length — exit 0 (2,557 tracked source-like files, max 500)
  • bun run test:unit — 4,567 pass, 0 fail, 598 files
  • bun run --cwd packages/coding-agent docs:check — 38 pages

Both reviewers also drove the two required live tmux reproductions themselves against the built dist/cli.js:

  • Repro A — engine SIGKILLed under a never-resolving inline ctx.ui.custom: the stuck component is torn down, the editor is remounted, Interactive engine stopped unexpectedly; restarting. is shown as a calm status, and a replacement child starts. Typing, Escape, and Ctrl+C all work afterwards.
  • Repro B — real sleep 400 bash tool in flight, engine kill -STOP, then Escape: the engine PID is unchanged and in state Ts — not terminated, not restarted. Typing still works. A subsequent Ctrl+C terminates the wedged child, starts a replacement under Interactive engine is not responding; restarting., and leaves the typed draft intact.
  • Full-scrollback greps for Engine terminated returned zero hits in both reviewers' captures.
  • A live engine child's bash descendant printed ENGINE_ENV_CLEAN; ps eww on a live engine child showed no ATOMIC_INTERACTIVE_* variables at all.

Known gaps recorded by the implementer

  • Atomic's internal helper spawns that omit env (git, rg, fd, tmux, gh probes) still inherit Bun's startup environment snapshot. Harmless for the engine control values now that the engine never receives them; recorded as deferred rather than claimed fixed.
  • Deferred, not implemented: a typed RpcTransportError class replacing the marker; bootstrap directory-path unlink hardening; setStatus/setTitle generation ownership; { text, draft } records from EarlyInputState.
  • Not reproduced live: process-error and stdin-error terminal causes (unit-tested, sharing one failGeneration path); the drain barrier is pinned only by unit test; a real workflow stage-chat prompt card was not driven, only the graph overlay that hosts it.

QA end-to-end video

Not applicable and not produced. There is no web or frontend UI in this change; the user-visible surface is the terminal TUI. End-to-end proof is tmux pane captures under /tmp/atomic-repro/ from the implementer plus independent captures from both reviewers, all against the built CLI. Both reviewers inspected the nominal QA video path and confirmed the file does not exist. No video is attached and none was invented.

Scope

102 files changed, +6,719 / −561, across 9 commits on top of merge-base 8490bbf61. The branch is behind origin/main by unrelated CI commits; all figures are against the merge-base.


The full implementation notes follow as a comment on this PR.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@flora131

Copy link
Copy Markdown
Collaborator Author

Implementation notes (full contents, part 1 of 2)

Verbatim contents of the run's implementation-notes artifact implementation-notes.md. Split across two comments because the file is 68,161 characters and GitHub caps a comment at 65,536. Part 1 covers the task statement, acceptance matrix, and iterations 1-7.


Implementation Notes

Task: Read the file at /tmp/atomic-repro/engine-death-findings.md first — it contains reproduced evidence, exact file/line references, the user contract, and the required verification for this task. Work in the atomic monorepo (Bun; never node/npm/npx/yarn/pnpm).

Fix the interactive-engine interrupt and engine-death handling in packages/coding-agent:

  1. Remove the 250 ms cooperative-abort timeout completely. In packages/coding-agent/src/modes/interactive-engine/isolated-runtime.ts, abortAndRecover currently races client.abort() against sleep(250) and kills plus restarts the engine child when the race is lost. Pressing Escape must simply request the cooperative abort and wait for it. Interrupting must never terminate the engine, never restart it, and never emit the diagnostic 'Engine terminated; result unknown; inspect side effects before retrying'. Remove that message and its now-dead handling, including the startsWith("Engine terminated;") branch in packages/coding-agent/src/modes/interactive/interactive-mode-base.ts and the corresponding assertion in test/unit/interactive-engine-transport.test.ts. Verify with ripgrep that no code path can produce that string again.

  2. Keep a real escape hatch for a genuinely wedged engine. Termination must be an explicit user action (Ctrl+C, matching the existing watchdog copy 'Esc interrupt · Ctrl+C terminate') or a clearly unresponsive-engine path with a calm, non-alarming notice — never an automatic consequence of pressing Escape. Ctrl+C must reach the host even when a remote custom UI or overlay owns input; see routeGlobalClearInput in packages/coding-agent/src/modes/interactive/interactive-global-clear.ts, which today defers the global clear route whenever blockingInlineCustomUiDepth > 0 or an overlay is up.

  3. Fix the total TUI lockup. RemoteComponentController in packages/coding-agent/src/modes/interactive-engine/remote-component.ts never closes host-mounted remote components when an engine generation dies; it only resets terminal modes on engine_ready. InputFormHostController and SessionPickerHostController both disposeAll() on engine_ready and are the pattern to follow, but teardown must additionally be driven by an engine-death signal so a crash with no restart, or a hung or failed restart, still recovers. On engine death the host must close every mounted remote component, resolve their ui.custom promises, release widget keys, reset terminal modes, remount the editor into editorContainer, restore focus, and unwind blockingInlineCustomUiDepth to 0.

  4. Recover from an unexpected engine exit instead of leaving a live TUI bound to a dead engine. Today packages/coding-agent/src/modes/rpc/rpc-client.ts only rejects pending requests on the child exit event.

  5. Scrub the interactive-engine environment variables. The engine child never removes ATOMIC_INTERACTIVE_ENGINE_CHILD, ATOMIC_INTERACTIVE_ENGINE_HOST_PID, ATOMIC_INTERACTIVE_ENGINE_GUARD_FILE, or ATOMIC_INTERACTIVE_ENGINE_API_KEY from its own process.env, so every bash tool subprocess, subagent child, MCP server, and hook inherits them (this is the mechanism behind issue Subagent idle watchdog never fires under an interactive session: leaked ATOMIC_INTERACTIVE_ENGINE_CHILD makes children heartbeat every 50 ms #2062, and it exports an --api-key credential into every child). Capture the values once at startup into a frozen snapshot, delete them from process.env, and route all later readers through the snapshot (current readers: src/main.ts:58, src/main.ts:92, src/main.ts:98, src/modes/rpc/rpc-mode.ts:51, src/modes/interactive-engine/engine-child-liveness.ts:19-22). Add defense in depth by scrubbing them in the subagents spawn env at packages/subagents/src/runs/foreground/execution-attempt.ts and the background runner equivalent.

  6. Never silently drop typed input. When a submit fails because the engine is gone or restarting, restore the typed text to the editor instead of discarding it with 'Error: Agent process stopped'.

Verification is part of the task, not optional. Add bun:test + node:assert/strict regression tests under test/unit covering: a wedged engine where Escape does not terminate and produces no 'Engine terminated' text; engine-death teardown restoring the editor, focus, and blockingInlineCustomUiDepth 0; child spawn environments free of the ATOMIC_INTERACTIVE_ENGINE_* variables; and submitted text restored on send failure. Then run both live tmux reproductions described in the findings file against the built local CLI and capture tmux output proving typing, Escape, and Ctrl+C all still work and the removed error string never appears. Finish with bun run typecheck, bun run lint, bun run check:file-length, and bun run test:unit all green. Keep every touched file at or under 500 physical lines. Add a packages/coding-agent/CHANGELOG.md entry under [Unreleased] and update the user-facing docs in packages/coding-agent/docs that describe interrupt or engine behavior.

Acceptance Matrix

# Contract clause Verification
1 250 ms cooperative-abort race removed from abortAndRecover read isolated-runtime.ts; test/unit/interactive-engine-wedged-abort.test.ts (live SIGSTOP engine, Escape, PID+generation unchanged)
2 Escape never terminates/restarts the engine same live test asserts engine PID + generation unchanged after Escape past the 1 s watchdog window
3 String Engine terminated; and result unknown; inspect side effects before retrying produced nowhere rg -n -F both strings over the repo returns no source/test match
4 startsWith("Engine terminated;") branch removed from interactive-mode-base.ts read file
5 Assertion removed from test/unit/interactive-engine-transport.test.ts read file
6 Explicit escape hatch = Ctrl+C (watchdog copy Esc interrupt · Ctrl+C terminate), calm notice interactive-global-clear.test.ts + live repro B (Ctrl+C replaces engine PID)
7 Ctrl+C reaches host while remote custom UI / overlay owns input routeGlobalClearInput escalation unit test; native form/selector cancel path preserved test
8 Engine-death teardown: components closed, ui.custom promises settled, widget keys released, terminal modes reset, editor remounted into editorContainer, focus restored, blockingInlineCustomUiDepth === 0 test/unit/interactive-engine-death-teardown.test.ts
9 Teardown does not depend on a later engine_ready; no dispose command leaks into a new generation same test asserts no engine_custom_dispose sent
10 Unexpected engine exit recovers (one automatic restart attempt) test/unit/interactive-engine-death-teardown.test.ts (fake client) + live repro A (new engine PID after SIGKILL)
11 Hung/failed restart still leaves editor usable fake-client test: restart pending → editor mounted+focused, depth 0
12 ATOMIC_INTERACTIVE_ENGINE_{CHILD,HOST_PID,GUARD_FILE,API_KEY} absent from child env test/unit/interactive-engine-env-scrub.test.ts (pure helper + live engine-child env probe)
13 Snapshot preserves child-mode selection, api-key transfer, liveness, guard file live default-main harness still binds engine + repro A/B run
14 Subagent foreground/background/detached spawn envs scrubbed after all merges unit test over the three env builders
15 Typed input restored to editor when the send fails test/unit/interactive-prompt-send-failure.test.ts; raw (untrimmed) draft preserved verbatim
16 Not restored after a turn actually started same test
17 tmux repro A + B captured pane captures under /tmp/atomic-repro/after-fix-*
18 bun run typecheck, lint, check:file-length, test:unit green command output
19 Every touched file ≤ 500 physical lines bun run check:file-length
20 CHANGELOG [Unreleased] entry + docs updated (extensions.md, keybindings.md, usage.md) read files

Interface decisions (open points resolved permissively/preservingly):

  • New host-local event InteractiveEngineGenerationEnded { generation, error, kind, expected } exactly as researched.
  • Diagnostic source union widened with "recovery"; shouldRenderEngineDiagnosticAsChatError now requires source === undefined so recovery notices stay out of the red chat error path.
  • Restored draft is the untrimmed editor text captured at submit (verbatim), falling back to the trimmed prompt when no raw draft matches.

Running Notes

Contract amendments received

None. No mid-run user steering arrived.

Implementation decisions

  • New modules. packages/coding-agent/src/utils/interactive-engine-env.ts (frozen startup snapshot + pure scrubInteractiveEngineEnv, exported from the package index), modes/interactive-engine/engine-generation.ts (host-local InteractiveEngineGenerationEnded type), modes/interactive-engine/engine-health.ts (EngineHealthController: diagnostic fan-out, unresponsive tracking, single-flight recovery, explicit terminate), modes/interactive-engine/engine-diagnostic-view.ts (host presentation policy), modes/interactive/interactive-prompt-restore.ts (send-failure classification + draft restore), packages/subagents/src/runs/shared/spawn-env.ts (buildSubagentSpawnEnv).
  • Escape. abortAndRecover is now settleBeforeAbort()await client.abort() → clear callback-active. No sleep, no restart, no diagnostic. The sleep import was removed from isolated-runtime.ts.
  • Ctrl+C gate. Research proposed arming the escape hatch on any pending cooperative abort. That would let a stray second Ctrl+C during a healthy ~5 ms abort replace the engine, so the gate is narrower: watchdog-confirmed unresponsive, or a cooperative abort outstanding ≥ UNANSWERED_ABORT_ESCALATION_MS (1 s, mirroring the watchdog threshold). Nothing fires automatically at that point; it only decides whether a deliberate Ctrl+C escalates. The unresponsive mark is cleared on any engine_heartbeat, so a self-recovering engine disarms it.
  • routeGlobalClearInput. Modal guards are unchanged for a healthy engine; only engineNeedsExplicitTermination() lets Ctrl+C escalate past an overlay / blocking inline custom UI / non-editor input owner. Native form and selector cancel behaviour is locked down by tests.
  • Remote teardown modes. RemoteComponentController.disposeAll(reason) distinguishes "generation-lost" (settle host promises, local-only proxy disposal) from "host-shutdown" (notify the still-live child, do not repaint a stopping TUI). RemoteComponent.dispose(notifyEngine) gained the flag so death cleanup can never address remote_component_1 in a replacement generation.
  • RpcClient.stop() now fails the engine monitor. Readiness has no deadline by design, so without this an explicit stop of a hung replacement child could never be observed and Ctrl+C could not rescue it.
  • shouldRenderEngineDiagnosticAsChatError now requires source === undefined (was !== "watchdog"), so the new source: "recovery" notices render as calm status instead of red chat errors.

Reproduced defect found only by live testing

The first clean tmux run of Reproduction A still froze. Tracing the built CLI showed the death teardown ran correctly (disposeAll → host donerestoreEditor), but a second engine_custom_open arrived immediately after and re-mounted the stale component. Root cause: restoring the draft /freeze-test to the editor made the next getUserInput() run recoverCookedStartupInput(), which treats a single command-like editor line as a submitted startup command and replayed it, re-running the command in the fresh engine. Fixed by marking startupCookedInputRecovered = true whenever a draft is restored (a host-restored draft is a draft, never cooked startup input). Covered by test/unit/interactive-prompt-send-failure.test.ts → "a restored draft is a draft, never cooked startup input".

Bun environment caveat (user-relevant)

Bun 1.3.14 snapshots the process environment at startup for child spawns that omit env; later delete process.env.X does not reach those children (verified directly: FOO_PRE=1 bun -e 'delete process.env.FOO_PRE; …' still shows 1 in the child, while Node shows null). All named leak paths build their child env from process.env and are therefore clean after the scrub — bash tool (getShellEnv), MCP (resolveEnv), subagents (buildSubagentSpawnEnv) — and extension ctx.exec was changed to pass env: { ...process.env } for the same reason. Atomic's own internal helper spawns that omit env (git/rg/fd/tmux/gh probes) still inherit Bun's startup snapshot; that is recorded as deferred work rather than silently claimed fixed.

Validation

  • bun run typecheck — clean. bun run lint — clean. bun run check:file-length — passed (2518 files, max 500).
  • bun test test/unit — 4439 pass, 1 skip, 0 fail (584 files).
  • rg -n -F 'Engine terminated;' and rg -n -F 'result unknown; inspect side effects before retrying' match only the two regression-guard constants and the changelog description; no production path can emit either string.
  • Negative checks (regression tests genuinely fail pre-fix): with the onGenerationEnded subscription and engine_ready disposal reverted in remote-component.ts, all three tests in interactive-engine-death-teardown.test.ts fail; with only the death subscription reverted, the ordering test fails.
  • Live tmux, built CLI (packages/coding-agent/dist/cli.js, rebuilt clean after removing all trace patches):
    • Reproduction A (inline ctx.ui.custom mounted, engine SIGKILLed): stale component gone, editor remounted with the restored /freeze-test draft, typing lands, Escape returns, Ctrl+C clears the editor, replacement engine bound (84123 → 84144). Captures: /tmp/atomic-repro/after-fix-repro-a.txt, after-fix-repro-a-keys.txt, after-fix-repro-a-ps.txt.
    • Reproduction B (!sleep 400, engine kill -STOP, Escape): engine PID unchanged (84423, STAT Ts), host responsive, typing works; then Ctrl+C replaced it (84423 → 84563, old engine gone) with Interactive engine is not responding; restarting.. Captures: after-fix-repro-b-escape.txt, after-fix-repro-b-ctrlc.txt, after-fix-repro-b-ps.txt.
    • Env leak: !env | grep ATOMIC_INTERACTIVE_ENGINE || echo ENGINE_ENV_CLEAN inside a real engine child's bash tool printed ENGINE_ENV_CLEAN. Capture: after-fix-engine-env.txt.
    • Neither forbidden string appears in any capture.

Deferred (out of contract)

  • Atomic's internal helper spawns that omit env (spawnSync("git"/"which"/"trash"), spawn("tmux"/"gh"/rg/fd)) still inherit Bun's startup environment snapshot. Harmless for the engine control values now that the engine never receives them, but a follow-up could pass env explicitly everywhere.
  • ATOMIC_SESSION_ID not matching the UUID in ATOMIC_SESSION_FILE, and orphan engine children from long-dead hosts, were listed as context-only observations in the findings and were not touched.

Iteration 2 — consolidated findings repaired

Contract amendments received

None. The only mid-run user message inherited through the research artifact was the research instruction itself ("Research whether each unresolved finding still applies and what objective-aligned implementation change would resolve it"), which did not amend the six-item contract.

1. Draft restoration now merges instead of overwriting

restoreUnsentPromptDraft reads the live buffer through a new getEditorText() on the target and writes mergeRestoredDraft(draft, current): the failed submission first, a blank line, then whatever was typed while the send was pending. Presence is measured on the raw string, so whitespace-only typing survives; neither side is trimmed. Focus still stays with a modal that owns input.

Deviation recorded: the draft is exactly what the editor handed to onSubmit. The built-in editor trims before invoking that callback, so the restored text is not a byte-for-byte pre-submit buffer; docs and changelog now say "exactly as the editor handed it over" rather than claiming whitespace preservation. Cursor position is not restored — pi-tui exposes no cursor setter, and setText places the cursor at the end.

2. Direct submit branches share the failure handler

restoreFailedSubmissionDraft(mode, error, draft) is the single host-bound entry point. The submit callback gained one outer catch that restores the untrimmed callback text and rethrows anything that is not a send failure; handleFollowUp (Alt+Enter) wraps both of its direct dispatches the same way. handleBashCommand and handleCompactCommand no longer swallow send failures: bash rethrows only while the command has produced no output (an acknowledged command keeps its rendered Bash command failed: …), and compaction rethrows only a send failure while other failures stay event-driven.

Tradeoff recorded: a silent command such as !sleep 400 produces no output, so if the engine dies mid-flight its text is returned as a draft even though the command had started. Nothing is re-executed automatically, and the alternative loses typed input, so this errs toward the contract.

3. The engine now launches with a clean environment

Deleting the variables cannot satisfy the contract under Bun 1.3.14, so the design changed rather than the scrub. spawnRpcClientProcess writes an owner-only (0600) bootstrap file — atomically, temp sibling plus rename, in a private temp directory — containing host PID, guardian path, and any API key, then spawns the engine with scrubInteractiveEngineEnv({ ...process.env, ...env }) and a private --internal-engine-bootstrap <path> argument. main() strips that argument before any CLI parsing, reads and unlinks the record, freezes the snapshot, and still deletes the legacy variables as defense in depth. The four env writes are gone; rg confirms no writer remains. Engine mode is now selected by the bootstrap argument, which nothing can inherit. The host also removes a bootstrap file on spawn failure and on child exit.

The regression test now uses the omitted-env Bun.spawnSync shape that actually reproduces the Bun defect (it previously passed { env: { ...process.env } }, which masked it) and additionally asserts the API key is absent from the child's argv and from both environments. Verified as a true regression: re-adding the four variables to the spawn environment makes it fail.

4. Ctrl+C rescues a pre-ready hung replacement

EngineHealthController records attemptStartedAt when a new replacement begins and clears it in the same finally that releases the attempt. needsExplicitTermination() now also returns true for an attempt overdue past ENGINE_UNRESPONSIVE_MS, which moved to activity-watchdog.ts so the watchdog and the escape hatch share one threshold. The clock is injectable, so the tests cross the threshold without sleeping. A repeat Ctrl+C force-stops an overdue replacement at most once per attempt (fenced by attempt id) and the in-flight termination then starts one more replacement; a fresh replacement is never stopped by a stray press. An attempt the user deliberately stopped no longer reports Interactive engine restart failed: ….

5. The env probe publishes atomically

The fixture writes <probe>.tmp and renames it onto the final path, so the test's existence poll can only observe complete JSON.

6. Rebased onto origin/main and committed

git switch -c fix/interactive-engine-recoverygit stash push -ugit rebase origin/main (clean, no conflicts) → git stash popgit add -A. HEAD is 2b06d3a53 on top of 8490bbf61; the worktree is clean and nothing is untracked. Staging mattered for the gate: check:file-length now sees 2534 tracked files instead of 2518, so the 16 new files are gated for the first time — and pass.

Iteration 2 validation (all on the rebased tree, after git add -A)

  • bun run typecheck — clean. bun run lint — clean. bun run check:file-length — passed (2534 files). bun run --cwd packages/coding-agent docs:check — passed (38 pages).
  • bun run test:unit — 4479 pass, 1 skip, 0 fail (586 files). Pre-commit hooks re-ran lint, file-length, and test:unit on commit and passed.
  • packages/coding-agent/test/unsupported-provider-headless.test.ts has one pre-existing failure ("RPC cycle_model clears persisted unsupported lock before the next prompt"); confirmed by stashing the whole change and re-running, so it is not caused by this work. The full packages/coding-agent/test suite exceeds a 40-minute run window and was not run to completion.
  • Static searches: rg -F 'Engine terminated;' and rg -F 'result unknown; …' over packages test match only two negative-test constants and the changelog description; rg 'sleep|250' over isolated-runtime.ts matches nothing; no source file writes the four engine variables.
  • Live tmux against the final rebased build (packages/coding-agent/dist/cli.js):
    • Repro A (inline custom UI + SIGKILL): stale component gone, editor restored, typing after engine death/Escape/Ctrl+C/still usable all work, replacement engine bound (478 → 500). Captures after-fix-repro-a.txt, after-fix-repro-a-ps.txt.
    • Draft merge, live: engine SIGSTOPped, never accepted submission submitted, world typed during the hang typed, engine SIGKILLed → editor shows the failed submission, a blank line, then the later text. Capture after-fix-repro-a-merge.txt.
    • Engine-child env: !env | grep ATOMIC_INTERACTIVE_ENGINE || echo ENGINE_ENV_CLEAN printed ENGINE_ENV_CLEAN. Capture after-fix-engine-env.txt.
    • Repro B (!sleep 400, SIGSTOP, Escape): engine PID unchanged (716), host responsive, typing works; Ctrl+C replaced it (716 → 805) and the unsent bash draft came back above the later text. Captures after-fix-repro-b-escape.txt, after-fix-repro-b-ctrlc.txt, after-fix-repro-b-ps.txt.
    • Repro C (new): engine SIGKILLed, its replacement frozen with SIGSTOP before engine_ready, threshold crossed, Ctrl+C → frozen child gone, third engine started, editor text intact, only the calm Interactive engine is not responding; restarting. shown. Captures after-fix-repro-c-preready.txt, after-fix-repro-c-ps.txt.
    • No capture contains either forbidden string.

QA E2E Video

Not applicable: there is no web or frontend UI in this change. The user-visible surface is the terminal TUI, verified end-to-end with the tmux skill against the built CLI (packages/coding-agent/dist/cli.js) for reproductions A, B, and the new pre-ready replacement case, with pane captures and ps output saved under /tmp/atomic-repro/after-fix-*. No playwright-cli video was produced and none was attempted, because no browser-renderable scenario exists in this diff.

Iteration 3 — round-two review findings repaired

Contract amendments received

Inherited through the research artifact, verbatim:

The work is now committed as 2b06d3a53 on top of origin/main (8490bbf61) with a clean tree, and the prior round's five blockers are closed … Review round two rejects it on eleven findings.

A state report, not a behavior change. The six-item contract, Bun-only rule, 500-line gate, and verification requirements are unchanged.

1. Ctrl+C escapes a remote proxy even while the engine is healthy

New modes/interactive-engine/remote-input-ownership.ts holds a per-runtime ownership probe registered by attachInteractiveEngineHost. RemoteComponentController answers it exactly: a focused remote overlay wins outright (OverlayHandle.isFocused()); otherwise any visible overlay means a native one owns input; otherwise the question is whether the component sitting in the editor's place is a live remote proxy. Widgets are excluded. routeGlobalClearInput checks that first and calls restartInteractiveEngineForRemoteUi(), which is deliberately not gated on engine health and reports the calm Restarting interactive engine. (new REMOTE_UI_RESTART_NOTICE) instead of falsely claiming the engine is unresponsive.

The old route test asserted the opposite (healthy remote UI defers); it was rewritten.

2. Escape's hidden 30-second abort deadline

"abort" added to LONG_LIVED_COMMANDS. Nothing else changed in rpc-client.ts. New test/unit/interactive-engine-abort-lifetime.test.ts starts a real engine with requestTimeoutMs: 60, SIGSTOPs it, and proves the abort stays pending past the deadline while get_state still times out, then rejects on stop(). Verified as a true regression: removing the set entry makes it fail.

3. Failed-replacement latch

replacementNeeded is set when an attempt fails on its own, cleared when a new attempt starts, and included in needsExplicitTermination(). Automatic recovery stays single-shot; each Ctrl+C starts exactly one attempt and a repeat failure re-arms. The pre-existing "stops recovering" test now also asserts Ctrl+C is armed.

4. Generation-owned RPC dialogs

New modes/interactive-engine/engine-dialog-host.ts. select, confirm, input, and editor are recorded with the generation that opened them; generation death aborts exactly those mounts and suppresses their replies so nothing is written to the replacement child. ExtensionUIContext.editor() gained a backward-compatible third opts argument (the research's sanctioned option) so the editor mount is cancellable, and hideExtensionSelector/Input/Editor are now instance-scoped: a late abort from a dead generation cannot dismiss a newer native dialog.

5. Teardown order and focus fence

RemoteComponentController.disposeAll unwinds newest-first. showExtensionCustom now hides an overlay through its own captured handle rather than the generic top-overlay call, and restoreEditor skips focusing the editor while an overlay survives.

Honest limitation: I could not construct a local reproduction where the old oldest-first order produced stale focus — with this pi-tui build the nested test passes either way, apparently because focus restoration skips unmounted targets. The new nested test (/freeze-nested, inline + overlay, SIGKILL) asserts the required end state, including a new focusIsStaleInline probe that is false on every heartbeat after the kill, but it is an end-state guard rather than a proven pre-fix failure.

6. Pending pastes survive draft restoration

restoreFailedSubmissionDraft reads getExpandedText?.() ?? getText(). A large paste lives in the editor's private registry behind a [paste #1 …] marker and setText() clears that registry, so reading the marker round-tripped dead text.

7 & 8. Bootstrap ownership and publication cleanup

writeInteractiveEngineBootstrap returns an InteractiveEngineBootstrapHandle { path, directory } and removes its own temp file and directory on any write/rename failure. Recursive cleanup is removeOwnedInteractiveEngineBootstrap(handle) — only the creator can do it. The child reader now calls rmSync(path, { force: true }) on the argv path and nothing else. Covered by unit tests including a hostile-path case (parent, sibling, and nested files survive), a real-CLI hostile-path case, and a forced-publication-failure case that asserts no leftover directory.

Iteration 3 validation

  • bun run typecheck, bun run lint, bun run check:file-length (2539 tracked files), bun run --cwd packages/coding-agent docs:check (38 pages) — all clean.
  • bun run test:unit — 4504 pass, 1 skip, 0 fail (589 files).
  • Negative checks: removing "abort" from LONG_LIVED_COMMANDS fails the new abort-lifetime test; re-adding the engine env vars to the spawn fails the env-scrub test (re-confirmed in iteration 2).
  • Live tmux on the rebuilt CLI:
    • Repro C (new, healthy remote UI): typing went to the proxy, one Ctrl+C replaced engine 11847 → 11888 with Restarting interactive engine., editor usable, /freeze-test draft restored. Captures after-fix-repro-c-remote-ui.txt, after-fix-repro-c-ps.txt.
    • Repro A: engine 11922 SIGKILLed under a mounted inline UI → component gone, typing/Escape/Ctrl+C work, replacement 11950. Captures after-fix-repro-a.txt, after-fix-repro-a-ps.txt.
    • Repro B: engine 12001 SIGSTOPped, Escape held for 35 s (past the old 30-second deadline) with the same PID and no Timeout waiting for response to abort; Ctrl+C then replaced it (12001 → 12064) and the unsent bash draft came back above the later text. Captures after-fix-repro-b-escape.txt, after-fix-repro-b-ctrlc.txt, after-fix-repro-b-ps.txt.
    • Control: Ctrl+C in the native /model selector cancelled locally and left engine 12099 running. Capture after-fix-native-cancel.txt.
    • Engine-child env probe: ENGINE_ENV_CLEAN. Capture after-fix-engine-env.txt.
    • rg 'Engine terminated;|result unknown; inspect side effects before retrying|Timeout waiting for response to abort' over every capture: no matches.

Iteration 4 — no new findings; recorded caveats closed

Contract amendments received

None. The consolidated research artifact for this iteration
(research/2026-07-29-read-the-file-at-tmp-atomic-repro-engine-death-findings-md-first-it-contains-rep.md)
is a 0-byte file — verified with wc -c and ls -la; no other findings
document was written in the last three hours anywhere in the repo or under
/tmp/atomic-repro (/tmp/atomic-repro/engine-death-findings.md is still the
original 14:38 copy). There were therefore no new findings to repair, and none
were invented.

Instead this iteration closed the two verification gaps I recorded honestly at
the end of iteration 3, both of which concerned evidence quality for contract
item 3 rather than new behavior.

Caveat 1 closed: the --no-session repro-C anomaly was cold-start timing

Iteration 3 reported that the research's exact repro-C command line
(--no-session --approve -e …) never mounted the custom UI. Re-running it on a
warm module cache mounted immediately, and the full repro C then passed with
that exact command line: engine 22833 healthy with the proxy owning input, one
Ctrl+C replaced it with 22920 under the calm Restarting interactive engine.,
the editor came back usable and the /freeze-test draft was restored. The
earlier failures were the first runs after rm -rf dist && build, where engine
startup outran the fixed sleep before the prompt was submitted. No behavioral
difference between --no-session and a session-backed run. Capture refreshed at
/tmp/atomic-repro/after-fix-repro-c-remote-ui.txt and after-fix-repro-c-ps.txt.

Caveat 2 closed: nested teardown order is now a true regression test

Iteration 3 could not show the newest-first ordering failing before the fix,
because the live harness's pi-tui build skips unmounted focus targets. The
ordering is now asserted directly at the controller level in
interactive-engine-terminal-control.test.ts: the fake host bridge records the
order in which each mount's host done runs, and the new test mounts an inline
proxy plus an overlay above it and asserts the overlay closes first. Verified as
a real regression — reverting .reverse() in disposeAll fails it with
"the overlay must close before the inline layer it was stacked on".

New test/unit/interactive-extension-custom-ui-overlay.test.ts covers the other
half of that finding against the real showExtensionCustom: closing an overlay
custom UI uses its own captured handle (not the generic top-overlay call), and
an inline close does not pull focus away from a surviving overlay. Also a true
regression — reverting both changes fails two of its three cases.

Iteration 4 validation (HEAD 75955b410 plus these tests)

  • bun run typecheck, bun run lint, bun run check:file-length (2540 tracked
    files), bun run --cwd packages/coding-agent docs:check (38 pages) — clean.
  • bun run test:unit — 4508 pass, 1 skip, 0 fail (590 files).
  • Live tmux repro C re-run with the research's exact command line; no capture
    contains Engine terminated;, result unknown; …, or
    Timeout waiting for response to abort.

Iteration 5 — round-four findings

Contract amendments received

Inherited through the research artifact, verbatim:

Round four is committed: 1b99e1ad9 (nested-teardown/overlay-close tests) on 75955b410 and 2b06d3a53, clean tree, on top of origin/main (8490bbf61).

A state report. The six-item contract, Bun-only rule, regression, tmux, docs, and changelog requirements are unchanged.

The 500-line "every touched file" conflict (research finding 5)

The research declares implementation "contract-blocked" until the user amends the
wording. I did not block. Per the steering contract, an instruction that
conflicts with the launch contract and cannot be resolved with the user is
implemented in the narrowest reading consistent with the launch contract,
with the conflict stated. The same launch contract requires updating
packages/coding-agent/CHANGELOG.md (6,142 lines) and
packages/coding-agent/docs/extensions.md (2,829 lines); both were already over
500 lines before this work began, and released changelog sections are immutable.
"Every touched file ≤ 500 lines" and "update these two files" cannot both hold
literally, so I read the limit as the repository gate it matches
(bun run check:file-length, source-like files only, verified green at 2,543
files) and left the two Markdown files over the limit. This needs a user
decision if the literal wording was meant.

Findings repaired

  1. Physical key identity. New interactive-key-identity.ts matches Escape
    and Ctrl+C through pi-tui's matchesKey/Key/isKeyRelease rather than raw
    prefixes, and the route consults it before app.clear. Escape can never enter
    a clear/terminate/restart branch; Ctrl+C keeps the host route when app.clear
    is rebound; a non-safety app.clear still clears; releases never act.
    CustomEditor got the same fixed Escape guard ahead of its configurable
    action loop, keeping Escape-as-autocomplete-cancel.
  2. Nested generation death. An unexpected death while a replacement attempt
    is in flight now latches replacementNeeded and returns instead of ignoring
    the event, so Ctrl+C is armed once the attempt settles and nothing retries
    automatically. Expected ends and shutdown do not latch.
  3. Line-widget ownership. EngineDialogHostController tracks widget key →
    latest writing generation, releases only keys the dead generation still owns,
    transfers ownership on rewrite, and clears everything it owns on dispose.
    Status and title ownership are deferred (see below), as the research
    permits.
  4. Pre-trim draft capture. CustomEditor snapshots getExpandedText() for
    the same synchronous dispatch that may submit, exposed as
    takeSubmittedDraft(); the submit handler prefers it and falls back to the
    callback argument. Covered by real-editor tests through a real Enter.
  5. Duplicate red error. All three submit paths now show the transport error
    only when restoration returned false.

Carry-forward items from round three

  • Cooperative-cancel deadlines: abort_bash, abort_compaction,
    abort_retry, and cancel_login_provider joined abort in the exempt set,
    which moved to a new rpc-command-timeouts.ts (rpc-client.ts was at exactly
    500 lines; it is now 469).
  • Heartbeat channel: heartbeats moved from a generic
    onInteractiveEngineMessage subscriber — which drained the buffered startup
    custom-UI messages into itself — to a dedicated interactiveEngine.onHeartbeat
    callback.
  • QueuedWriter: the in-flight frame is tracked and rejected by close(), so
    a callback-level EPIPE can no longer leave a prompt's or abort's promise
    pending forever.
  • Workflow Ctrl+C: rather than always intercepting, the route now gives the
    first Ctrl+C to the focused remote proxy (so the workflows prompt card's
    ctrl+c Skip and the stage chat's ctrl+c Close work) and escalates only when
    the same component still owns input on the next press. This reconciles the
    round-three "always intercept" instruction with the round-four
    workflow-preservation requirement and with "breaking changes are not allowed";
    the launch contract says Ctrl+C must reach the host, not that it must do so on
    the first press. Flagged as a deliberate divergence from round-three
    research.

Deferred (recorded, not implemented)

  • Typed RpcTransportError replacing message-substring send-failure
    classification. The substring set is exact for the errors RpcClient raises
    and is covered by tests; a typed error is a larger refactor across the client,
    writer, and restore path.
  • Per-submission { text, draft } records instead of the single
    submittedDraftText slot. Correct for the one in-flight prompt today; queued
    submissions would need the record.
  • Generation ownership for setStatus and setTitle. The research explicitly
    allows deferring these; neither affects keyboard recovery, and title
    restoration needs a host-private updateTerminalTitle() callback because the
    terminal API has no getTitle().

Iteration 5 validation

  • bun run typecheck, bun run lint, bun run check:file-length (2,543 tracked
    files), bun run --cwd packages/coding-agent docs:check (38 pages) — clean.
  • bun run test:unit — 4,526 pass, 1 skip, 0 fail (591 files).
  • Negative checks (each fails when its fix is reverted): pre-trim capture (4 of 5
    cases), nested-death latch, and — from earlier iterations — abort exemption,
    teardown order, overlay close targeting, env launch.
  • Live tmux on the rebuilt CLI:
    • Remap run ({"app.clear":"escape"}, trapped remote UI): three physical
      Escapes left engine 45092 untouched; the first Ctrl+C went to the component;
      the second replaced it with 45150 under Restarting interactive engine.;
      editor usable. Captures after-fix-remap-keys.txt, after-fix-remap-ps.txt.
    • Repro A: engine 45186 SIGKILLed under a mounted inline UI → component
      gone, typing/Escape/Ctrl+C work, replacement 45214.
    • abort_bash: !sleep 400 + Escape, waited 36 s — no
      Timeout waiting for response to abort… anywhere. Capture
      after-fix-abort-bash.txt.
    • Repro B: engine 45245 SIGSTOPped, Escape held 35 s with the same PID,
      then Ctrl+C replaced it with 45332 and the unsent bash draft returned above
      the later text.
    • Native control: Ctrl+C in /model cancelled locally, engine 45596
      untouched.
    • Engine env: ENGINE_ENV_CLEAN.
    • No capture contains Engine terminated;, result unknown; …, or
      Timeout waiting for response to abort.
  • Not re-run this iteration: the workflows prompt-card/stage-chat tmux scenario.
    The preservation behavior is covered by route unit tests
    ("the first Ctrl+C reaches a remote proxy…", "a component that handled the
    first Ctrl+C disarms the escape"), but I did not drive a real workflow run.

Iteration 6 — round-five findings

Contract amendments received

Inherited through the research artifact, verbatim:

Round five is committed as e597e94cc ("route safety keys by physical identity and keep recovery armed") on 1b99e1ad9 / 75955b410 / 2b06d3a53, clean tree, over origin/main (8490bbf61).

A state report. The six-item contract, Bun-only rule, required tests, live tmux checks, docs, changelog, and line-limit wording are unchanged.

The two questions the research routed to the user

No supervisor or originating stage was reachable: intercom lists only six idle
sibling subagents in this workflow group, none of them the user. Per the steering
contract I implemented the narrowest reading and state both conflicts.

  1. First Ctrl+C in a trapped remote UI. Implemented the research's own narrow
    design, which is also the smallest one that satisfies the launch contract's
    "Ctrl+C must reach the host" on the first press without breaking bundled
    extension bindings: unresponsive engine wins first; a mount that declared the
    new handlesCtrlC option receives the press (and is closed if it still owns
    input on the next one); an undeclared mount is closed on the first press
    through the ordinary close path, leaving the engine and its other components
    alone. Behavior change for third-party components that consume Ctrl+C
    without declaring it
    — they now close instead. Every bundled surface that
    binds the key declares it.
  2. 500-line "every touched file". Unchanged from iteration 5: read as the
    repository's source-like gate (bun run check:file-length), with the required
    CHANGELOG.md (6,151 lines) and docs/extensions.md (2,831 lines) left as
    they were. Still needs a user decision if the literal wording was meant.

Findings repaired

  1. Remote Ctrl+C ownership — new optional handlesCtrlC on ctx.ui.custom()
    options, carried additively on engine_custom_open, recorded per mount, and
    answered by RemoteComponentController.remoteProxyHandlesCtrlC(). New
    dismissRemoteProxy() closes exactly one record through done(undefined)
    while it is still registered, so showExtensionCustom() restores the editor
    or hides that overlay and the finalizer sends engine_custom_dispose to the
    healthy child. The two-press latch now applies only to declared components.
    restartInteractiveEngineForRemoteUi and the remote-ui termination reason
    are gone; a healthy engine is never replaced for a UI reason.
  2. Startup-window exitRpcClient retains the terminal event and replays it
    to a listener that attaches afterwards, only while
    retained.generation === this.generation.
  3. Stale buffered frames — new GenerationBuffer tags buffered engine
    messages and extension UI requests with their generation; observation rejects
    anything from a replaced or already-ended generation; publishGenerationEnded
    drops that generation's buffers before notifying listeners. restartArgs moved
    to rpc-client-process.ts as restartCliArgs to keep rpc-client under the gate
    (485 lines).
  4. Per-submission drafts — new InteractiveSubmission { text, draft } carried
    through onInputCallback, pendingUserInputs, getUserInput(), and
    runUserPromptTurn() (which still accepts a bare string). submittedDraftText
    is gone. A failed send restores its own draft plus every still-queued
    submission's draft in FIFO order and clears only what it restored.
  5. Bootstrap directory path — deferred, as the research directs.

Deferred (recorded, not implemented)

  • Typed RpcTransportError instead of message-substring send-failure matching.
  • Bootstrap directory-path unlink hardening (beyond_objective per the review).
  • setStatus / setTitle generation ownership.

Iteration 6 validation

  • bun run typecheck, bun run lint, bun run check:file-length (2,543 files),
    bun run --cwd packages/coding-agent docs:check (38 pages) — clean.
  • bun run test:unit — 4,539 pass, 1 skip, 0 fail (593 files).
  • New tests: interactive-engine-generation-lifecycle.test.ts (replay, stale
    buffer, pure buffer), interactive-engine-remote-dismiss.test.ts (focused
    close, declaration, nested survival, widgets), rewritten
    interactive-global-clear-route.test.ts (16 cases), three new send-failure
    cases (overlapping identical-normalizing drafts on one host, FIFO queue
    restore, queue untouched on a non-send failure).
  • Negative checks, each failing when its fix is reverted: retained replay and
    buffer drop (2 of 3 lifecycle tests fail), per-submission draft (3 send-failure
    tests fail).
  • Live tmux on a CLI rebuilt from this tree (dist/cli.js newer than every
    source file), captures under /tmp/atomic-repro/after-fix-*:
    • undeclared inline /freeze-test: first Ctrl+C printed
      freeze-test inline closed: undefined, engine 66120 unchanged, typing worked.
    • undeclared overlay /freeze-overlay: same, engine 66120 unchanged.
    • declared /freeze-declared: press 1 reached the component (presses seen: 1),
      press 2 closed it from the host, engine 66328 unchanged.
    • repro A: engine 66328 SIGKILLed under a mounted inline UI → component gone,
      Interactive engine stopped unexpectedly; restarting., replacement 66401,
      typing/Escape/Ctrl+C all worked.
    • repro B: engine 66550 SIGSTOPped, 7 Escapes over ~35 s left the same PID;
      Ctrl+C replaced it with 66786 under Interactive engine is not responding; restarting. and returned the unsent draft to the editor.
    • !sleep 400 + Escape, 40 s: zero Timeout waiting for response to ….
    • remapped {"app.clear":"escape"}: three Escapes left the trapped component
      and engine 67657 alone; Ctrl+C closed the component, engine still 67657.
    • workflows run picker (undeclared): closed on the first Ctrl+C, engine 67657
      alive. Workflow graph overlay (declared): closed through its own binding,
      engine 67657 alive.
    • !env | grep -c ATOMIC_INTERACTIVE_ENGINEENGINE_ENV_CLEAN.
    • No capture contains Engine terminated;, result unknown; …, or
      Timeout waiting for response to abort….
  • Covered by unit tests rather than live runs this iteration: Kitty key-release
    input, startup child exit before host attachment, overlapping exact-space
    drafts. A real workflow stage-chat prompt card was not driven; the graph
    overlay that hosts it was.

Iteration 7 — round-six findings

Contract amendments received

None. The research artifact's own "Contract amendments received" section reads
"None", and no user message arrived this iteration.

Finding 1 — raw writer failures dropped drafts (fixed)

QueuedWriter forwarded stream-callback errors unchanged, and draft restoration
recognized a send failure by matching five message fragments. A real dying
engine produces write EPIPE, Cannot call write after a stream was destroyed,
or write after end, none of which matched — so those submissions showed a red
error and the typed text was discarded.

Implemented the research's additive discriminant in a new
packages/coding-agent/src/modes/rpc/rpc-transport-error.ts:

  • markRpcTransportFailure() adds a non-enumerable rpcFailureKind: "transport"
    to the existing Error, so identity, instanceof, code, errno, and
    syscall all survive; a frozen or non-Error value is wrapped with the
    original as cause and a string code copied across. Idempotent.
  • isRpcTransportFailure() is the guard; rpcTransportError() builds one.

Applied at every site the research listed: QueuedWriter.close() (marks before
storing closedError, so the active frame, the queue, and later writes share it),
the runPump catch, the child error and stdin error handlers,
createProcessExitError() (covers the exit handler and the exited-child check in
request()), the immediate-startup-exit throw, stop(), failTransport(),
requireWriter(), request()'s not-started throw, and request()'s write catch.
isEngineSendFailure() checks the marker first and keeps the message list only
as a compatibility net for injected test errors. Deliberately not classified:
request timeouts after a successful write, RPC error responses, provider/model
failures, and anything after agent_start.

rpc-client.ts is 492 lines; the helpers live in the new module, as the research
required.

Finding 2 — the literal 500-line rule (unresolved, restated)

No user or supervisor is reachable (intercom lists only idle sibling
subagents), so per the steering contract I again implemented the narrowest
reading — the repository's source-like gate — and state the conflict. The
required Markdown files remain over the limit:
packages/coding-agent/CHANGELOG.md (6,159), docs/extensions.md (2,847),
packages/workflows/CHANGELOG.md (1,427). The research's own analysis is that
splitting the changelog would alter immutable released sections and require a new
archive/runtime format, which is outside this fix. Still needs a user decision.

Iteration 7 validation

  • bun run typecheck, bun run lint, bun run check:file-length (2,547 files),
    bun run --cwd packages/coding-agent docs:check (38 pages) — clean.
  • bun run test:unit — 4,544 pass, 1 skip, 0 fail (593 files).
  • New tests: two writer-level cases in interactive-engine-transport.test.ts
    (coded EPIPE rejects the active and queued frames as classified failures with
    .code and identity intact, pendingBytes === 0, later write() rejects,
    offerLatest() returns false; a frozen ERR_STREAM_DESTROYED is wrapped with
    its cause and code), and three end-to-end cases in
    interactive-prompt-send-failure.test.ts driving a real QueuedWriter over a
    controlled Writable: EPIPE / ERR_STREAM_DESTROYED / ERR_STREAM_WRITE_AFTER_END
    restore /freeze-test\n\n\ntyped during the hang with no red error, a
    post-agent_start failure does not restore, and queued submissions come back
    in FIFO order.
  • Negative checks: removing the isRpcTransportFailure short-circuit fails 3
    tests; removing the writer's marking fails 4.
  • Live tmux on a CLI rebuilt from this tree (dist/cli.js newer than every
    source file), captures after-fix-r6-*.txt:
    • undeclared inline and undeclared overlay: first Ctrl+C closed only the
      component, engine 80399 unchanged both times.
    • declared component: press 1 reached it (presses seen: 1), press 2 closed it
      from the host, engine 80399 unchanged.
    • Kitty release events (\e[99;5:3u, \e[27;1:3u): editor draft intact, engine
      unchanged — no double action.
    • repro A: engine 80399 SIGKILLed under a mounted inline UI → component gone,
      Interactive engine stopped unexpectedly; restarting., replacement 80577,
      typing/Escape/Ctrl+C all worked.
    • repro B: !sleep 500, engine 80633 SIGSTOPped, 7 Escapes over ~35 s kept the
      same PID; Ctrl+C replaced it with 80833 under Interactive engine is not responding; restarting. and returned both unsent submissions to the editor in
      order, with no red error beside them.
    • remapped {"app.clear":"escape"}: three Escapes left the trapped component
      and engine 81455 alone; Ctrl+C closed the component, engine still 81455.
    • !env | grep -c ATOMIC_INTERACTIVE_ENGINEENGINE_ENV_CLEAN.
    • No capture contains Engine terminated, result unknown,
      Error: write EPIPE, Error: Agent process stopped, or an abort timeout.
  • A live broken-pipe race was not attempted; the research says the deterministic
    controlled-writer test is the right pin for the EPIPE case.

@flora131

Copy link
Copy Markdown
Collaborator Author

Implementation notes (full contents, part 2 of 2)

Continuation: iterations 8-10.


Iteration 8 — round-seven findings

Contract amendments received

None. The research artifact's own "Contract amendments received" section reads
"None", and no user message arrived this iteration.

Finding 1 — idle Alt+Enter lost the raw draft (fixed)

Alt+Enter is an app action, so CustomEditor.handleInput() returns before its
own pre-trim snapshot runs and the submit handler's only draft is the callback
argument. The idle branch passed the already-trimmed text, so the carrier was
built from the wrong bytes.

Fix is the research's one-liner in interactive-process-lifecycle.ts: pass
rawText. The submit handler already trims for delivery and keeps the argument
as the fallback draft, so the agent still receives the normalized prompt while a
failed send restores the exact expanded buffer. No new API, no snapshot state, no
change to interactive-input-handling.ts (498 lines).

Checked-in regression: "idle Alt+Enter restores the exact expanded buffer when
the send fails" in test/unit/interactive-submit-send-failure.test.ts, driving
handleFollowUp → setupEditorSubmitHandler → InteractiveSubmission →
runUserPromptTurn → restore with a classified rpcTransportError.

Finding 3 — handlesCtrlC migration placement (fixed)

Added a ### Breaking Changes bullet stating the required caller action, trimmed
the ### Added entry to the option's existence with a pointer, and left the
### Fixed entry explaining the host behavior. The runtime default was NOT
changed back to first-press forwarding — that would restore the trapped-first-press
behavior earlier rounds rejected.

Finding 4 — first-party MCP migration (fixed)

/mcp setup, the main /mcp panel, and the MCP OAuth panel now mount with
handlesCtrlC: true; both panel classes consume Ctrl+C for cancel and cleanup.
New test/unit/mcp-panel-ctrlc-declaration.test.ts drives all three real command
entry points, asserts the recorded options, and feeds Ctrl+C into the mounted
panel so the declaration is tied to the behavior it protects. Added an MCP
[Unreleased] ### Fixed entry and named the MCP panels in the docs migration note.

Finding 2 — the literal 500-line rule (unresolved, restated)

Unchanged and still blocking under the literal wording. No user or supervisor is
reachable (intercom peers are idle sibling subagents), so the narrowest reading
stands: the repository's source-like gate. Oversized touched Markdown is now
packages/coding-agent/CHANGELOG.md, docs/extensions.md,
packages/workflows/CHANGELOG.md, and — new this iteration —
packages/mcp/CHANGELOG.md (805 lines). Needs a user decision.

Deferred (recorded, not implemented)

  • Carrying { text, draft } from EarlyInputState so startup-replayed commands
    can restore surrounding whitespace. The research notes early input is already
    trimmed at main-early-input.ts:49-52, so { text, draft: text } is honest
    today; fixing it is a separate early-input change, not another onSubmit
    workaround.
  • Bootstrap directory-path unlink hardening; typed RpcTransportError class
    replacing the marker; setStatus/setTitle generation ownership.

Iteration 8 validation

  • bun run typecheck, bun run lint, bun run check:file-length (2,548 files),
    bun run --cwd packages/coding-agent docs:check (38 pages) — clean.
  • bun run test:unit — 4,548 pass, 1 skip, 0 fail (594 files).
  • Negative checks: reverting the Alt+Enter call site to text fails the new test
    with actual "draft with outer spaces" vs expected " draft with outer spaces ";
    removing handlesCtrlC: true from the three MCP mounts fails all three MCP tests.
  • Live tmux on a CLI rebuilt from this tree (dist/cli.js 00:39, newer than every
    source file; the bundled dist/builtin/mcp/commands.ts carries the
    declarations), captures after-fix-r7-*.txt:
    • remote UI matrix on one healthy engine 93261, unchanged throughout: undeclared
      inline and undeclared overlay each closed on the first Ctrl+C; declared inline
      and declared overlay each received press 1 (presses seen: 1) and were closed
      by the host on press 2.
    • /mcp (setup panel, no config present) mounted and closed on the first Ctrl+C
      with engine 93261 untouched.
    • workflow graph overlay opened from /workflows and closed on Ctrl+C, engine
      93261 untouched.
    • Kitty release events (\e[99;5:3u, \e[27;1:3u) and two Escapes left the
      draft and engine 93261 alone.
    • repro A: engine 93261 SIGKILLed under a mounted inline UI → component gone,
      Interactive engine stopped unexpectedly; restarting., replacement 95083,
      typing worked.
    • idle Alt+Enter: engine 95253 SIGSTOPped, alt enter spaced draft
      submitted with Alt+Enter, Ctrl+C failed the send → the draft returned to the
      editor with its leading spaces intact (❯ alt enter spaced draft) and no
      red error; replacement 95320.
    • repro B: !sleep 500, engine 95320 SIGSTOPped, 7 Escapes over ~35 s kept the
      same PID; Ctrl+C replaced it with 96074 and returned the unsent submission.
    • remapped {"app.clear":"escape"}: three Escapes left the trapped component
      and engine 96348 alone; Ctrl+C closed the component, engine still 96348.
    • !env | grep -c ATOMIC_INTERACTIVE_ENGINEENGINE_ENV_CLEAN.
    • No round-seven capture contains Engine terminated, result unknown,
      Timeout waiting for response, Error: Agent process stopped, or
      Error: write EPIPE.
  • Unverified by the live run: trailing whitespace in the restored Alt+Enter draft.
    A tmux pane capture pads every line, so trailing spaces are not observable; the
    checked-in unit test asserts the exact string including them.
  • The MCP live check shows the panel closing on the first Ctrl+C, which looks the
    same whether the panel or the host closed it. The ownership discriminator is the
    unit test, which asserts the mount options and runs the panel's own handler.

Iteration 9 — round-eight findings

Contract amendments received

None. The research artifact's own "Contract amendments received" section reads
"None", and no user message arrived this iteration.

Finding 1 — quiet accepted work was restored as unsent (fixed)

Ownership was inferred from the first output chunk, so !touch marker && sleep 400 — side effects, no output — was indistinguishable from a command the child
never received. Implemented the research's correlated admission design:

  • protocol: new engine_request_accepted { requestId, command }, parsed and
    validated; INTERACTIVE_ENGINE_PROTOCOL_VERSION bumped 1 → 2 so a mismatched
    pair cannot silently fall back to the unsafe policy;
  • child: createRpcInputLineHandler emits it and awaits
    waitForRawStdoutBackpressure() before handleCommand, wired only for
    interactive-engine children;
  • host: new rpc-pending-requests.ts owns registration, admission marking,
    resolution and rejection; an accepted request is rejected with a per-request
    copy (asAcceptedRequestFailure) that keeps code/errno/syscall and the
    original as cause, never mutating the shared exit error;
  • exit ordering: jsonl.ts gained an onDrained callback and the exit path
    rejects only after that drain (bounded by EXIT_DRAIN_TIMEOUT_MS), so a
    queued admission frame is never missed;
  • restoration: isEngineSendFailure() now returns
    isRpcTransportFailure(error) && !isRpcRequestAcceptedFailure(error),
    decisively, so the legacy message list cannot reclassify an accepted failure;
  • interactive-bash-compact.ts lost its acknowledged heuristic.

rpc-client.ts ended at 497 lines: pending-request ownership, the stderr bound
(appendBoundedStderr), and a shared failGeneration helper moved out or were
deduplicated to make room.

Finding 2 — shutdown did not fence restart (fixed)

  • RpcClient gained a restart permit: stop() bumps restartRevision even when
    no process is attached (that is exactly the window), restart() takes the
    permit, calls the new private stopCurrentGeneration(), and throws
    RESTART_CANCELLED_MESSAGE if the permit was voided before it spawns.
  • EngineHealthController.shutdown() is now an idempotent promise: it sets
    stopped, clears rescue/latch state, stops the child, and joins the in-flight
    attempt and termination. recover() checks stopped before attempt, and a
    cancelled attempt reports no failure diagnostic and arms nothing.
  • IsolatedInteractiveRuntime.dispose() awaits it before stopping the client.

Finding 3 — the literal 500-line rule (unresolved, restated)

Unchanged. No user or supervisor is reachable, so the narrowest reading stands:
the repository's source-like gate, green at 2,549 files. Oversized touched
Markdown remains packages/coding-agent/CHANGELOG.md, docs/extensions.md,
packages/workflows/CHANGELOG.md, and packages/mcp/CHANGELOG.md. The literal
all-touched-file criterion is not satisfied and needs a user decision.

Tests changed rather than added

Two live death-teardown tests asserted the OLD behavior — that /freeze-inline
and /freeze-overlay came back as drafts after the engine died. Those commands
were accepted and had already mounted their UI, so under the new ownership rule
they must not be offered back; both now assert an empty editor. The bash
"output proves acceptance" test was replaced by admission-based cases.
engine-monitor-readiness.test.ts hard-coded protocolVersion: 1 and now
imports the constant.

Iteration 9 validation

  • bun run typecheck, bun run lint, bun run check:file-length (2,549 files),
    bun run --cwd packages/coding-agent docs:check (38 pages) — clean.
  • bun run test:unit — 4,560 pass, 1 skip, 0 fail (597 files).
  • New tests: interactive-engine-request-admission.test.ts (real engine: quiet
    accepted bash not restorable, undelivered send still restorable, admit-then-die
    fixture), rpc-pending-requests.test.ts (mixed accepted/unaccepted rejection,
    drain ordering, drain timeout, stale drain, response settlement),
    interactive-engine-shutdown-fence.test.ts (shutdown joins the attempt and
    reports no failure, post-shutdown death starts nothing, real-client stop during
    restart spawns no replacement).
  • Negative checks: disabling the child's admission emission fails 2 admission
    tests; removing the restart permit check fails the real-client fence test;
    making shutdown() a no-op promise fails the join test.
  • Live tmux on a CLI rebuilt from this tree (dist/cli.js 01:49, newer than
    every source file), captures after-fix-r8-*.txt:
    • silent bash: !touch /tmp/atomic-repro/silent-marker && sleep 400, marker
      created, no output, engine 23103 SIGKILLed → the editor came back EMPTY (the
      command was not offered back) and the failure was reported as
      Bash command failed: Agent process stopped; replacement 23128.
    • remote UI matrix on healthy engine 23128, unchanged throughout: undeclared
      inline and overlay closed on the first Ctrl+C; declared inline and overlay
      took press 1 (presses seen: 1) and were host-closed on press 2.
    • repro A: engine 23128 SIGKILLed under a mounted inline UI → component gone,
      recovery notice, replacement 23230, typing worked; Escape and a Kitty release
      event left the draft and PID alone.
    • repro B: !sleep 500, engine 23276 SIGSTOPped, 7 Escapes over ~35 s kept the
      same PID; Ctrl+C replaced it with 23342.
    • dispose race: engine killed to start automatic recovery, then quit ~0.3 s
      later; host 23274 exited and no --mode rpc child survived.
    • remapped {"app.clear":"escape"}: three Escapes inert, Ctrl+C closed the
      component, engine 23407 unchanged; /mcp panel closed on the first Ctrl+C
      with the same engine; ENGINE_ENV_CLEAN.
    • No capture contains Engine terminated, result unknown, a
      Timeout waiting for response, or a restored draft beside an error. The two
      Agent process stopped errors present are the intended reports for accepted
      work, with an empty editor beside them.
  • Unverified: the drain barrier is pinned only by the unit test. The real
    admit-then-die fixture passes with and without the deferral, because Bun
    flushes the child's stdout before exit in that scenario; a filler large
    enough to defer parsing trips the interactive 256 KB/turn frame cap instead.

Iteration 10 — round-nine findings

Contract amendments received

None. The research artifact's own "Contract amendments received" section reads
"None", and no user message arrived this iteration.

Finding 1 — automatic restart destroyed queued admission state (fixed)

Death publishes immediately (unchanged — the TUI must not wait on a pipe), but
the generation now enters a bounded terminal phase before it is retired:

  • new rpc-terminal-drain.ts. claim() records the owning cause without
    starting the wait; begin() starts or joins the phase and settles exactly once
    with the claimed error; observe() consumes frames from the dead generation
    and honours only engine_request_accepted; settled() lets callers join.
  • every terminal cause routes through one failGeneration() — exit, spawn
    error, stdin error, and malformed transport — which begins the phase and then
    publishes death.
  • stopCurrentGeneration() claims BEFORE terminating (so a deliberate stop keeps
    Agent process stopped and an exit that already claimed keeps its own error),
    and only after settlement does it detach the reader, retire the generation, and
    let the replacement start.
  • handleLine() gives the drain first refusal, so ownership frames land even
    after the monitor is gone, and no other dead-generation frame reaches host
    state.
  • jsonl.ts now reports completion exactly once for every ended stream,
    including one that ends with nothing buffered (previously that wait could only
    end on the timeout).

rpc-client.ts finished at 496 lines: the exit handler folded into
failGeneration, failTransport delegates to it, and userBashWithUpdates
moved to rpc-client-waits.ts as runUserBashWithUpdates.
RpcPendingRequests.rejectAllAfterDrain was removed as dead code.

Finding 2 — the literal 500-line rule (unresolved, restated)

Unchanged and still unmet under the literal wording. No user or supervisor is
reachable, so the repository's source-like gate stands as the narrowest reading;
it is green at 2,554 files. packages/coding-agent/CHANGELOG.md,
docs/extensions.md, packages/workflows/CHANGELOG.md, and
packages/mcp/CHANGELOG.md remain over 500 lines. Needs a user decision.

Iteration 10 validation

  • bun run typecheck, bun run lint, bun run check:file-length (2,554 files),
    bun run --cwd packages/coding-agent docs:check (38 pages) — clean.
  • bun run test:unit — 4,567 pass, 1 skip, 0 fail (598 files).
  • New tests: the round-nine regression an admitted command is never restored when automatic recovery starts before dead stdout drains (backlog fixture with
    ~1.6 MB of frames written synchronously ahead of the admission, a real side
    effect, a descendant holding stdout, and recovery wired to the death event);
    an explicit stop preserves ownership of a request the child had taken;
    rpc-jsonl-drain.test.ts (empty end, buffered end, multi-turn backlog,
    unterminated final frame); drain cases in rpc-pending-requests.test.ts
    (first-cause-wins, join, ownership-only observe).
  • Negative checks, each failing when reverted: pre-fix stop ordering (detach and
    re-reject before the drain) relabels the exit error; putting admission behind
    the stale-generation guard discards it; removing the empty-stream onDrained
    hangs the drains into their timeout.
  • One behavior correction found by the suite: with the drain in place, the
    pending abort in interactive-engine-abort-lifetime.test.ts was settling with
    the exit error instead of Agent process stopped. Fixed by having the stop
    claim the cause before terminating, which also matches the research's
    "first cause wins" rule.
  • Live tmux on a CLI rebuilt from this tree (dist/cli.js 03:13, newer than
    every source file), captures after-fix-r9-*.txt:
    • admission race: !touch /tmp/atomic-repro/r9-marker && sleep 400, marker
      created, engine 44151 SIGKILLed → editor came back EMPTY, replacement 44178.
    • remote UI matrix on engine 44178, unchanged throughout: undeclared inline and
      overlay closed on press 1; declared inline and overlay took press 1
      (presses seen: 1) and were host-closed on press 2.
    • repro A: engine 44178 SIGKILLed under a mounted UI → component gone, recovery
      notice, replacement 44279, typing worked, Escape and a Kitty release inert.
    • repro B: !sleep 500, engine 44321 SIGSTOPped, 7 Escapes over ~35 s kept the
      PID; Ctrl+C replaced it with 44606.
    • dispose during recovery: engine killed, quit ~0.3 s later; host exited with no
      children and zero stray --mode rpc processes.
    • remapped {"app.clear":"escape"}: Escapes inert, Ctrl+C closed the component,
      engine 44652 unchanged; /mcp panel and the workflow graph each closed on
      Ctrl+C with the same engine; ENGINE_ENV_CLEAN.
    • No round-nine capture contains Engine terminated, result unknown, a
      Timeout waiting for response, or a restored draft beside an error.
  • Not separately reproduced live: process-error and stdin-error terminal
    causes. They share the single failGeneration path with exit, which the unit
    tests cover; I did not construct a live spawn failure.

@mintlify

mintlify Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
bastani 🟢 Ready View Preview Jul 30, 2026, 10:46 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@mintlify

mintlify Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
bastani 🟡 Building Jul 30, 2026, 10:45 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@flora131
flora131 marked this pull request as ready for review July 30, 2026 15:46
@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown

Too many files changed for review. (117 files found, 100 file limit)

Bypass the limit by tagging @greptile-apps to review.

flora131 added 11 commits July 30, 2026 20:56
… input

Escape used to race the engine's cooperative abort against a 250 ms deadline
and kill plus restart the engine child whenever it lost, reporting
"Engine terminated; … result unknown; inspect side effects before retrying".
Escape now only requests the engine's own cancellation and waits for it, and
that message no longer exists in any code path.

Terminating a wedged engine is an explicit Ctrl+C, armed only when the engine
is provably not answering: watchdog-confirmed unresponsive, a cooperative abort
unanswered past the same one-second threshold, or a replacement still waiting
for readiness past it. That last case has no heartbeat and no watchdog
coverage, so it was previously unrecoverable. Ctrl+C reaches the host even
while an engine-owned custom UI or overlay owns input, and host-native
selectors, dialogs, and forms keep Ctrl+C-as-cancel while the engine is
healthy.

Engine death is now a host-local lifecycle event. The host closes every remote
component from the dead generation, settles its ui.custom() promises, releases
widget keys, resets terminal modes, remounts and refocuses the editor, and
unwinds the blocking inline custom-UI depth, without waiting for a replacement
engine_ready. One automatic replacement attempt follows, with calm status text.

The engine child is launched with an environment that never contains the four
engine control values; they travel in an owner-only bootstrap file read once
and unlinked. Deleting them from process.env afterwards cannot work under Bun,
where a child spawned without an explicit env inherits the runtime's
launch-time environment.

A submission the engine never accepted returns to the editor, ahead of anything
typed while the send was pending, on every submit route including /atomic,
deferred commands, compaction-time extension commands, streaming steer, bash,
/compact, and Alt+Enter.

Assistant-model: Claude Opus 5
…ery repeatable

Ctrl+C is now always handled by the host while an engine-owned ctx.ui.custom()
component or overlay holds input, even with a healthy engine: such a component
forwards every key to the child, so one that never resolves trapped the escape
key too. Ownership is answered exactly by the remote component controller
rather than inferred from overlay presence or inline depth, so native
selectors, dialogs, input forms, session pickers, and unrelated native overlays
keep Ctrl+C as their own cancel. That path reports "Restarting interactive
engine." instead of falsely calling a healthy engine unresponsive.

Escape's wait is now genuinely unbounded: abort joined LONG_LIVED_COMMANDS, so a
stopped or blocked child no longer turns Escape into a red timeout after 30
seconds while the engine is still working.

A replacement that fails on its own now latches Ctrl+C armed, so recovery stays
available without Atomic ever retrying on its own.

RPC select, confirm, input, and editor dialogs are owned by the generation that
opened them: engine death cancels exactly those mounts and suppresses their
replies, so a dead generation can no longer leave a dialog on screen or answer
through the replacement child. Dialog hides are instance-scoped so stale
cleanup cannot dismiss a newer dialog.

Remote mounts unwind newest-first, each overlay is hidden through its own
handle, and the editor is refocused only when no surviving modal owns input.

Draft restoration reads expanded editor text, so a pending large paste survives
instead of being reduced to a dead marker.

Bootstrap cleanup is ownership-scoped: the child unlinks exactly the file named
on the command line, recursive removal requires the handle the host received
when it created the directory, and a failed publication removes its own
temporary credential file and directory before rethrowing.

Assistant-model: Claude Opus 5
…e targeting

The newest-first unwind and the exact-overlay-handle close were implemented but
only guarded by end-state assertions in the live harness, which passes either
way because pi-tui skips unmounted focus targets.

The fake host bridge now records the order in which each mount's host close
callback runs, so an inline proxy with an overlay stacked above it asserts the
overlay closes first. A new suite drives the real showExtensionCustom to assert
that an overlay is hidden through its own handle rather than the generic
top-overlay call, and that an inline close never takes focus from a surviving
overlay.

Both fail when the corresponding fix is reverted.

Assistant-model: Claude Opus 5
…covery armed

Escape and Ctrl+C were classified through the configurable app.clear action, so
binding app.clear to Escape sent Escape into the engine stop/restart branch and
left Ctrl+C with no host route. Both are now matched by physical key identity
through pi-tui's parser, with key-release events filtered, and the editor
applies the same fixed Escape guard before its configurable handlers.

A focused remote proxy now receives the first Ctrl+C, so extension UIs that bind
it keep working; the host takes the next press against the same component. That
preserves the workflows prompt-card Skip and stage-chat Close while keeping the
escape hatch.

Every cooperative-cancellation command is exempt from the generic request
deadline, not just abort, so cancelling a running bash no longer produces a red
timeout while the engine is still working.

A child dying while a replacement is starting latches Ctrl+C instead of being
ignored, so the host is never left with no engine and nothing armed.

Line widgets are generation-owned, so a dead generation's lines are released
while a newer generation's content survives stale cleanup.

The editor snapshots its expanded buffer for the dispatch that submits it, so a
restored draft is what was typed rather than the trimmed callback argument, and
a restored draft no longer also raises a red transport error.

Heartbeats moved off the consumptive generic engine-message channel, and the
queued writer rejects its in-flight frame so a callback-level EPIPE cannot hang
a prompt forever.

Assistant-model: Claude Opus 5
…mote UI

A remote ctx.ui.custom() component owns every key while it holds input, so a
component that never resolves swallowed Ctrl+C. The previous escape needed two
presses and then replaced the whole engine, discarding everything else that
generation was doing.

Ownership is now declared per mount through a new handlesCtrlC option. An
unresponsive engine is still terminated on the first press, since a wedged child
cannot run a local handler either. A component that declared the option keeps
its own Skip, Close, or cancel binding, and is closed if it still owns input on
the next press. An undeclared component is closed on the first press through the
ordinary close path, so its promise resolves with undefined and the engine keeps
running. The bundled workflow surfaces declare it.

Engine death is retained rather than transient, so a child that exits between
startup returning and the host attaching is still recovered instead of leaving a
live TUI bound to a dead engine.

Buffered custom-UI frames and extension UI requests are tagged with the
generation that produced them and dropped when it dies, so a stale mount frame
cannot remount UI that death teardown just closed, or collide with the
replacement child's identical component ids.

Each submission carries its own raw draft end to end instead of sharing one
slot, so two entries that differ only in whitespace can no longer restore each
other's text, and submissions still queued behind a failed send come back with
it in the order they were entered.

Assistant-model: Claude Opus 5
…ror text

A submission that the engine never accepted is still the user's text, but the
host decided that by matching five message fragments. A dying engine produces
write EPIPE, Cannot call write after a stream was destroyed, or write after end,
none of which matched, so those submissions were reported as red errors and the
typed text was thrown away. Node also documents error.message as free to change
in any release.

The transport boundary already knows the frame never landed, so it says so. A
non-enumerable marker is added to the existing error, leaving its identity,
instanceof, code, errno, and syscall untouched, because RpcClient rejections are
public. A frozen or non-Error value is wrapped with the original as cause.

Marking happens in the queued writer, the child exit, error, and stdin handlers,
an explicit stop, a malformed transport, a missing writer, a not-started client,
and the request write catch. Request timeouts after a successful write, RPC
error responses, provider failures, and anything after agent_start stay
unclassified and still surface.

Assistant-model: Claude Opus 5
…CP panels

Alt+Enter is an app action, so the editor returns before its own pre-trim
snapshot runs and the submit handler's only draft is the value the Alt+Enter
path hands over. That value was the already-trimmed text, so an idle follow-up
the engine never accepted came back without the whitespace the user typed. The
raw expanded buffer is passed instead; the handler still trims it for delivery.

The /mcp, /mcp setup, and MCP OAuth panels bind Ctrl+C for their own cancel and
cleanup, so they now declare handlesCtrlC. Without it the host would close them
on the first press and skip their handlers.

The changelog gains a Breaking Changes bullet for the migration existing
ctx.ui.custom components need: an extension that consumes Ctrl+C keeps that
binding only by declaring handlesCtrlC. The runtime default is unchanged, since
forwarding an undeclared first press is exactly the trap this work removed.

Assistant-model: Claude Opus 5
…e disposal

Ownership of a failed submission was inferred from the first byte of output, so
a command that changes the working tree and prints nothing looked exactly like
one the child never received. Killing the engine during `!touch marker && sleep
400` put that line back in the editor, inviting a second run.

The child now announces ownership of every correlated request and flushes the
announcement before its handler can touch the shell, an extension, the queue, or
compaction. The host restores a draft only for a transport failure that arrived
without an announcement, and reports an accepted failure as an ordinary failure.
The exit rejection waits for the dead child's stdout to finish parsing so a
queued announcement is never missed, bounded so a descendant holding stdout
cannot strand the caller. The protocol version becomes 2, because a child that
cannot announce must not bind to a host that assumes it can.

Disposal now fences engine recovery. A replacement sits between its own stop and
its spawn with no child attached, so a disposal-time stop found nothing to do
and returned, and the attempt then started an engine after teardown finished.
An explicit stop voids the restart permit, a superseded restart fails as
cancelled instead of quietly succeeding, and health shutdown joins the attempt
before disposal returns.

Assistant-model: Claude Opus 5
…s replaced

An engine child announces that it owns a request before it starts the work, but
that announcement can still be unparsed in the pipe when the child dies. Because
automatic recovery starts in the same turn as the death event, its stop detached
the reader, retired the generation, and re-failed those requests with `Agent
process stopped` — so work that had already run was reported as never sent and
offered back to the user for a second run.

Death still publishes immediately: the TUI must never wait on a pipe. The
generation now gets a bounded settling window on top of it. Its stdout keeps
being read, only its ownership frames are honoured, its requests are classified
exactly once with the error of whatever ended it, and the replacement starts
only afterwards. An explicit stop claims that error before terminating, so a
deliberate stop keeps its own wording while an exit keeps its own.

Every terminal cause now shares one path: exit, spawn error, stdin error,
malformed transport, and explicit stop. The JSONL reader also reports completion
for a stream that ends with nothing buffered, which previously left that wait to
expire on its timeout.

Assistant-model: Claude Opus 5
…the watchdog latch

Three review findings from the recovery branch, plus the leak that surfaced
while reproducing them.

isEngineSendFailure no longer falls back to matching error text. Every
rejection RpcClient raises is already built through rpcTransportError, and an
accepted request is re-marked per request, so admission is decisive on its own.
The legacy marker list could classify any provider, extension, or command error
quoting a phrase like "Agent process stopped" as unsent, restoring a draft the
engine had already run and hiding the real failure. The tests that leaned on
that fallback now inject typed transport errors, which is what production
raises, and a new case pins that identical wording resolves differently based
only on admission.

recover() clears the unresponsive latch when a replacement attempt starts. The
watchdog verdict describes the generation being replaced; a fresh child emits no
heartbeat before engine_ready, so the latch kept needsExplicitTermination() true
through the whole pre-ready window and the first Ctrl+C would stop a replacement
that never misbehaved. The fence stays time-bounded: an overdue replacement arms
Ctrl+C again on its own account.

The abort-lifetime test now resumes and stops its child in a finally block. It
SIGSTOPs the engine, and a failed assertion before the resume leaked a frozen
child holding this process's stdio pipes - the same leak class that hung the
Windows job. One such orphan was still running from an earlier local run.

Assistant-model: Claude Opus 5
Rebasing onto main crossed the pi toolchain-parity change (#2079), which moved
the root suites from bun test to vitest under Node and adopted pi's biome rule
set. This adapts everything the branch added:

- Migrate branch-added tests off bun:test/Bun.* to the vitest API and
  test/helpers/runtime.js (sleep, spawnProcess, spawnSyncCollect, moduleDir,
  bunExecutable), including test.serial -> test.sequential.
- default-main-driver fixture: spawn through the helpers so the fixture host
  still runs under Bun while the suite runs under Node.
- admission-backlog-engine fixture: a Node host hands the child a non-blocking
  stdout pipe, so the synchronous admission burst raised EAGAIN and killed the
  child mid-write; writeAllSync retries until the host drains.
- bootstrap publication-failure test: scope the observed temp root, because
  vitest runs test files in parallel and other files create bootstrap
  directories in the shared tmpdir during the snapshot window.
- Align branch-added sources with the biome config on main, and rewrite the
  two noAssignInExpressions sites (queued-writer pump loop, engine-health
  attempt id) without behavior change.
flora131 added 4 commits July 30, 2026 21:28
…n contracts

The branch changed several InteractiveMode contracts without updating the
coding-agent package suite and one integration helper; CI has been red on
exactly these files since before the rebase. Align them with the intended
behavior:

- Queued prompts and input callbacks now carry InteractiveSubmission
  ({ text, draft }) instead of bare strings, so a failed send can restore the
  exact editor buffer. Updated assertions and stub types in the startup-input,
  first-run-onboarding, status-autocomplete, and paused-queued-messages suites.
- runUserPromptTurn subscribes to the session to detect turn start; session
  stubs in the deferred-startup, startup-latency, loader-continuity, and
  resource-gate suites now provide subscribe().
- restoreFailedSubmissionDraft reads mode.pendingUserInputs; the resource-gate
  fake mode now seeds it.
- IsolatedInteractiveRuntime and the engine dialog host observe generation
  death; client/runtime stubs in rpc-bash-streaming and
  startup-resource-ordering now provide onGenerationEnded().
- The showExtensionCustom focus fence consults ui.hasOverlay() before
  restoring editor focus; the shared integration overlay host helper now
  declares it.

Also fix three load-sensitive tests the full parallel suite exposed while
landing this change, per the repository's fix-it-where-it-lives policy:

- interactive-engine-generation-lifecycle: tolerate ESRCH when the engine
  child loses the startup race and is already gone before the explicit
  SIGKILL; an early death is still the generation death under test.
- subagents-async-event-journal: temp-root cleanup races the journal drain
  these tests deliberately provoke, so recursive removal can hit ENOTEMPTY
  when a final async append lands mid-delete; use rm's bounded retry.
- subagents-foreground-intercom-detach: gate the fake children on a release
  file written only after the detach commit is processed, instead of fixed
  output delays a loaded event loop can outlive; otherwise the commit finds
  the attempt closed, nothing detaches, and the detached-exit wait hangs
  until the suite timeout.
…S temp symlinks

os.tmpdir() on macOS is a /var -> /private/var symlink and jiti resolves
transitive imports to realpaths, so the recorded manifest keyed chain files
under /private/var while the test compared literal /var fixture paths and
reported them missing. Linux and Windows runners have no such symlink, which
is why CI stayed green while every macOS checkout failed these two tests.
…s build

npm ci --ignore-scripts skips lifecycle scripts by design and the workspace
natives package has no install hook, so a fresh clone never compiles
packages/natives/native/*.node; the CLI then silently degrades (pty:true falls
back to pipes, native grep/find and tree-sitter block ops fall back to JS) and
five coding-agent test files fail. Document the explicit
'npm run build --workspace=@bastani/atomic-natives' step, the required Node and
Rust toolchains, the node-run dist path the published bin actually uses, and
the current CI shape. Verified end to end on a fresh checkout: install, natives
build, 'bun packages/coding-agent/src/cli.ts', and build + 'node dist/cli.js'.
Windows has no POSIX permission bits: writeFileSync's mode option maps only to
the read-only attribute and stat reports 0o666 for any writable file, so the
0o600 assertion can never hold there. The record's protection on Windows is the
per-user temp directory ACL. Keep the secret-free path and directory assertions
on every platform.
@flora131
flora131 merged commit 2a6b716 into main Jul 31, 2026
14 of 15 checks passed
@flora131
flora131 deleted the fix/interactive-engine-recovery branch July 31, 2026 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant