fix: prevent reconnect loops during server stalls - #5561
Conversation
Root-cause work for the constant disconnect/resync loop: the client RPC pinger killed the socket after a single missed 5s ping window, while the single-threaded server blocks that window with synchronous SQLite reads, synchronous trace writes, and a per-connect executable scan storm (27k+ shell.isExecutableFile probes observed on one session bootstrap). Reconnecting reset backoff to 1s and re-triggered the same storm, making the loop self-sustaining. - effect patch: tolerate 2 missed ping windows (fail on the 3rd, ~15s of silence) instead of hard-failing after one; keep pinging in between - shell: memoize PATH-scan command resolution per (platform, PATH, PATHEXT, command) for 30s so session bootstrap stops re-scanning; explicit-path probes stay uncached (installers probe just-written files) - observability: clamp db.query.text to 200 chars and any string attribute to 500 in the trace file sink (was 54% of trace bytes, rotating 10MB every 2-4 min and destroying diagnostic history) - supervisor: raise the first reconnect backoff rung from 1s to 3s so a stall-kill cannot cycle at 1s against a still-starved server Verified: client-runtime 510/510, shared suite identical to baseline (5 pre-existing Windows-environment failures, zero regressions), typecheck green for shared/client-runtime/web. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
While a thread runs, the composer replaced Send with Stop on every platform, leaving mid-run steering reachable only through the invisible Enter-to-submit path - and structurally impossible on narrow viewports, where Enter inserts a newline and the collapsed send button was hard-disabled during runs. Steering itself (thread.turn.start during an active turn) has always worked, including over the relay. - render a steer Send button beside Stop while running, using the same form-submit path and the same disable predicate as the normal send - stop hard-disabling the collapsed narrow-viewport send button during phase === "running" - onSend: surface a toast when a message is dropped because the environment is not connected, instead of silently swallowing it; the draft stays in the composer Verified: web typecheck green, composer test suites pass, lint clean on changed files (one unrelated pre-existing test failure baselined via stash against the clean tree). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tabbing back to the app on a dead transport paid a fixed 15s probe timeout plus a 3s backoff sleep before reconnecting even began, then re-downloaded the full shell snapshot over HTTP on every wake - even though the event store already supports cursor resume and threads already use it. - probe timeout 15s -> 3s on desktop/web (mobile already 3s) - skip the first backoff rung when a foreground wake probe fails: the user is actively returning on a dead socket. Only that first attempt skips; a failed immediate reconnect resumes normal backoff, so the reconnect-storm protection stays intact - shell warm resume: subscribe with the cached snapshotSequence cursor instead of always reloading the HTTP snapshot; the server already falls back to a streamed snapshot when the replay gap overflows - memoize editor discovery (resolveAvailableEditors PATH scan, up to 5s) with a 60s TTL above the shared command-resolution cache, so serverGetConfig stops rescanning on every connect - raise SHELL_RESUME_MAX_GAP 1000 -> 5000 (shell replay is coalesced; one busy agent thread while away no longer forces full snapshots) Verified: client-runtime 512/512 (4 new/reworked wake-path tests), client-runtime + server typecheck green, externalLauncher tests 5/5, targeted server subscribeShell tests 7/7. Two pre-existing environmental server test failures (Windows symlink EPERM in setup) unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ApprovabilityVerdict: Needs human review This PR introduces significant runtime behavior changes to connection resilience including retry timing, wake probe handling, caching, and shell synchronization logic across multiple packages. Additionally, there is an unresolved review comment identifying a potential bug in the shell resume fallback path. You can customize Macroscope's approvability policy. Learn more. |
- command resolution cache: expire via the monotonic clock (Clock.currentTimeNanos) instead of wall-clock millis, so a backward system-clock adjustment can no longer keep expired entries alive past the 30s TTL - trace attribute truncation: slice to the per-key limit instead of the hard-coded 200, so general string attributes over 500 chars keep 500 (db.query.text still clamps to 200) Verified: shared typecheck green, shared tests identical to baseline, server externalLauncher memo tests 5/5 (TestClock drives currentTimeNanos consistently with millis). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
One finding: the new command-resolution memo lives in a module-global mutable Map in packages/shared/src/shell.ts rather than being owned by a layer/service, which hides the cache's lifetime from the Effect environment. Everything else reviewed (the Effect.cachedWithTTL memo in ExternalLauncher.make, the supervisor Ref state, the shell-state warm-resume path, and the trace-attribute helpers) follows the service conventions: subpath namespace imports, inline Context.Service interfaces, real make/layer exports, and no new ManagedRuntime/runPromise boundaries or service-instance injection.
Posted via Macroscope — Effect Service Conventions
Follow-ups stacked on the connection-resilience RFC (pingdotgg#5561): - supervisor: the retry ladder was non-monotonic after the first rung was raised to 3s ([3s, 2s, 4s, ...] retried the second failure faster than the first). Drop the stale 2s rung: [3s, 4s, 8s, 16s]. - observability: clamp oversized strings nested inside arrays/objects too (e.g. error stacks preserved by normalizeJsonValue), not just top-level values, so the trace-retention win holds. Adds a unit test. - shell: move the command-resolution cache into the Effect environment as a Context.Reference (same idiom as HostProcessPlatform) so tests and embedders can provide an isolated instance, per the Effect Service Conventions check. Also document the 30s negative-caching window. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Update: rather than a separate PR, I pushed the follow-up fixes directly onto this branch as |
|
@t3dotgg Reviewed
Regression run against this branch tip, on Windows:
For pressure testing: the underlying fixes have been soaking on our fork's hosted deployment since yesterday, on the exact workload that motivated this PR — a relay session against a 24-core laptop drowning in agent threads, operated by an impatient user with a phone. Disconnect-loop symptoms gone; steering works mid-run. Happy to pick up the deferred items (steer-button visual pass, mobile parity) here or in a follow-up, and to file the pinger tolerance upstream on effect. Sent from my Fable 5 |
| const httpSnapshot = yield* snapshotLoader.load(prepared); | ||
| if (Option.isSome(httpSnapshot)) { | ||
| yield* applyItem({ kind: "snapshot", snapshot: httpSnapshot.value }); | ||
| yield* Ref.set(lastAuthoritativeSession, session); |
There was a problem hiding this comment.
Stale shell resume after HTTP fail
Medium Severity
On a new RPC session, makeSubscribeInput tries an authoritative HTTP shell refresh when lastAuthoritativeSession does not match. If that load returns Option.none (the documented “use the socket snapshot instead” path), the code still subscribes with afterSequence from the existing in-memory snapshot left over from cache or the prior session, so resume skips the full socket snapshot the fallback is meant to use.
Reviewed by Cursor Bugbot for commit fb2b8c5. Configure here.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c7aba01. Configure here.
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…adder Upstream changed RETRY_DELAYS_MS from [1s, 2s, 4s, 8s, 16s] to [3s, 4s, 8s, 16s] — a slower first reconnect, part of pingdotgg#5561's work on reconnect loops during server stalls. Two of the fork's issue-#21 tests encoded the old rungs and failed on timing alone; the tolerant-budget behaviour they cover is unchanged. - "keeps the normal budget ... never reached the backend" assumed the first retry was 1s "either way", because RETRY_DELAYS_MS[0] used to equal SLOW_RETRY_DELAY_MS. It no longer does, which makes the test stronger: the normal and tolerant paths now diverge at the first retry, so it asserts nothing has moved at 1s and the attempt starts at 3s. The second retry is retryDelayMs(1) === 4s, not 2s. - "falls back to the normal budget after too many consecutive slow timeouts" waited retryDelayMs(3), which was 8s on the old ladder and is 16s (the top rung) on the new one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both halves of the ledger are regenerated against the same merge-base (`2c7267ad4`) for the first time in three syncs — the file list had been measured at `30c96228` while churn and risk were still in the `64bf01619` window. Surface: 37 upstream-owned files, +1835 / -318. The 113 commits this sync absorbed are now inside the churn window, so every figure moved; ChatView.tsx is 63 → 85. Findings from the runbook's step 3 — reviewing seams upstream touched *without* conflicting — recorded on their rows: - `supervisor.ts` silently dropped upstream's pingdotgg#5561 behaviour. Upstream marks `wakeProbeFailed` when a wake probe fails and reads it to skip the first backoff rung. The fork's shared `runLivenessProbe` never set it, so the Ref was written by nobody and read as always-false. - `ComposerPrimaryActions.tsx` had both its hoists re-derived: upstream grew a size axis on the stop button where the fork had an emphasis axis, and restyled the send button onto the message-action tokens, which the fork's hoisted copy was quietly reverting. - The queue branch gained a `!directAnnotation` exclusion — upstream's new annotation parameter would otherwise have been dropped silently by a text-only enqueue. - Three mirrors re-verified as still faithful: `hasOpenBlockingRequest` (byte-identical), `authenticateWithOperateScope`, and the steering allowlist — which is now backed by upstream's own adapter tests rather than by reading. Still open, deliberately: the `getSnapshot()` boot-path OOM risk, unchanged since 2026-08-02 and still wanting its own PR with coverage; and the fork's notification row remains absent from upstream's settings-search catalog. Per the ledger's self-reference rule this would ideally ride with each seam edit; a sync regenerates the whole ledger at the end instead, so it lands once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…adder Upstream changed RETRY_DELAYS_MS from [1s, 2s, 4s, 8s, 16s] to [3s, 4s, 8s, 16s] — a slower first reconnect, part of pingdotgg#5561's work on reconnect loops during server stalls. Two of the fork's issue-#21 tests encoded the old rungs and failed on timing alone; the tolerant-budget behaviour they cover is unchanged. - "keeps the normal budget ... never reached the backend" assumed the first retry was 1s "either way", because RETRY_DELAYS_MS[0] used to equal SLOW_RETRY_DELAY_MS. It no longer does, which makes the test stronger: the normal and tolerant paths now diverge at the first retry, so it asserts nothing has moved at 1s and the attempt starts at 3s. The second retry is retryDelayMs(1) === 4s, not 2s. - "falls back to the normal budget after too many consecutive slow timeouts" waited retryDelayMs(3), which was 8s on the old ladder and is 16s (the top rung) on the new one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both halves of the ledger are regenerated against the same merge-base (`2c7267ad4`) for the first time in three syncs — the file list had been measured at `30c96228` while churn and risk were still in the `64bf01619` window. Surface: 37 upstream-owned files, +1835 / -318. The 113 commits this sync absorbed are now inside the churn window, so every figure moved; ChatView.tsx is 63 → 85. Findings from the runbook's step 3 — reviewing seams upstream touched *without* conflicting — recorded on their rows: - `supervisor.ts` silently dropped upstream's pingdotgg#5561 behaviour. Upstream marks `wakeProbeFailed` when a wake probe fails and reads it to skip the first backoff rung. The fork's shared `runLivenessProbe` never set it, so the Ref was written by nobody and read as always-false. - `ComposerPrimaryActions.tsx` had both its hoists re-derived: upstream grew a size axis on the stop button where the fork had an emphasis axis, and restyled the send button onto the message-action tokens, which the fork's hoisted copy was quietly reverting. - The queue branch gained a `!directAnnotation` exclusion — upstream's new annotation parameter would otherwise have been dropped silently by a text-only enqueue. - Three mirrors re-verified as still faithful: `hasOpenBlockingRequest` (byte-identical), `authenticateWithOperateScope`, and the steering allowlist — which is now backed by upstream's own adapter tests rather than by reading. Still open, deliberately: the `getSnapshot()` boot-path OOM risk, unchanged since 2026-08-02 and still wanting its own PR with coverage; and the fork's notification row remains absent from upstream's settings-search catalog. Per the ledger's self-reference rule this would ideally ride with each seam edit; a sync regenerates the whole ledger at the end instead, so it lands once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: t3-turbo-simulation <simulation@t3turbo.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Theo Browne <me@t3.gg>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rebuilt on main after #5561 and #5404 landed overlapping reconnect work. What main already covers is dropped from this branch: the backoff ladder rework replaces our jitter, wakeProbeFailed replaces our probe-failure fast path, and the authoritative-session guard replaces our shell warm-cache short-circuit. What remains is the mobile-specific delta: - New advisory `network-path-changed` wakeup: WiFi<->cellular keeps isConnected true while invalidating the socket's path, so the mobile adapter emits a wakeup on interface-type changes while active and the supervisor probes the session (3s bound) instead of waiting for the ping timeout. A failed probe rides main's wakeProbeFailed fast path; flapping interfaces cannot cut backoff delays short. - Client activity is re-reported on every newly connected session generation; the AppState-triggered report races the reconnect and is dropped, leaving provider/VCS work paused server-side for up to 25s after a resume. Generation dedup is per-supervisor since replacements restart the counter. - The RPC onPingTimeout hook is wired to logging so zombie-socket disconnects are distinguishable from ordinary closes. - Server websocket idleTimeout drops to 30s from Bun's 120s default (clients ping every 5s, so live connections never idle), releasing a suspended phone's half-dead socket and buffers four times sooner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: codex <codex@users.noreply.github.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native subagent observability (#5219), wired per its spec's v2 merge plan: - getWorkflowScript RPC re-homed onto the v2 WS surface (contracts, rpc group, ws handler, auth scope, client atom). - AgentsPanel fed by the spec's mapper swap: projectedSubagentsToRuntime maps orchestration-v2 subagent entities into the panel model; deriveAgentPanelModel's v2Projection leg is now live and the v1 fold never runs. Agents surface wired into ChatView + RightPanelTabs. Other ports and reconciliations: - Shell reconnect-loop fix (#5561) ported into the v2 shell sync (same-session resubscribes resume from the in-memory cursor), with the cursor-resume regression test adapted to v2 fixtures. - Mobile end-follow latch (#5566) ported onto the v2 ThreadFeed. - Claude ede_diagnostic interrupt classification (#5557) ported into ClaudeAdapterV2 (aborted_tools/aborted_streaming => interrupted; CLI telemetry never becomes the failure banner). #5559 needs no v2 port (unknown system subtypes are already ignored). - Plan sidebar removed from the v2 ChatView/ChatComposer per main's plans-fold-into-chat rework (#5558); rightPanelStore stays at main's surface set. - SettingsPanels rebuilt as main's refactored version plus the branch's composer-context setting; sidebar snooze respects the time format (#4438 follow-through). - v1-only leftovers deleted: zombie v1 adapters/ingestion/tests the v2 rewrite removes, the v1-bound transfer-budget CI harness (#5350, needs a v2 rebuild), and main's v1 client pagination machinery (#5493 client side; the 037 keyset migration is kept — server-side v2 windowing is a follow-up). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>


Server event-loop stalls could make one delayed pong kill the RPC connection, then immediately reconnect into the same expensive bootstrap work. Users saw repeated disconnects and
Syncing...even when the underlying network was healthy.This tolerates short pong delays, uses a safer reconnect ladder, and avoids repeated bootstrap work by caching command and editor discovery and resuming shell state from an authoritative cursor. Failed HTTP snapshot refreshes still request a complete socket snapshot, trace attributes are bounded to preserve useful diagnostics, and sends attempted while disconnected now explain why they were not sent.
Focused tests cover pinger tolerance, wake recovery, shell synchronization, discovery caching, and trace truncation.
🤖 Final review fixes and PR cleanup by GPT-5.6 Codex using the Codex harness in T3 Code.
Note
Medium Risk
Changes core connection supervisor, RPC protocol (patched dependency), and shell subscription resume semantics; mis-tuned wake/backoff or resume cursors could cause extra sync work or delayed reconnects.
Overview
Hardens client–server connection behavior against event-loop stalls, aggressive reconnect loops, and expensive session bootstrap.
Transport & supervisor: Patches Effect RPC ping/pong to allow two missed 5s windows before closing (~15s silence). Raises the first reconnect delay from 1s → 3s and, when a foreground wake probe fails, skips the first backoff sleep once (then normal backoff resumes). Supervisor and session tests are updated for the new timing and wake behavior.
Connect cost: Adds a 30s PATH command-resolution cache in shared shell code and 60s memoized editor discovery on the server so repeat connects avoid huge filesystem scans. Trace file sink now clamps oversized span attributes (including
db.query.text) without mutating live spans.Shell sync: On the same live RPC session, foreground resubscribe resumes from the in-memory
snapshotSequenceinstead of always forcing a full HTTP refresh; failed HTTP refresh falls back to a full socket snapshot. New/rewritten shell-sync tests cover these paths.Web: Sending while the environment is unavailable shows a warning toast (draft preserved) instead of silently attaching an annotation.
Reviewed by Cursor Bugbot for commit c44dad4. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Harden connection resilience by fixing idle disconnects with backoff, pinger tolerance, and shell cursor reuse
EnvironmentSupervisorretry ladder first rung from 1s to 3s; a failed foreground wake probe now triggers an immediate reconnect instead of waiting for the first backoff sleepExternalLauncher(60s TTL) and command path resolution inshell.resolveCommandPathForPlatform(30s TTL) to reduce redundant filesystem scansMacroscope summarized c44dad4.