fix(cloud-agent-next): recover stalled ingest reconnects and stuck preparing UI - #5447
Conversation
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Executive SummaryIncremental change scopes preparation settlement to observed attempts and the workspace-ready boundary; the previous synthesized-failure issue is resolved. Files Reviewed (5 files)
Previous Review Summaries (2 snapshots, latest commit 4cfd380)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 4cfd380)Status: 1 Issue Found | Recommendation: Address before merge Executive Summary
Overview
Issue Details (click to expand)WARNING
Files Reviewed (11 files)
Fix these issues in Kilo Cloud Previous review (commit 35c0b7a)Status: 2 Issues Found | Recommendation: Address before merge Executive Summary
Overview
Issue Details (click to expand)WARNING
Files Reviewed (7 files)
Reviewed by grok-4.6 · Input: 99.5K · Output: 13.7K · Cached: 668.5K Review guidance: REVIEW.md from base branch |
…eparing UI Wrapper reconnects no longer stop after 3 attempts. Hung reconnect handshakes wait 90s so a blocked Durable Object can still accept the in-flight socket instead of leaving zombie ingest connections. Retrying is bounded by wall clock rather than attempt count — a stale run/ connection fence is rejected with 409 forever and a failed WebSocket upgrade exposes no status to classify — so the wrapper still reports a disconnect and aborts its Kilo turn instead of retrying into a socket nobody will accept. Early preparation failures now emit a terminal attempt, and the SDK clears a stale preparing composer status when delivery fails. A held delivery is excluded from both: the message stays queued and is retried moments later, so inventing a failed attempt for it would flash a spurious preparation failure. A hold raised after preparation already started still terminalizes its attempt. An interrupt clears the stale preparing status rather than raising an error banner, since the user cancelling is not a failure.
35c0b7a to
4cfd380
Compare
pandemicsyn
left a comment
pandemicsyn
left a comment
There was a problem hiding this comment.
Not positive on this one, i think the bot finding is correct. At services/cloud-agent-next/wrapper/src/connection.ts:1351, scheduleReconnect() checks the five-minute RECONNECT_TOTAL_BUDGET_MS and, once elapsed, stops the campaign and calls onDisconnect. The main callback then aborts the active session and closes/drains the wrapper, so an ingest outage that lasts past the budget cannot reconnect later within that turn i think ? Which i guess now that i write it out loud is actually ok ?
prepapproved either way
| if (reconnectAttempt > MAX_RECONNECT_ATTEMPTS) { | ||
| logToFile(`reconnection failed after ${MAX_RECONNECT_ATTEMPTS} attempts — giving up`); | ||
| const elapsed = Date.now() - reconnectStartedAt; | ||
| if (elapsed >= RECONNECT_TOTAL_BUDGET_MS) { |
There was a problem hiding this comment.
bot: The reconnect loop still gives up after five minutes instead of retrying until close().
Suggested fix: Do not terminate the reconnect campaign based on RECONNECT_TOTAL_BUDGET_MS. Keep the capped backoff retry loop active until close() cancels it, and replace the budget-exhaustion unit test with coverage that verifies retries remain scheduled beyond five minutes. If permanent fencing must terminate the turn, detect that specific server response rather than applying a wall-clock cutoff to every reconnect failure.
There was a problem hiding this comment.
Keeping the five-minute ceiling. Bun does not expose the WebSocket upgrade status, so transient outages are indistinguishable from permanent rejections such as a stale fence. Five minutes allows several 90s stalled handshakes to recover, while still ensuring a permanently invalid wrapper eventually aborts and drains instead of retrying forever. I’ll align the PR description with that behavior.
2 participants
Summary
Stops cloud-agent sessions from hanging after a stalled ingest reconnect or an early environment-preparation failure.
Ingest reconnect
close()instead of giving up after 3 attempts.agent_0d012c64).WORKER_URL/ network).Preparing status
finalize()now creates the preparation attempt when no progress was observed, so clients get a terminal status instead of staying on "Setting up environment".preparingcomposer status whencloud.message.failedarrives.Test plan
packages/cloud-agent-sdk:src/service-state.test.ts(166 passed)services/cloud-agent-next:src/session/preparation-progress.test.tsandtest/unit/wrapper/reconnection.test.ts(77 passed)