Skip to content

fix(cloud-agent-next): stop failing a positively running attach - #6268

Merged
eshurakov merged 1 commit into
mainfrom
eshurakov/wise-oasis
Sep 17, 2026
Merged

eshurakov merged 1 commit into
mainfrom
eshurakov/wise-oasis

Conversation

@eshurakov

Copy link
Copy Markdown
Contributor

Problem

A positively running session.attach was always reconciled as an unconfirmed outcome, so the turn failed as environment_failed and a healthy runtime was retired while the wrapper was still preparing the session.

reconcileSessionOperation required every running operation carrying a wrapper executionDeadlineAt to persist that deadline, but the only persistence helper for it (recordSessionOperationExecutionDeadline) is prompt-only and returns undefined when the authorization is not session.prompt. For an attach the supplied closure therefore reported false, reconcile returned {state:'uncertain', reason:'unverified'}, and operationDispatchError mapped that to a non-retryable runtime_unhealthy — "Original operation outcome is unconfirmed".

Observed on a control-plane canary session (workspace_512d5b8f) with no agent activity yet: the wrapper had been streaming preparation step_output for 77 s and heartbeating normally, yet 77 ms after the attach dispatch's transport error the message went queued -> failed with cause environment_failed, and the sandbox was stopped 88 ms later. Both turns of the session failed this way. The existing reconciles an ambiguous attach through the existing operation receipt path test already built this exact combination — attach authorization, dispatched proof, running lookup carrying executionDeadlineAt — but asserted only request counts, so the message failure was invisible.

Change

  • Apply the execution-deadline persistence requirement only to session.prompt. A running attach now stays running (session_busy, retryable) and is observed to completion inside the original head deadline instead of being terminalized.
  • Emit a session_operation_reconcile diagnostic carrying reason and lookupState on the missing and unverified returns, so the remaining unconfirmed branches are decidable instead of inferred. The transport branch and the two post-completion returns are deliberately left uninstrumented.

Not changed

  • The policy for genuinely unconfirmed outcomes, prompt retry behaviour, the wrapper, and the outcome classifier.
  • An authorization-gated session.attach RPC retry was prototyped and dropped: fix(cloud-agent-next): recover an attach the replacement runtime never saw #6225 already recovers a lost attach at the reconcile layer (releaseUnconfirmedAttach plus a fresh dispatch in the same delivery), so the retry added no functional gain.

Verification

  • vitest run on the touched suites: 373 passed.
  • pnpm test:integration (Workers runtime, 35 files): 701 passed. Repeated runs under full parallel load show 5 s timeouts in sandbox-terminal.test.ts; that suite passes 31/31 in isolation and does not touch this path.
  • typecheck, lint, format:check, git diff --check: clean.
  • Each added test was shown failing against the pre-change source.

Coverage limit

The fixed behaviour and the new diagnostic were not exercised against a live Worker/DO/wrapper path. Forcing a reconcile while an attach is genuinely still running needs a transport fault, and the levers available locally (docker pause, a DO eviction via a Worker restart) retire the sandbox before dispatchSessionOperation re-runs with a dispatched proof — reproduced 3/3 including with no fault injection at all. The fix is therefore covered at unit and DO-orchestration level, and the no-regression path was verified end to end (a real cold-to-hot session with a real sandbox and wrapper attach completing all four messages).

reconcileSessionOperation required every running operation carrying a
wrapper executionDeadlineAt to persist that deadline, but the only
persistence helper for it is prompt-only and always returns undefined for
an attach. A positively running attach therefore always reconciled as
unverified, became runtime_unhealthy "Original operation outcome is
unconfirmed" with retryable: false, and failed the turn as
environment_failed while retiring a healthy runtime.

- apply the execution-deadline persistence requirement only to session.prompt,
  so a running attach stays running and is observed to completion instead of
  being terminalized while the wrapper is still preparing
- emit a session_operation_reconcile diagnostic carrying reason and lookupState
  on the missing and unverified returns, so the remaining unconfirmed branches
  are decidable instead of inferred
@kilo-code-bot

kilo-code-bot Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (3 files)
  • services/cloud-agent-next/src/sandbox-session/session-operation.ts
  • services/cloud-agent-next/src/sandbox-session/session-operation.test.ts
  • services/cloud-agent-next/src/sandbox-session/session-message-queue.test.ts

Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants