Skip to content

fix(cloud-agent-next): align wrapper runtime generation with the lease - #6080

Merged
pandemicsyn merged 1 commit into
mainfrom
eshurakov/lucid-willow
Sep 11, 2026
Merged

fix(cloud-agent-next): align wrapper runtime generation with the lease#6080
pandemicsyn merged 1 commit into
mainfrom
eshurakov/lucid-willow

Conversation

@eshurakov

Copy link
Copy Markdown
Contributor

Problem

Modern-runtime-authorization (scoped credentials) sessions could wedge permanently after the wrapper was restarted. Every follow-up message failed pre-dispatch with Runtime credential proxy grant is unavailable, and no wrapper was ever restarted.

Root cause

The session DO's runtime.wrapperGeneration and the physical wrapper lease instance.instanceGeneration were independent counters. On a restart with a stale active identity, AgentRuntime.send cleared the identity with incrementGeneration: true and then allocateWrapperRuntimeState incremented again (runtime +2), while the lease advanced by one (+1). CloudAgentSession.runtimeProxyFence() requires the two to be equal, so issueRuntimeCredentialProxyGrant always returned null. clearAllocatedWrapperRuntimeState incremented again on each failed delivery, so the gap only grew and the session could never recover.

Fix

  • AgentRuntime.send passes the allocated physical instance generation into allocateWrapperRuntimeState, so a newly minted runtime identity adopts the lease generation.
  • clearAllocatedWrapperRuntimeState no longer increments the generation, and the clear that precedes reallocation in send drops its now-dead increment.
  • The lease is the single source of truth for the delivered generation. Stale connections stay fenced because clearing removes wrapperConnectionId/wrapperRunId, and every allocation mints a new connection/run identity.

Tests

  • New agent-runtime-runtime-proxy-grant.test.ts drives the real AgentRuntime.send allocation path and the real CloudAgentSession.issueRuntimeCredentialProxyGrant over shared storage. It fails on the previous code (expected 9 to be 2) and passes after the fix.
  • Existing unit/integration expectations were updated where they encoded the removed increment.

Verification (run from services/cloud-agent-next)

  • pnpm test — 211 files, 6086 passed, 3 skipped
  • pnpm test:integration — 32 files, 660 passed
  • pnpm typecheck — pass
  • pnpm lint — 0 warnings / 0 errors
  • pnpm format:check — clean

Not verified end to end: this change is validated at the allocation/grant contract level. A live sandbox-restart check against production/Axiom is still required after deploy.

Modern-runtime-authorization sessions could wedge permanently after the wrapper restarted: every follow-up failed pre-dispatch with "Runtime credential proxy grant is unavailable".

The session DO's runtime wrapper generation and the physical wrapper lease instance generation were independent counters. A restart cleared the stale runtime identity with incrementGeneration:true and then allocateWrapperRuntimeState incremented again (+2), while the lease advanced by one (+1). runtimeProxyFence() requires the two to be equal, so the grant could never be issued and no wrapper was ever restarted; clearAllocatedWrapperRuntimeState incremented again on each failed delivery, so the gap only grew.

Make the lease the single source of truth for the delivered generation: AgentRuntime.send passes the allocated instance generation into allocateWrapperRuntimeState, and the cleanup paths stop bumping the counter independently. Stale connections stay fenced because clearing removes the connection and run identity.

Add a regression test that drives the real AgentRuntime.send allocation and CloudAgentSession.issueRuntimeCredentialProxyGrant path. It fails on the previous code (runtime 9 vs lease 2) and passes after the fix.
@kilo-code-bot

kilo-code-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the wrapper runtime-generation/lease alignment fix and its tests; the source change correctly makes the physical lease the single source of truth for the delivered generation, and stale connections remain fenced by the freshly minted connection/run identity. No changed-line defects found.

Files Reviewed (6 files)
  • services/cloud-agent-next/src/session/agent-runtime.ts
  • services/cloud-agent-next/src/session/wrapper-runtime-state.ts
  • services/cloud-agent-next/src/session/agent-runtime.test.ts
  • services/cloud-agent-next/src/session/wrapper-runtime-state.test.ts
  • services/cloud-agent-next/src/session/agent-runtime-runtime-proxy-grant.test.ts
  • services/cloud-agent-next/test/integration/session/execute-directly-failure.test.ts

Note: allocateWrapperRuntimeState now adopts an explicit generation that can be lower than the previously persisted runtime generation. This is safe here because identity (connection/run) is reminted on every allocation, but the generation is no longer strictly monotonic, so callers that only compare wrapperGeneration ordering (e.g. global-feed producer replacement) should be re-examined if a lower-generation reconnect can race an older live producer socket.


Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants