Skip to content

fix(desktop): bound the shared resume flight and adopt its straggler - #96523

Open
JoaoMarcos44 wants to merge 1 commit into
NousResearch:mainfrom
JoaoMarcos44:fix/single-flight-resume-late-settle
Open

fix(desktop): bound the shared resume flight and adopt its straggler#96523
JoaoMarcos44 wants to merge 1 commit into
NousResearch:mainfrom
JoaoMarcos44:fix/single-flight-resume-late-settle

Conversation

@JoaoMarcos44

Copy link
Copy Markdown
Contributor

Summary

Closes #96522.

singleFlightSessionResume() shares ONE promise per stored session id across every Desktop surface that can recover a dead runtime, and the slot is released only by that promise's .finally(). An unsettled run() therefore makes the conversation unrecoverable for the life of the window, and the rejection-driven retry ladder never arms because nothing ever rejects.

This PR bounds the flight and decides what the straggler means — because a deadline on its own trades a hang for a stranded runtime.

Root cause

The RPC is already bounded: HermesGateway sets requestTimeoutMs = DEFAULT_GATEWAY_REQUEST_TIMEOUT_MS (30s). The run() bodies are not just that RPC — they resolve the owning profile first, and resolveStoredSession() probes backends sequentially on a cache miss, one 30s getSession() per configured profile. Settlement time scales with profile count; a probe wedged the way #93454 describes never settles at all.

Because the map is module-level, a wedge created by any ONE entry point wedges all five:

Call site Surface
resolve-target-session.ts:101 route resolver
utils.ts:122 (resumeStoredRuntimeSession) shared recovery
submit.ts:592 prompt submit
use-session-actions/index.ts:1393 cold resume
use-session-tile-delegate.ts:198 tile resume

The half a deadline does not fix

withTimeout() says it outright:

/** Rejection raised by withTimeout. The bounded work is NOT cancelled — the
 * caller decides what a straggler that settles later means. */

For session.resume the straggler is not inert. _claim_or_reuse_live() registers the record before returning, so a late resume has already minted a real runtime on the gateway. Dropping it strands that runtime for the reaper AND makes the next recovery mint another — the per-resume orphan shape this module was built to stop (#91276). The module already solved the identical problem for drift-aborts via registerRecoveredRuntime() / takeRecoveredRuntime(); nothing wired the timeout path into it.

%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#8b0000', 'mainBkg': '#0a0204', 'primaryTextColor': '#ffccd5', 'primaryBorderColor': '#ff0038', 'lineColor': '#ff0038'}}}%%
graph TD
    A["🩸 Five Recovery Surfaces"] -->|"one shared slot per stored id"| B["🔥 Single Flight"]
    B --> C["⛓️ Profile Ladder<br/>30s per configured backend"]
    C --> D["📡 session.resume RPC<br/>30s budget"]
    B --> E{"⏳ Settlement Ceiling<br/>90s = 30 + 30 + 30"}
    E -->|"settles in budget"| F["⚔️ Runtime Adopted<br/>slot released"]
    E -->|"exceeded"| G["💀 Flight Rejected<br/>slot released · retry ladder arms"]
    G --> H["🕯️ Straggler Still Lands<br/>work was never cancelled"]
    H -->|"no newer flight owns the id"| I["🔮 Recovered-Runtime Cache<br/>next action adopts it"]
    H -->|"newer flight owns the id"| J["🚫 Discarded<br/>its caller adopts its own result"]
    I --> F
Loading

Implementation

  • SESSION_RESUME_SETTLEMENT_TIMEOUT_MS = 90_000, derived rather than picked round: active-profile probe (30s, Electron DEFAULT_FETCH_TIMEOUT_MS) + one cross-profile probe (30s) + the resume RPC (30s). A slow-but-healthy multi-profile recovery is never aborted; past that point the flight has held the slot longer than any single healthy recovery needs.
  • The flight is wrapped in the repo's existing withTimeout(). Its .finally() still frees the slot, so the next caller gets a FRESH attempt and the existing rejection-driven ladder arms.
  • onTimeout attaches a late-settle handler: a straggler that yields a session_id is handed to registerRecoveredRuntime(). Guarded two ways — a straggler that rejects minted nothing, and one that lands while a newer flight already owns the stored id is discarded, because that flight's caller will adopt its own result.
  • singleFlightSessionResume() takes an optional timeoutMs so tests can pin the boundary; every production caller uses the derived default.

Two files, +167 / −6. No new dependency, no new module, no behavior change on the success path.

Test plan

apps/desktop/src/app/session/hooks/use-prompt-actions/single-flight-resume.test.ts13 passed (6 existing + 7 new).

New test Pins
a never-settling resume rejects at the deadline instead of wedging the slot forever the ceiling fires
a later caller for the same stored id gets a FRESH attempt, not the dead flight the slot is released
a joiner of an already-wedged flight inherits the same deadline joiners are bounded too
a slow-but-legitimate resume (profile probe + RPC) still settles inside the ceiling 89s healthy resume is not aborted
a runtime minted by a timed-out resume is adopted, not stranded on the gateway the straggler reaches the cache
a straggler is NOT cached when a newer flight already owns the stored id no stale adoption
a straggler that fails minted nothing and caches nothing failure path is inert

Falsification: with the production file reverted to origin/main and the tests kept, 6 of the 7 fail — by timing out at 5000 ms, which is the wedge itself. The seventh (a slow-but-legitimate resume…) passes on old code because old code has no ceiling at all; it exists so a future tightening of the constant cannot silently start aborting legitimate recoveries.

Local run notes

The desktop vite.config.ts in this checkout imports @rolldown/plugin-babel, which is declared in apps/desktop/package.json but absent from the installed tree, and npm install refuses on this machine (EBADENGINE: repo requires npm <11.10.0 || >=11.17.0, local is 11.4.1). The suite was therefore run through a throwaway vitest config providing only the @ alias — the module under test needs nothing else. That config was deleted and is not part of this diff. Hosted CI remains the authority.

Non-duplicate analysis

Type of change

  • Bug fix (non-breaking change that fixes an issue)

Checklist

  • Derived, not guessed, constant with the derivation written into the code
  • New tests fail on the prior behavior (falsification recorded above)
  • No new dependency, module, or public surface beyond one exported constant and one optional parameter
  • Success path unchanged

singleFlightSessionResume() shares one promise per stored session id
across every surface that can recover a dead runtime, and the slot is
only released by that promise's .finally(). An unsettled run() therefore
makes the conversation unrecoverable for the life of the window: every
later caller joins the dead flight instead of starting its own attempt,
and the rejection-driven retry / Retry-UI ladder never arms because
nothing ever rejects.

The resume RPC itself is bounded (HermesGateway's 30s request budget),
but run() bodies resolve the owning profile first, and
resolveStoredSession() probes backends sequentially on a cache miss -
one 30s getSession() per configured profile. Settlement time therefore
scales with profile count, and a probe wedged the way NousResearch#93454 describes
never settles at all.

Bound the flight with the existing withTimeout() helper. The ceiling is
derived from the longest LEGITIMATE settlement rather than picked round
- active-profile probe (30s) + one cross-profile probe (30s) + the
resume RPC (30s) = 90s - so a slow-but-healthy multi-profile recovery is
never aborted while a genuinely wedged one is.

A deadline alone would trade a hang for an orphan. withTimeout() does
not cancel the work it bounds, and the straggler here is not inert:
_claim_or_reuse_live() registers the record before returning, so a late
resume has already minted a REAL runtime on the gateway. Dropping it is
the per-resume orphan shape this module exists to prevent (NousResearch#91276). Hand
a late arrival to the same recovered-runtime cache the drift-abort path
uses, so the next resume-shaped action adopts it instead of minting a
second one - unless a newer flight already owns the stored id, whose
caller will adopt its own result.

Six of the seven new tests fail on the prior behavior by timing out,
which is the wedge itself. The seventh pins the ceiling against a 89s
healthy resume so a future tightening cannot silently start aborting
legitimate recoveries.

Closes NousResearch#96522

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSgsNweukuU4n2WYX19JW
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/desktop Electron desktop app (apps/desktop/*) sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Aug 27, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference; please use your judgment.

fix(desktop): bound the shared resume flight and adopt its straggler — prevents singleFlightSessionResume wedge and strands no gateway runtime.

  • apps/desktop/src/app/session/hooks/use-prompt-actions/single-flight-resume.ts — wraps the run() promise with withTimeout(SESSION_RESUME_SETTLEMENT_TIMEOUT_MS) (95s ceiling derived from 3×30s = active-profile getSession probe + one cross-profile probe + resumeStoredRuntimeSession RPC, each on DEFAULT_FETCH_TIMEOUT_MS). isTimeoutError deadline rejects the shared slot so .finally() clears _inFlightResumeByStoredSessionId, un-wedging later callers. Subsequent caller for same storedSessionId gets a fresh attempt rather than joining the dead flight; joiners of an already-wedged flight inherit the same deadline (shared promise), so no caller hangs past the ceiling.
  • Straggler adoption: if the timed-out RPC later lands a registered runtime (90s+ probe), its session_id is cached via registerRecoveredRuntime so the next resume-shaped action reuses it instead of minting a duplicate; if a newer flight already claimed the slot, the straggler is dropped (not cached), avoiding wrong-runtime aiming. Failure to mint caches nothing. afterEach fix vi.useRealTimers() ensures fake-timer tests don't leak.
  • Tests in single-flight-resume.test.ts cover all edges: never-settling → timeout, fresh attempt after timeout, joiner inherits deadline, 89s healthy slow resume still settles inside ceiling, late land → takeRecoveredRuntime hit, late land contended → not cached, late failure → not cached. Thorough regression for the share-by-promise slot.
  • Tight ceiling is deliberately conservative (95s, not 90s) so legitimate 89s probe chains are not aborted — good tradeoff between liveness and correctness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/desktop Electron desktop app (apps/desktop/*) P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Desktop: an unsettled session.resume wedges the shared single-flight slot, and breaking it strands a live runtime

3 participants