Skip to content

fix(desktop): stop reporting transient token-refresh failures as expired sessions - #73844

Open
Sora-bluesky wants to merge 4 commits into
NousResearch:mainfrom
Sora-bluesky:fix/desktop-ticket-transient
Open

fix(desktop): stop reporting transient token-refresh failures as expired sessions#73844
Sora-bluesky wants to merge 4 commits into
NousResearch:mainfrom
Sora-bluesky:fix/desktop-ticket-transient

Conversation

@Sora-bluesky

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes mechanism 2 of #73722: a transient failure while refreshing the native OAuth access token was reported as "Your remote gateway session has expired", sending the user to re-sign-in for a network blip.

ensureNativeAccessToken() is deliberate about the split: a 401 on refresh means the refresh token is dead, so it clears the stored tokens and resolves null; a transient failure (timeout, 5xx, network) throws so the tokens survive for a retry. The call site in mintGatewayWsTicket() erased that distinction with .catch(() => null). With a native (cookieless) sign-in there is no cookie session to fall back to, so the fallback leg 401s and gatewayTicketFailure() classifies the whole thing as an expired session. The reporter's desktop.log shows it live: liveness probe drops, the next resolve says "expired", and the retry seconds later connects with the same credentials.

Two adjacent defects surfaced while fixing it, and both are covered here because a naive fix (just dropping the catch) would regress them:

  • The dead-RT 401 branch never matched in production. ensureNativeAccessToken checks error.statusCode === 401, but fetchJson rejects with a message-prefixed Error("401: …") and no statusCode property, and the server's dead-RT branch answers 401 {"error":"session_expired"}. So a genuinely dead refresh token always took the throw path, and only the swallowed catch kept the reauth flow working by accident. The new isNativeRefreshAuthRejection() predicate recognizes both shapes, making the intended "dead RT → clear tokens → cookie fallback → sign-in" flow real.
  • Cookie and native sessions can coexist (logout clears both). A native-path failure now falls back to the cookie session and uses its ticket when it works; only when that fallback also fails does the original native error propagate, so a cookie-side 401 in a native-only config cannot overwrite a transient failure with a false "session expired".

The end state for each input: dead refresh token → sign-in prompt (as intended); transient failure with a live cookie → connects through the cookie (as baseline did); transient failure with no cookie → retryable connectivity error ("Could not reach the remote Hermes gateway while refreshing its WebSocket ticket. Try reconnecting.") instead of a false sign-in demand, which is the mechanism-2 fix itself.

Testability

mintGatewayWsTicket lived in main.ts, which imports electron and node-pty at module scope and is unreachable from the test suite. The function moved to connection-config.ts as a dependency-injected helper, the same pattern resolveTestWsUrl already uses there, and main.ts keeps a same-named adapter so all three call sites (freshGatewayWsUrl, buildRemoteConnection, the connection test's mintTicket injection) are untouched.

Tests

New cases in connection-config.test.ts:

  • the production 401 shape (Error("401: …"), no statusCode) is recognized as an auth rejection, plus negative cases for transient shapes
  • a transient refresh failure with a live cookie session mints through the cookie (fails on the old code: the transient error propagated without ever trying the cookie)
  • a transient failure with no usable cookie propagates the original native error, untagged, so it classifies as transport
  • no stored native tokens still falls back to the cookie session; a native bearer success mints without touching the cookie leg

79/79 in connection-config.test.ts, tsc --noEmit clean.

Relation to #73821

Independent fixes for the two mechanisms in #73722. #73821 handles the renderer's boot path; this PR handles the main process's ticket minting. Either lands without the other.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/desktop Electron desktop app (apps/desktop/*) area/auth Authentication, OAuth, credential pools sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data labels Jul 30, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the native-refresh error shape and covering the ticket and REST sibling paths. The current-main premise is confirmed: apps/desktop/electron/main.ts:6311, :7570, and :10131 swallow refresh failures, while fetchJson emits untagged Error("401: …") at :4214-4216.

Problems

  • apps/desktop/electron/native-access-token.ts:31 makes authEpoch global, although refresh flights are scoped by base URL. An explicit auth change for gateway A invalidates an in-flight refresh for gateway B; :66/:74 return null and discard B's valid rotation. This is reachable because Desktop supports distinct per-profile remote hosts (apps/desktop/electron/connection-config.ts:377-380).

Suggested changes

  • Keep an epoch per normalized base URL, pass the affected URL to invalidateExplicitAuthChange, and add a two-gateway regression test proving invalidating A does not cancel B's refresh.

Automated hermes-sweeper review.

deps: NativeAccessTokenCoordinatorDeps
): NativeAccessTokenCoordinator {
const refreshFlights = new Map<string, Promise<string | null>>()
let authEpoch = 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

authEpoch is global while refreshFlights is per base URL. A login/logout for remote gateway A will make an in-flight refresh for independent gateway B return null at lines 66/74 and drop its rotated token. Please scope the epoch and invalidation to normalized base URL, with a two-host regression test.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform area/sessions Session lifecycle, resume, persistence, history labels Jul 30, 2026
@Sora-bluesky

Copy link
Copy Markdown
Contributor Author

Confirmed and fixed in 397805ccb.

The epoch check sits above storeTokens, so the discard was worse than a wasted refresh: gateway B's rotation was dropped after the server had already consumed the refresh token, leaving the old one in the keychain. B's next refresh then looks like token reuse and B is logged out for real. It also returned null from ensure(), which is the branch this PR exists to keep out of the "session expired" path, so the bug reintroduced the thing the PR removes. At the merge base the rotation was stored unconditionally, so this was mine.

The epoch is now per normalized gateway and invalidateExplicitAuthChange takes the gateway it applies to.

One thing your report and my first fix both missed. The logout handler bumped the epoch unconditionally while _clearNativeTokens sat inside if (baseUrl), and boot re-sign-in calls oauthLogoutConnectionConfig() with no URL (boot-failure-overlay.tsx:178). So a no-URL logout changed nobody's native tokens yet cancelled every in-flight refresh. The invalidation now lives in the same if (baseUrl) block as the token clearing, and the global epoch is gone rather than left as an unused path, since no clear-all native-token operation exists. Partition-wide cookie clearing is unchanged.

On reachability, connection-config.ts:377-380 is the SSH profile parser, not the OAuth base URL path. The per-profile remote host precedence is at main.ts:7347-7360, the per-profile backend pool at main.ts:1046, ticket minting against that profile's base URL at main.ts:6260-6272, and cloud discovery where each agent runs its own PKCE exchange at main.ts:6286-6296. The timing precondition is an auth change on one gateway inside another's refresh window, which is bounded by the 10s timeout at main.ts:6224.

Tests now drive the real IPC handlers rather than the coordinator alone. Deleting either production invalidation call fails them; with the previous tests both deletions stayed green.

@oliver-mee

Copy link
Copy Markdown
Contributor

Production confirmation for the second defect you describe, the one where the dead-RT 401 branch never matches. I hit it last week and the logs show it clearly.

Setup: Hermes Desktop in remote mode against a self-hosted gateway, native (cookieless) sign-in, OIDC provider is Authelia 4.39.20 with rotating refresh tokens and reuse detection.

My refresh token chain was genuinely revoked provider-side, which is the case the branch exists for. The provider's refresh-token table shows the newest two entries as revoked=1 with nothing issued after, and every subsequent refresh returned "The refresh token has not been found". The gateway answered the desktop with 401 session_expired each time.

The desktop never cleared those tokens. native-oauth-tokens.json still held the dead token set days later, and instead of prompting once for a fresh sign-in the app retried on a loop:

  • 50 refresh_failure events, hourly, over roughly 18 hours against a chain that was already dead
  • the provider eventually logged Rate Limit Exceeded on its token endpoint from the retries
  • in desktop.log, each attempt then surfaced as 401: {"error":"unauthenticated","reason":"no_cookie"}, which is the cookie fallback leg failing in a native-only setup, exactly as you describe

That matches your reading of the code. fetchJson rejects with new Error(\${res.statusCode}: ...`)and nostatusCodeproperty, soerror.statusCode === 401inensureNativeAccessTokencannot be true and a dead refresh token always takes the throw path rather than the clear-and-reauth path.isNativeRefreshAuthRejection()` recognising both shapes would have turned my 18 hours of hourly retries into a single sign-in prompt.

Worth noting for anyone reading this thread that the revocation itself has a separate cause: concurrent refreshes replaying the same rotating token, tracked in #55712, with #55717 and #71548 coalescing the cookie and native paths respectively. This PR is the other half, what happens once the chain is already dead, and my logs show both mechanisms firing one after the other.

Happy to test this branch against a rotating-RT provider with reuse detection if that is useful. I can reproduce the dead-chain state on demand.

@Sora-bluesky

Copy link
Copy Markdown
Contributor Author

@oliver-mee thanks — this is the production trace the branch was missing, and it matches the code on current main:

  • ensureNativeAccessToken refreshes through postJsonNoAuthfetchJson, and fetchJson rejects with new Error(\${res.statusCode}: ...`)and no.statusCode property (apps/desktop/electron/main.ts:4714`).
  • The dead-RT branch tests error.statusCode === 401 (main.ts:6947), so it can never match; the rejection takes the throw path and the token file is kept, so every later refresh attempt replays the dead chain. Your 50 refresh_failure events and the provider-side rate limit are that loop.

isNativeRefreshAuthRejection() in this PR accepts both shapes (message-only 401: ... and a tagged .statusCode), and the tests pin the message-only production shape. Your note on the revocation cause is right too — the replay of a rotating token is #55712 (with #55717 / #71548 coalescing the cookie and native refresh legs); the dead-RT part of this PR is what happens once the chain is already revoked, alongside the transient-vs-expired fix it started from.

The branch currently conflicts with main in apps/desktop/electron/main.ts. I will rebase it and ping here once it is on current main — a run against your Authelia setup with reuse detection would be very useful then, since I cannot reproduce a provider-side revoked chain locally.

@Sora-bluesky
Sora-bluesky force-pushed the fix/desktop-ticket-transient branch from 7f5b8f9 to 78806d1 Compare August 17, 2026 11:27
@Sora-bluesky

Copy link
Copy Markdown
Contributor Author

@oliver-mee rebased onto main at b52b725. The only conflict was the new remote-gateway headers plumbing, which now flows through the pure ticket minter and requestJsonForProfile as well (with a test for both mint paths). CI is green on the rebased head, so a run against your Authelia setup with reuse detection would be very useful now.

@Sora-bluesky
Sora-bluesky force-pushed the fix/desktop-ticket-transient branch from 78806d1 to 7a961e6 Compare August 17, 2026 23:54
Sora-bluesky and others added 4 commits August 18, 2026 13:46
…red sessions

ensureNativeAccessToken splits refresh failures on purpose: a dead
refresh token clears the stored tokens and resolves null, a transient
failure throws so the tokens survive for a retry. mintGatewayWsTicket
erased that split with .catch(() => null), so with a native
(cookieless) sign-in a network blip fell through to the cookie path,
401ed, and was misreported as an expired session.

Two adjacent defects surfaced while fixing it. The dead-RT 401 branch
never matched in production because fetchJson rejects with a
message-prefixed Error and no statusCode property; the new
isNativeRefreshAuthRejection predicate recognizes both shapes. And
cookie and native sessions can coexist, so a native-path failure now
falls back to the cookie session and only propagates the original
native error when that fallback also fails.

mintGatewayWsTicket moved to connection-config.ts as a
dependency-injected helper (the resolveTestWsUrl pattern) so all of
this is unit-testable; main.ts keeps a same-named adapter.

Fixes mechanism 2 of NousResearch#73722.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rors

Two failure modes in the native OAuth path:

- ensureNativeAccessToken could run concurrently, so two callers sent
  the same rotating refresh token; the loser's 401 then cleared the
  winner's freshly stored tokens and forced a re-login. Refreshes now
  share one in-flight promise per gateway (single-flight), a 401 only
  clears tokens when the stored refresh token is still the one that
  was sent (compare-and-clear), and an explicit login/logout bumps an
  auth epoch that makes a stale in-flight refresh discard its result.

- both REST call sites swallowed every ensure error into the cookie
  fallback, so a transient refresh timeout with an empty cookie jar
  surfaced as a false 401. The fallback now keeps the original native
  error and rethrows it when the cookie path also fails auth,
  mirroring the WS-ticket-minting pattern.

Extracted into native-access-token.ts / oauth-rest-request.ts with
behavioral tests for the race, the stale-loser 401, the logout epoch,
and both transient outcomes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The refresh coordinator kept a single authEpoch while refresh flights are
keyed per gateway, so an explicit auth change on one gateway cancelled an
in-flight refresh on another. The epoch check runs before storeTokens, so the
other gateway's rotation was discarded after the server had already consumed
its refresh token: the next refresh looked like token reuse and logged that
gateway out for real. It also dropped ensure() to null, which is the branch
this PR exists to keep out of the "session expired" path.

The epoch is now per normalized gateway. invalidateExplicitAuthChange takes
the gateway it applies to, and the logout handler calls it in the same
if (baseUrl) block that clears that gateway's native tokens, so invalidation
and token clearing share one condition. A logout with no URL clears the shared
cookie partition and no longer touches native epochs, since it does not change
any native tokens. There is no clear-all native-token operation, so the global
epoch is gone rather than left as an unused path.

Tests drive the real IPC handlers: a URL-bearing logout must stop its own
gateway's stale rotation from being stored and must leave another gateway's
refresh alone, a no-URL logout must not disturb either, and an explicit login
must invalidate its own older flight. Deleting either production invalidation
call fails these.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A bare `88,547` sat on line 2 of native-access-token.test.ts. It parses as a
comma expression so vitest ran fine, but tsc rejects it (TS2695) and
check:lint failed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Sora-bluesky
Sora-bluesky force-pushed the fix/desktop-ticket-transient branch from 7a961e6 to f547562 Compare August 18, 2026 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools area/sessions Session lifecycle, resume, persistence, history comp/desktop Electron desktop app (apps/desktop/*) P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants