fix: auth refactor — eliminate refresh race, increase buffer, add error semantics - #194
Conversation
Two independent refresh mechanisms (SessionProvider.refetchInterval and a setInterval in useAuth) fired concurrently every ~4 minutes, causing "invalid_grant" failures with OAuth providers that use rotating refresh tokens (e.g., NVIDIA Starfleet SSO). The second concurrent refresh consumed an already-invalidated token and killed the session. Remove the duplicate setInterval from useAuth — session refresh is now handled solely by SessionProvider's refetchInterval. Make the interval config-driven (from TOKEN_REFRESH_BUFFER_SECONDS) instead of hardcoded. Also increase the default TOKEN_REFRESH_BUFFER_MINUTES from 5 to 15. The previous 5-minute window was insufficient for deployments running long operations (deep research with ECI runs 20-40+ minutes). Enterprise deployments should set TOKEN_REFRESH_BUFFER_MINUTES=30 via env var. Includes a regression test that verifies useAuth never creates its own setInterval, preventing re-introduction of the race condition. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Greptile SummaryThis PR is the first phase of an auth refactor addressing three root causes of auth failures: it removes the duplicate Confidence Score: 5/5Safe to merge; only P2 findings, all on non-critical observability paths The core auth fixes (race condition elimination, buffer increase, error code propagation) are well-implemented and thoroughly tested. Both findings are P2: one is a narrow RUM tracking guard that misses specific subclass codes (no functional regression), the other is dead code complexity. No P0/P1 issues found. No files require special attention. Important Files Changed
Sequence DiagramsequenceDiagram
participant FE as Frontend
participant SP as SessionProvider
participant MW as AuthMiddleware
participant JV as JWTValidator
participant RUM as Datadog RUM
Note over SP: refetchInterval = config.sessionRefreshIntervalSeconds<br/>(no duplicate setInterval in useAuth)
SP->>MW: GET /session (Bearer token)
MW->>JV: validate_token_with_error(token)
alt Token valid
JV-->>MW: (user_dict, None)
MW-->>SP: 200 OK
else Token expired
JV-->>MW: (None, "token_expired")
MW-->>FE: 401 {error: "token_expired"}
FE->>RUM: trackRumError("Auth failure: token_expired")
else Token invalid
JV-->>MW: (None, "token_invalid")
MW-->>FE: 401 {error: "token_invalid"}
FE->>RUM: trackRumError("Auth failure: token_invalid")
else No token
MW-->>FE: 401 {error: "token_missing"}
end
Note over FE: WebSocket path
FE->>MW: WS connect (AuthError raised in workflow)
MW-->>FE: ERROR msg {message: exc.error_code}
FE->>RUM: trackRumError("WebSocket auth failure")
Reviews (6): Last reviewed commit: "Merge upstream develop into auth refacto..." | Re-trigger Greptile |
The backend previously returned the same 401 "Invalid or expired auth token" for all failure types. Frontends had no way to distinguish "refresh your token" from "reauthenticate completely", contributing to silent failure modes. Backend changes: - Add validate_with_error() to TokenValidator base class (backward- compatible default wraps validate()) - Override in JWTValidator: ExpiredSignatureError -> "token_expired", InvalidTokenError -> "token_invalid" - Middleware 401 responses now include an "error" field with machine- readable codes: "token_missing", "token_expired", "token_invalid" - Add TokenExpiredError/TokenInvalidError subclasses of AuthError - WebSocket handler catches AuthError and sends typed "auth_error" message before closing, so frontends get actionable feedback Frontend changes: - authenticatedFetch emits Datadog RUM custom errors on 401 responses with the error code as a facet (no-op without Datadog SDK) - WebSocket client emits RUM events for auth_error messages Also fixes a pre-existing flaky test (test_matches_key_by_kid) caused by stale JWKS cache timestamps on long-uptime machines. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TypeScript strict mode rejects `window as Record<string, unknown>` because Window's type doesn't have an index signature. Cast through `unknown` first: `window as unknown as Record<string, unknown>`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Per review feedback, consolidate the two-method pattern into a single validate() that returns (user_dict, error_code) directly. There are no external consumers of the old dict|None signature — StarfleetValidator and NVAuthValidator in aiq-bp-internal will be updated in a coordinated internal MR. - TokenValidator.validate() now returns tuple[dict | None, str | None] - Remove validate_with_error() from base class and JWTValidator - Middleware calls validate() directly (no _validate_token_with_error) - Update all tests for new return type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…adog The DD_RUM.addError() calls added in the previous commit were dead code for the primary auth failure path: token refresh failures are detected server-side in the JWT callback, propagated via the session object, and then signOut() immediately redirects — no RUM emission happened before the page unloaded. Add trackRumError() call in the useAuth hook BEFORE handleSignOut() so the event is buffered before navigation starts. The Datadog SDK's sendBeacon flushes buffered events on page unload. Also extract a shared trackRumError() utility to DRY up the DD_RUM access pattern (was duplicated with type casts in 3 files). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resolve conflicts with the current request-auth helper extraction. Preserve machine-readable auth error codes through shared HTTP/WebSocket identity resolution and keep SessionProvider as the single refresh scheduler.
AjayThorve
left a comment
There was a problem hiding this comment.
LGTM, tested locally, no regressions to the middleware
0d53241
into
NVIDIA-AI-Blueprints:develop
…or semantics (NVIDIA-AI-Blueprints#194) * fix: eliminate dual token refresh race and increase refresh buffer Two independent refresh mechanisms (SessionProvider.refetchInterval and a setInterval in useAuth) fired concurrently every ~4 minutes, causing "invalid_grant" failures with OAuth providers that use rotating refresh tokens (e.g., NVIDIA Starfleet SSO). The second concurrent refresh consumed an already-invalidated token and killed the session. Remove the duplicate setInterval from useAuth — session refresh is now handled solely by SessionProvider's refetchInterval. Make the interval config-driven (from TOKEN_REFRESH_BUFFER_SECONDS) instead of hardcoded. Also increase the default TOKEN_REFRESH_BUFFER_MINUTES from 5 to 15. The previous 5-minute window was insufficient for deployments running long operations (deep research with ECI runs 20-40+ minutes). Enterprise deployments should set TOKEN_REFRESH_BUFFER_MINUTES=30 via env var. Includes a regression test that verifies useAuth never creates its own setInterval, preventing re-introduction of the race condition. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
This PR addresses three of the five root causes of persistent auth failures documented in the auth flow analysis. It is the first PR of a multi-phase auth refactor.
Commit 1: Eliminate dual refresh race + increase buffer (Phases 1 & 2)
setIntervalinuseAuththat fired concurrently withSessionProvider.refetchInterval. With rotating refresh tokens (e.g., NVIDIA Starfleet SSO), concurrent refreshes consumed an already-invalidated token →invalid_grant→ random logouts.SessionProvider.refetchIntervalnow usesconfig.sessionRefreshIntervalSecondsinstead of hardcoded4 * 60.TOKEN_REFRESH_BUFFER_MINUTESdefault changed from 5 → 15 minutes (covering most deep research operations).Commit 2: Machine-readable auth error codes + RUM observability (Phase 3)
validate_with_error()on TokenValidator: New method returns(user, error_code)instead of justuser | None. Default wrapsvalidate()for backward compatibility.ExpiredSignatureError→"token_expired",InvalidTokenError→"token_invalid"."error"field:"token_missing","token_expired","token_invalid"— frontends can distinguish "refresh" from "reauthenticate".TokenExpiredError/TokenInvalidErrorsubclasses ofAuthErrorwitherror_codeattribute.AuthErrorcaught in workflow handler → sends typed"auth_error"message over WS.authenticatedFetchand WebSocket client emitDD_RUM.addError()on auth failures (no-op without Datadog SDK).Files changed
Frontend (
frontends/ui/src/)adapters/auth/session.tsadapters/auth/config.tsapp/providers.tsxadapters/api/authenticated-fetch.tsadapters/api/websocket-client.ts.env.example,README.mdadapters/auth/config.spec.tsadapters/auth/session.spec.tsxBackend (
frontends/aiq_api/src/aiq_api/)auth/base.pyvalidate_with_error()default methodauth/jwt_validator.pyauth/middleware.pyauth/errors.pyTokenExpiredError,TokenInvalidErrorauth/__init__.pywebsocket_reconnect.pytests/test_auth.pyTest plan
npm run test:ci— 1183 frontend tests passpytest test_auth.py— 39 backend tests pass (10 new)"error": "token_expired"for expired JWT"error": "token_invalid"for bad JWTREQUIRE_AUTH=true, verify single refresh path in Network tabREQUIRE_AUTH=false, verify app loads with no auth UI🤖 Generated with Claude Code