auth(xai-oauth): cross-profile shared token store, race fix

[VinceZcrikl]

Problem

xAI uses single-use rotating refresh tokens — every successful refresh mints a new pair and the previous refresh_token is immediately invalid server-side. When two profiles under the same hermes-root both pick model.provider: xai-oauth, each keeps an INDEPENDENT copy of the rotating refresh_token in its profile-local auth.json; there's no cross-profile sync.

The race:

1. Profile A's token nears expiry → posts refresh_token R1 → xAI mints R2, invalidates R1, A stores R2.
2. Profile B still has R1 cached locally → posts R1 → xAI sees reuse → revokes the entire token family.
3. Both profiles end up with last_auth_error.code == "xai_refresh_failed", relogin_required: True — and the user discovers it later when a tool (x_search, xAI TTS, etc.) silently disappears from the schema because check_x_search_requirements() returns False.

Real users hit this with no warning. The diagnostic in the field is the "Refresh token has been revoked" last_auth_error.message on a profile that never did anything "wrong" — it just happened to wake up after a sibling refreshed.

Fix

Same pattern as the existing Nous shared store (_nous_shared_store_* + _try_import_shared_nous_state): a single authoritative copy of the OAuth tokens at <hermes-root>/shared/xai_auth.json plus a cross-profile file lock that serializes refreshes so siblings can't race on the single-use token.

New helpers (mirror Nous)

• _xai_shared_auth_dir / _xai_shared_store_path — same HERMES_SHARED_AUTH_DIR override and pytest seat belt as Nous.
• _xai_shared_store_lock — file lock at xai_auth.json.lock. Held alone for the entire refresh+persist cycle (lock-ordering exception, documented in the docstring, just like _try_import_shared_nous_state does for Nous).
• _write_shared_xai_state — atomic O_EXCL 0o600 write via secure_parent_dir + tmp + rename. Skips when refresh_token or access_token is empty.
• _read_shared_xai_state — None on missing / malformed / incomplete payload.
• _clear_shared_xai_state — wipe after terminal failure so dead tokens don't poison siblings.
• _merge_shared_xai_oauth_state — copy fresher shared tokens into a profile-local state when shared's refresh_token differs from local's. Returns True iff the state changed.

Wiring

• _save_xai_oauth_tokens now mirrors the rotated pair to the shared store after the per-profile auth.json write. Failures in the shared write are logged and swallowed — the local save remains the source of truth.
• resolve_xai_oauth_runtime_credentials now performs its refresh path under _xai_shared_store_lock instead of _auth_store_lock. Inside the lock it:
  • re-reads the per-profile state,
  • merges fresher shared tokens in and persists them locally (skipping the HTTP refresh entirely if a sibling has rotated since this profile last looked),
  • only then makes the actual HTTP refresh call,
  • on terminal failure (invalid_grant / xai_refresh_failed), clears the shared store in addition to the per-profile quarantine, so a sibling doesn't inherit dead tokens.
• _auth_store_lock is no longer held across the HTTP refresh — persistence helpers acquire it briefly and internally.

Lock ordering invariant

Following the existing documented exception on _nous_shared_store_lock: the runtime refresh path holds the shared lock alone for the whole refresh + persist cycle (callers must not enter it with _auth_store_lock already held). The persistence helpers (_save_xai_oauth_tokens, the terminal-failure quarantine block) acquire _auth_store_lock internally and briefly.

Tests

tests/hermes_cli/test_auth_xai_shared_store.py (13 cases, all passing):

• Seat belt rejects real home under pytest (mirrors Nous test).
• HERMES_SHARED_AUTH_DIR redirect honored.
• _read_shared_xai_state returns None for missing / malformed / incomplete payload.
• Write + read round-trip preserves refresh_token / access_token / token_type / id_token / expires_in / discovery / last_refresh.
• File mode is 0o600 where the platform supports chmod.
• Write skipped when refresh_token or access_token is empty.
• _clear_shared_xai_state removes the file and is idempotent on already-absent files.
• _merge_shared_xai_oauth_state no-op when refresh_tokens match.
• _merge_shared_xai_oauth_state copies fresher shared tokens in when refresh_tokens differ.
• _merge_shared_xai_oauth_state returns False when shared store is missing.
• Integration: _save_xai_oauth_tokens mirrors to both per-profile auth.json and the shared store.

Conventions match test_auth_nous_provider.py's shared-store tests — shared_store_env fixture redirecting HERMES_SHARED_AUTH_DIR to tmp_path.

Regression coverage

Full tests/hermes_cli/test_auth_xai_oauth_provider.py + test_xai_oauth_pkce_token_exchange.py + test_auth_nous_provider.py suites still green: 154/154 passed in 8.72s.

python -m pytest tests/hermes_cli/test_auth_xai_oauth_provider.py \
                 tests/hermes_cli/test_xai_oauth_pkce_token_exchange.py \
                 tests/hermes_cli/test_auth_nous_provider.py -q

Out of scope

• MCP-server-style per-tool toggling for xAI tools.
• Concurrent-thread race test — the _file_lock primitive is already covered by Nous's existing suite; the new tests exercise the integration points.

🤖 Generated with Claude Code

[alt-glitch]

Duplicate of #28375 — both implement cross-profile shared xAI OAuth token store to prevent refresh-token race. #28375 is broader (also covers credential_pool.py and docs).

[VinceZcrikl]

Closing as a duplicate of #28375 — flagged by the maintainer. Confirmed:

• Same root cause: cross-profile race on xAI's single-use rotating refresh token (profiles A and B both holding the same refresh_token locally → second refresh attempt hits "invalid_grant: revoked" → entire token family dies).
• fix(auth): share xAI OAuth refresh across profiles #28375 is broader: also touches agent/credential_pool.py (pooled-credential path I didn't address) and adds docs at website/docs/guides/xai-grok-oauth.md.
• Different (and arguably cleaner) approach: fix(auth): share xAI OAuth refresh across profiles #28375 routes xAI through the existing global root auth store via credential_pool.py integration. Mine added a parallel <hermes-root>/shared/xai_auth.json file mirroring the Nous shared-store pattern — same end result, but fix(auth): share xAI OAuth refresh across profiles #28375's path avoids the duplicate "shared file + cross-profile lock" surface area by leaning on infrastructure that already exists.

Sorry for the noise — I should have searched open PRs for "xai oauth share" / "refresh token" before opening this. Will check upstream first next time.

If anything from this PR is useful to fold into #28375 (e.g., the _clear_shared_* on terminal failure pattern, or the additional test cases for malformed payloads / file-mode bits), happy to send a follow-up suggestion comment over there — let me know.