Skip to content

fix(codex): adopt refresh_token from auth.json even without access_token (#70097) - #70111

Closed
JonthanaHanh wants to merge 1 commit into
NousResearch:mainfrom
JonthanaHanh:fix/70097-codex-credential-pool-adoption
Closed

fix(codex): adopt refresh_token from auth.json even without access_token (#70097)#70111
JonthanaHanh wants to merge 1 commit into
NousResearch:mainfrom
JonthanaHanh:fix/70097-codex-credential-pool-adoption

Conversation

@JonthanaHanh

Copy link
Copy Markdown
Contributor

Summary

Fixes two defects in the openai-codex credential pool recovery path that cause profiles to go terminally DEAD when a sibling profile rotates the shared token pair.

Defect 1 — adoption path silently no-ops

Root cause: _sync_codex_entry_from_auth_store() (credential_pool.py:768) skips adoption when store_access is empty. When another process rotated the token pair and the auth store only has last_refresh + a new refresh_token (no access_token), the stale profile's entry keeps the consumed refresh_token and replays it, getting refresh_token_reused → terminally DEAD.

Fix: Also adopt when store_refresh differs from entry_refresh, even when store_access is empty. Preserve the entry's existing access_token via store_access or entry.access_token.

Defect 2 — false "auth refreshed" success log

Root cause: _try_refresh_codex_client_credentials() (run_agent.py:4583) returns True whenever resolve_codex_runtime_credentials() returns any non-empty credentials — including the same stale token when the underlying refresh failed (failure is logger.debug-only). The conversation loop then logs "auth refreshed after 401" right before the retry fails with the identical token_expired.

Fix: Compare the access token before/after the refresh. If unchanged, return False so the 401-retry path doesn't claim success.

Changes

File Lines Change
agent/credential_pool.py +25 Adopt refresh_token even without access_token
run_agent.py +15 Return False when refresh produced no new token

Fixes #70097

…ken (NousResearch#70097)

Two defects in the openai-codex credential pool recovery path:

Defect 1 — adoption path silently no-ops when store_access is empty

_sync_codex_entry_from_auth_store() skipped adoption when the auth
store had no access_token (only last_refresh).  When another process
rotated the token pair, the stale profile's entry kept the consumed
refresh_token and replayed it, getting refresh_token_reused and going
terminally DEAD.

Fix: also adopt when store_refresh differs from entry_refresh, even
when store_access is empty.  Keep the entry's existing access_token
in that case (store_access or entry.access_token).

Defect 2 — false 'auth refreshed' success log

_try_refresh_codex_client_credentials() returned True whenever
resolve_codex_runtime_credentials() returned any non-empty credentials,
including the same stale token when the underlying refresh failed.
The conversation loop then logged 'auth refreshed after 401' right
before the retry failed with the identical token_expired.

Fix: compare the access token before/after the refresh.  If unchanged,
return False so the 401-retry path logs the truth.

Fixes NousResearch#70097
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating the refresh-token-only adoption case. The Codex pool defect is still present on current main: agent/credential_pool.py:813-816 requires store_access, while agent/credential_pool.py:1219-1225 then refreshes with the unchanged entry token.

Problems

  • The run_agent.py portion is based on stale behavior. A forced Codex refresh now either returns refreshed/recovered credentials (hermes_cli/auth.py:3976-3978) or raises; run_agent.py:5035-5037 catches that failure and returns False already.
  • The PR changes two recovery paths but adds no regression tests. In particular, the refresh-only auth-store state needs a direct test.

Suggested changes

  • Keep the credential-pool hunk and add a regression covering a missing stored access token plus a changed refresh token, including preservation of the pool entry's access token and clearing stale status.
  • Drop or re-demonstrate the unchanged-token run_agent.py guard against current main before retaining it.

This is an automated hermes-sweeper review.

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/auth Authentication, OAuth, credential pools P2 Medium — degraded but workaround exists labels Jul 30, 2026
@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 30, 2026
@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Merged via #77028. Your fix for both defects (refresh_token adoption without access_token + false auth-refreshed log) was cherry-picked with authorship preserved. A follow-up commit widens the source guard to also cover manual:device_code entries — the recommended quarantine-safe configuration — which was unreachable in the original PR. Credits to @imgyf for the source guard analysis.

kshitijk4poor added a commit that referenced this pull request Aug 2, 2026
The _sync_codex_entry_from_auth_store source guard returned early for
source='manual:device_code', which is the recommended quarantine-safe
configuration (hermes auth add openai-codex produces SOURCE_MANUAL_DEVICE_CODE).
The PR's fix for refresh_token adoption was unreachable for these entries.

Widen the guard to accept both 'device_code' and 'manual:device_code'.

Follow-up to #70111. Issue reporter (imgyf) confirmed this caused a
12-of-16 fleet outage on Aug 1.

Co-authored-by: imgyf <imgyf@users.noreply.github.com>
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
The _sync_codex_entry_from_auth_store source guard returned early for
source='manual:device_code', which is the recommended quarantine-safe
configuration (hermes auth add openai-codex produces SOURCE_MANUAL_DEVICE_CODE).
The PR's fix for refresh_token adoption was unreachable for these entries.

Widen the guard to accept both 'device_code' and 'manual:device_code'.

Follow-up to NousResearch#70111. Issue reporter (imgyf) confirmed this caused a
12-of-16 fleet outage on Aug 1.

Co-authored-by: imgyf <imgyf@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

4 participants