Skip to content

fix(agent): rebind pool entry id after env credential refresh (salvage #79180) - #82149

Merged
teknium1 merged 1 commit into
mainfrom
salvage/79180-pool-entry-rebind
Aug 9, 2026
Merged

teknium1 merged 1 commit into
mainfrom
salvage/79180-pool-entry-rebind

Conversation

@teknium1

@teknium1 teknium1 commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Rate-limit rotation now quarantines the credential that is actually exhausted, instead of benching a healthy fallback key for days because the agent's pool entry id had gone stale.

Salvage of #79180 by @686f6c61 — cherry-picked onto current main with authorship preserved; applied cleanly, no follow-up fixes needed. Fixes #79156.

Root cause: _try_refresh_env_client_credentials adopts a key from the environment mid-run without calling sync_credential_pool_entry_id(), so the agent keeps pointing at the old pool entry. mark_exhausted_and_rotate resolves by credential_id first — ignoring the disagreeing api_key_hint — and quarantines the wrong entry: the healthy key gets a multi-day cooldown while the exhausted one stays live.

Changes (3 layers)

  • run_agent.py: rebind the pool entry id right after adopting an env credential; don't treat a pool-rotated key as a boot-time env adoption.
  • agent/credential_pool.py: when credential_id and api_key_hint disagree, trust the hint (the key actually in use).
  • Tests at both layers: tests/agent/test_credential_pool.py (+55) and tests/run_agent/test_env_credential_turn_refresh.py (+37).

Validation

Before After
Pool rotation → env refresh → new key 429s healthy fallback quarantined for days exhausted key quarantined, fallback stays live
Boot-time env adoption unchanged unchanged (guarded from false-positive)
Targeted suites 79/79 pass

Duplicate #79224 (strict subset, filed ~80 min later) will be closed with credit once this lands. Contributor also flagged an adjacent hermes auth reset no-op (_merge_disk_cooldown_state resurrecting EXHAUSTED) — separate issue, not in scope here.

Infographic

Quarantine the right key

Per-turn .env adoption could rewrite agent.api_key while leaving
_credential_pool_entry_id on a previously rotated fallback. The next 429
then marked the healthy fallback exhausted via credential_id precedence
(#79156).

- Sync pool entry id after a successful env credential refresh
- First look does not stomp a pool-rotated key with the env primary
- mark_exhausted_and_rotate prefers api_key_hint when it disagrees with
  credential_id

Fixes #79156
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

ran on d989b74

⚠️ Warnings

OSV vulnerability scan · View job

21 known vulnerabilities found in pinned dependencies.

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.


debug info

CI timings

CI timings · View report · View job

Wall time 3m55s vs 6m54s (-43.2%). 13 job(s) slower, 7 faster, 3 unchanged.

  • Python tests / Run tests slice 2/12: -27.0s
  • Python tests / Run tests slice 11/12: -27.0s
  • Python tests / Run tests slice 5/12: -20.0s
  • Python tests / Run tests slice 10/12: +16.0s
  • Python tests / Run tests slice 12/12: +6.0s

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/auth Authentication, OAuth, credential pools sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data labels Aug 9, 2026
@teknium1
teknium1 merged commit bf7c716 into main Aug 9, 2026
46 checks passed
@teknium1
teknium1 deleted the salvage/79180-pool-entry-rebind branch August 9, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(agent): per-turn env credential refresh leaves stale pool entry id — healthy fallback key quarantined on next 429

3 participants