fix(credential-pool): exhaust all entries sharing a failed API key (#68565 salvage) - #70135
Merged
Conversation
… 402 A 402/429/401 is an API-key–level failure (account out of balance, rate-limited, or key rejected), but the same key can back more than one pool entry — e.g. an explicit pool entry plus a `model_config` entry auto-seeded from `model.api_key`, both carrying the identical `runtime_api_key`. `mark_exhausted_and_rotate(api_key_hint=...)` only marked the *first* matching entry, leaving the sibling OK. `_select_unlocked()` then kept handing back the same depleted key, so the billing-recovery `continue` loop in the conversation retry path never converged: the request hung until the client disconnected (~2.5min observed against DeepSeek), emitting only `response.created` with no 402 ever surfaced to the user. Mark every entry sharing the failed key so the pool can reach the "no available entries" state and let the error propagate immediately. Adds a regression test covering two entries backed by the same key.
Follow-up to the #68565 salvage: batch the sibling _mark_exhausted calls behind a single _persist() instead of one auth.json write per sibling.
teknium1
force-pushed
the
salvage/68565-same-key-siblings
branch
from
July 23, 2026 15:09
2376d43 to
f4919db
Compare
Contributor
૮ >ﻌ< ა ci reviewran on f4919db all good! |
alt-glitch
added
type/bug
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Salvage of #68565 by @airclear:
mark_exhausted_and_rotate(api_key_hint=...)now exhausts every pool entry sharing the failed API key, not just the first match. A 402/429/401 is a key-level failure — when the same key backs multiple entries (e.g. an explicit pool entry plus themodel_configentry auto-seeded frommodel.api_key), marking only one left_select_unlocked()handing back the identical depleted key forever: the billing-recovery loop never converged and the request hung ~2.5 min with no error surfaced.Premise verified empirically: the PR's regression test fails on unpatched main (rotation returns the same-key sibling) and passes with the fix.
Changes
agent/credential_pool.py(contributor commit, cherry-picked): after marking the hinted entry, sweep siblings with the sameruntime_api_keyand exhaust them too. Per-entry_mark_exhaustedpreserves DEAD-vs-EXHAUSTED classification; guarded byif api_key_hint, so hint-less callers are unchanged.agent/credential_pool.py(follow-up):_mark_exhausted(persist=False)+ one batched_persist()for the sibling sweep — one auth.json write instead of one per sibling.tests/agent/test_credential_pool.py(contributor): regression test with two entries backed by one key.Validation
mark_exhausted_and_rotatereturns None, error propagatestest_credential_pool.py,test_credential_pool_routing.py,test_credential_pool_interrupt.py).HERMES_HOME: both same-key entries exhausted + persisted in one write, rotation lands on the distinct key, and exhausting that converges to None.Closes #68565's scope. Complements #69553/#69843 (attribution) — this covers the key-to-many-entries direction.
Credit
Core fix and regression test by @airclear (#68565), cherry-picked with authorship preserved. Persistence batching added on top.
Infographic