Repository navigation
fix(auth): prefer accounts without backoff in least-used rotation - #12375
Merged
diegosouzapw merged 2 commits intoSep 2, 2026
Merged
diegosouzapw merged 2 commits into
diegosouzapw merged 2 commits into
Conversation
The least-used strategy in getProviderCredentials() sorted candidates by lastUsedAt only. round-robin's fallback branch in the same function already ranks backoffLevel first (cheaper-inference#340), but least-used had no such tie-break, so after a 429 excluded the active account the replacement could be an account that was simply the oldest by lastUsedAt while still carrying a backoff from its own recent rate limit. That account served one request, its lastUsedAt moved to now, and the next request settled on the healthy account, which is the one-request detour with two cache misses reported in cheaper-inference#12279 on Codex. Prepend the same backoffLevel comparison to the least-used comparator: lower backoff first, then the existing never-used / oldest-lastUsedAt / priority order. Accounts with equal backoff (the common case, all zero) sort exactly as before. Test: tests/unit/sse-auth.test.ts seeds two accounts where the least recently used one carries backoffLevel 2 with an active rateLimitedUntil (so the auto-decay does not clear it) and selects with allowRateLimitedConnections so it stays in the pool; the healthy account must win. Fails on the base (backed-off account selected), passes with this change. Closes cheaper-inference#12279 Co-authored-by: Leon Marcos <leonaniagomez@gmail.com>
diegosouzapw
merged commit Sep 2, 2026
0389b07
into
cheaper-inference:release/v3.8.51
16 checks passed
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…eaper-inference#12375) The least-used strategy ranked candidates by lastUsedAt alone, so after a 429 excluded the active account the replacement could be one that was merely oldest while still carrying its own backoff — it served a single request before the next one settled on a healthy account, the one-request detour with two cache misses reported on Codex. least-used now applies the backoffLevel tie-break the round-robin fallback branch already had, ahead of the existing never-used / oldest / priority order. Validated in a combined worktree with all 25 PRs of this batch boarded together: typecheck:core clean, 443/443 node-runner tests plus 14/14 vitest across every test file the batch touches, and check-changelog-integrity, check:cycles (418 files), check:provider-consistency (272 REGISTRY entries, 355 canonical providers), check:docs-counts, check:docs-sync (42 locales) and check-file-size all green. Thanks @pacocartones.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
least-usedaccount strategy ranked candidates bylastUsedAtonly. After a 429 excluded the active account, the replacement could be an account that was merely the oldest bylastUsedAtwhile still carrying a backoff from its own recent rate limit; it served one request and the next request settled on a healthy account (the one-request detour with two cache misses reported on Codex).least-usednow applies the samebackoffLeveltie-break theround-robinfallback branch already has: lower backoff first, then the existing never-used / oldest / priority order.Related Issues
Validation
tests/unit/sse-auth.test.ts(67/67),npm run typecheck:core0,npm run check:changelog-integrityOK,git diff --checkcleannpm run lint— repository-wide eslint exit 0 (run with--pass-on-unpruned-suppressions; the literal command reports only pre-existing unused global suppressions on this base)release/v3.8.51@158647618; focused checks rerun afterwardTests Added Or Updated
tests/unit/sse-auth.test.ts: new casegetProviderCredentials least-used prefers an account without backoff over the least recently used one (#12279)(RED on the base: backed-off account selected; GREEN with the fix).Coverage Notes
src/sse/services/auth.ts: the added comparator lines are covered by the new test; the two existing least-used tests (never-used preference, oldest-timestamp preference) still pass and cover the equal-backoff path.Reviewer Notes
least-used, not only Codex. With all candidates atbackoffLevel0 (the common case) the order is identical to before.backoffLevelis auto-decayed to 0 ingetProviderCredentialsoncerateLimitedUntilhas passed, and rate-limited accounts are pre-filtered unless the caller setsallowRateLimitedConnections(combo targets withallowRateLimitedConnection). The tie-break therefore takes effect for accounts still inside their rate-limit window that a caller deliberately keeps in the pool; the test seeds exactly that shape.createProviderConnectiondoes not persistbackoffLevel, so the test writes it viaupdateProviderConnection.chatCore.tscalling the plain selector instead of the quota-preflight one) is the other half of the triage and is not changed here.