Skip to content

fix(combo): scope Claude model failures to the model, not the account - #12340

Merged
diegosouzapw merged 4 commits into
cheaper-inference:release/v3.8.51from
Kizuno18:fix/claude-per-model-failure-scope
Sep 18, 2026
Merged

diegosouzapw merged 4 commits into
cheaper-inference:release/v3.8.51from
Kizuno18:fix/claude-per-model-failure-scope

Conversation

@Kizuno18

@Kizuno18 Kizuno18 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

A priority combo whose steps are five different models on one Claude OAuth connection stops at step 1: whatever kills the first step is recorded against the connection, and the pre-dispatch cooldown gate then skips every later step because they share that connection. The drop order is silently a single-model route.

hasPerModelQuota() returns false for claude, so in markAccountUnavailable() the model-lockout branch never runs and the failure lands on the connection row instead. getPersistedConnectionCooldownSkipReason() takes it from there.

To keep the model-specific bug in #12334 out of the picture, I reproduced it with a first step naming a model that cannot exist. The 404 lands and step 2 is never dispatched; point step 2 at a different account and the same combo works:

COMBO: Trying model 1/2: claude/claude-bogus-model-x
COMBO: All models failed | claude/claude-bogus-model-x: model — model: claude-bogus-model-x (HTTP 404)
COMBO: Trying model 2/2: agy/claude-sonnet-4-6      ← only when step 2 is another account

A Claude OAuth connection multiplexes Fable 5, Opus 5/4.8/4.7/4.6, Sonnet and Haiku behind one credential — the multiplexing the doc comment on hasPerModelQuota already describes. Its quota is a different question, though: a 429 on a Max subscription is account-wide, and shouldMarkAccountExhaustedFrom429("claude", …) pins that in account-fallback-service.test.ts. Adding claude to hasPerModelQuota breaks that test, and rightly so.

So this splits the two: a new hasPerModelFailureScope() covers the non-quota statuses (404, 5xx, a stream that ended with no content), auth.ts uses it only when status !== 429, and quota scope stays exactly as it was. targetExhaustion gets the same treatment for 404, which names one model the account cannot serve rather than a bad connection — the empty-content 502 was already exempt through isEmptyContentFailure.

Related Issues

Validation

  • Change type: routing
  • Focused tests: tests/unit/combo-claude-per-model-scope.test.ts plus every existing test touching hasPerModelQuota or targetExhaustion — 17 files, 226 passing, 0 failing
  • npx eslint on the four touched files: same 3 pre-existing no-unused-vars errors in accountFallback.ts before and after (line 2413 shifts to 2437), no new ones
  • Reconciled with release/v3.8.51
  • Production-code changes come with a new test

Full unit suite on this branch: 47 failures. On the same checkout with the patch stashed: 46. The one extra is Vietnamese locale has no internal missing markers or empty fallbacks, which fails on a clean tree when run on its own:

not ok 1 - Vietnamese locale has complete key parity with English
not ok 2 - Vietnamese locale has no internal missing markers or empty fallbacks

so it is suite variance, not this change.

Tests Added Or Updated

  • tests/unit/combo-claude-per-model-scope.test.ts (new) — mirrors combo-provider-cooldown-sibling.test.ts: failure scope for claude, quota scope unchanged, the connection-level override still winning, and the two sides of the pre-dispatch gate (a real cooldown still protects siblings; no cooldown lets step 2 dispatch).

Coverage Notes

hasPerModelFailureScope() is covered by the new file. The auth.ts and targetExhaustion.ts call sites are covered indirectly by the existing lockout tests for gemini, github, nvidia, ollama, openrouter, vertex and agentrouter, which all still pass — they pin that nothing outside claude changed scope.

Reviewer Notes

The risk worth looking at is the 429 split. I deliberately left quota scope alone so a Max rate limit keeps retiring the account, since dropping to another model there would just burn a second upstream call. If you would rather have per-model quota for claude too, that is a one-line change to hasPerModelQuota plus an update to the shouldMarkAccountExhaustedFrom429 expectation — happy to do it either way.

Reproduced against diegosouzapw/omniroute:main @ sha256:88996a83 and the previous sha256:611c555b, on a single claude_max OAuth connection.

A priority combo whose steps are five models on one Claude OAuth connection
stops at step 1. hasPerModelQuota() returns false for the claude provider, so
markAccountUnavailable() records a model-specific 404 or 5xx against the
connection row, and getPersistedConnectionCooldownSkipReason() then skips every
sibling step before dispatch. Verified with a combo whose first step names a
model that cannot exist: the 404 lands and step 2 is never tried, while the
same combo works as soon as step 2 points at a different account.

A Claude OAuth connection multiplexes Fable 5, Opus 5/4.8/4.7/4.6, Sonnet and
Haiku behind one credential, which is the multiplexing hasPerModelQuota already
describes. Its quota is a separate question: a 429 on a Max subscription is
account-wide, and shouldMarkAccountExhaustedFrom429 pins that. So this adds
hasPerModelFailureScope() for the non-quota statuses and leaves 429 alone.

Combo exhaustion gets the same treatment for 404, which names one model the
account cannot serve rather than a bad connection. The empty-content 502 was
already exempt through isEmptyContentFailure.

Closes cheaper-inference#12334
diegosouzapw and others added 2 commits September 3, 2026 08:01
…caps

Move hasPerModelFailureScope into a leaf so frozen accountFallback.ts and
auth.ts stay at their file-size caps after cheaper-inference#12334. Register
combo-claude-per-model-scope.test.ts in Stryker tap.testFiles so the
mutation-test-coverage gate sees the covering unit test.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@cryptiklemur

Copy link
Copy Markdown
Contributor

Would this allow harnesses (like CC) to see that fable is 429ing, and automatically change to Opus, like it normally does in CC (without a proxy)?

@Kizuno18

Copy link
Copy Markdown
Contributor Author

@cryptiklemur Not for a 429 in this PR. OmniRoute still treats Claude 429s as account-wide, so another model on the same OAuth connection remains on cooldown. This fixes model-specific failures such as a 404, allowing the combo to try the next model on that connection. Distinguishing a Fable-only limit from a shared account limit would need a separate change.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Good fix — a Claude OAuth connection multiplexing Fable/Opus/Sonnet/Haiku
behind one credential shouldn't have a 404/5xx on one model exhaust the
whole connection and stall a priority combo at step 1. The
hasPerModelFailureScope() split (quota stays account-wide via
hasPerModelQuota, non-quota failures get model scope for Claude) is the
right generalization. Probed the test file locally: 7/7 pass. This is
complementary to #12957 in the same batch (different layer of the
model-lockout system — no file-line overlap, verified) — good to merge
either order.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks @Kizuno18 — merging via the release merge-train. Validated in local merge-train (.claude/worktrees/merge-train-20260918-111718-suite.log) on the devbox @ train tip 7bb373fba5e241964c0ffb17bd03a804700839bb, boarded with 55 sibling PRs: typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity green; 747/747 changed-area node:test cases + 476/476 vitest green (fast parity mode — the full suite ran today on the release tip via the base-red train and runs again on the 3b train). Merged --admin per merge-gates §7.

@diegosouzapw
diegosouzapw merged commit 6d0dc5a into cheaper-inference:release/v3.8.51 Sep 18, 2026
16 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…cheaper-inference#12340)

* fix(combo): scope Claude model failures to the model, not the account

A priority combo whose steps are five models on one Claude OAuth connection
stops at step 1. hasPerModelQuota() returns false for the claude provider, so
markAccountUnavailable() records a model-specific 404 or 5xx against the
connection row, and getPersistedConnectionCooldownSkipReason() then skips every
sibling step before dispatch. Verified with a combo whose first step names a
model that cannot exist: the 404 lands and step 2 is never tried, while the
same combo works as soon as step 2 points at a different account.

A Claude OAuth connection multiplexes Fable 5, Opus 5/4.8/4.7/4.6, Sonnet and
Haiku behind one credential, which is the multiplexing hasPerModelQuota already
describes. Its quota is a separate question: a 429 on a Max subscription is
account-wide, and shouldMarkAccountExhaustedFrom429 pins that. So this adds
hasPerModelFailureScope() for the non-quota statuses and leaves 429 alone.

Combo exhaustion gets the same treatment for 404, which names one model the
account cannot serve rather than a bad connection. The empty-content 502 was
already exempt through isEmptyContentFailure.

Closes cheaper-inference#12334

* fix(combo): extract Claude per-model failure scope to keep file-size caps

Move hasPerModelFailureScope into a leaf so frozen accountFallback.ts and
auth.ts stay at their file-size caps after cheaper-inference#12334. Register
combo-claude-per-model-scope.test.ts in Stryker tap.testFiles so the
mutation-test-coverage gate sees the covering unit test.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(claude): priority combo never advances past step 1 when all steps share one Claude OAuth connection

3 participants