Repository navigation
fix(routing): keep native turns retryable during quota cooldowns - #15486
Merged
diegosouzapw merged 7 commits intoOct 10, 2026
Merged
diegosouzapw merged 7 commits into
diegosouzapw merged 7 commits into
Conversation
# Conflicts: # tests/unit/native-codex-auto-resume.test.ts
…unt level modelScopedOnly made an account-level block (quota policy, persisted cooldown, isModelAvailable miss) keep the pin for every turn, so a plain turn with a healthy alternate sat on 503/429 until the account came back instead of auto-resuming (cheaper-inference#13180/cheaper-inference#13564). resolvePinnedTurnUnusable() keeps the pin only when the turn cannot safely move (opaque state, pending tool call, no healthy alternate) and otherwise hands it to the alternate. Adds a changelog fragment for the new model_cooldown 429/503 contract. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
diegosouzapw
merged commit Oct 10, 2026
8d00ab5
into
cheaper-inference:release/v3.8.52
32 of 51 checks passed
Collaborator
|
Thanks @JxnLexn — this is in the upcoming release. |
diegosouzapw
added a commit
that referenced
this pull request
Oct 10, 2026
…ore decompositions (rail G2) (#16070) * chore(quality): ratchet file-size baseline down after the combo/chatCore decompositions (rail G2) Run check:file-size --update on release/v3.8.52 @35ef635c. combo.ts drops out of the baseline (4080 -> 1198, under the 1200 cap) and chatCore.ts tightens 6426 -> 3938 after #12746/#12811 and #14725. 29 more frozen ceilings lowered, 3 more entries removed as under cap, 6 testFrozen ceilings lowered. Down-only: no ceiling raised, no entry added. Frozen file entries 67 -> 63. * chore(quality): re-measure the file-size ratchet on the fresh release tip The base advanced 125 commits after the first measurement (#15926 grew codex.ts to 1526, #15486 regrew combo.ts to 1211), so the 1519 ceiling failed CI. Rebuilt from the tip baseline @7510677f2f with --update: 35 frozen ceilings lowered, 5 removed, 6 testFrozen lowered, none raised.
shipsfromrio
added a commit
to shipsfromrio/OmniRoute
that referenced
this pull request
Oct 14, 2026
check:mutation-test-coverage --strict fails a PR when a unit test that imports a mutated module is missing from stryker.conf.json tap.testFiles, because its mutant kills would not count. Both new test files import accountFallback.ts (isModelLocked, getModelLockoutInfo), and this PR's Quality Gates run listed them. They now sit in alphabetical order in the rate-limit-* block, as cheaper-inference#15486 did for its own tests. The gate's other entries on the release tip (12 tests across 5 modules) predate this branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and resulting behavior
A valid native Responses continuation could be terminated with HTTP 400 after its accounts became temporarily unavailable. The client was told to start a new turn even though the blocker was a quota limit or cooldown. Generic account availability misses were also treated as model-level failures merely because the provider supports per-model quota.
Preserve the turn binding through temporary unavailability:
Retry-Afterreflects the internal cooldown, not a guaranteed upstream reset time.The account catalog repair remains separate in #15487.
Regression coverage
The isolated SQLite reproduction creates an exhausted session-quota account and a sibling in a persisted overload cooldown, with no active model lock. Before the fix, the same-turn continuation returns HTTP 400. After the fix it returns retryable unavailability, retains the pin, and succeeds in the same turn once the cooldown expires.
Additional cases cover opaque continuation state, pending tool calls, plain input with a healthy alternate, quota-lock recovery, permanent/unknown lock reasons, and the maximum resume count. Auto-resume tests now expect retryable quota responses while retaining the no-model-switch assertions. Account fixtures are cleaned between scenarios so earlier accounts cannot accidentally satisfy sibling failover.
Validation
docs/architecture/QUALITY_GATES.md, and an existing dependency audit failure before ESLint. This patch changes no dependencies or generated documentation.Deployment verification
Deployed the successful GitHub-built image to LeonAPI:
ghcr.io/jxnlexn/omniroute:sha-39f7a67, revision39f7a6746b54f31bf6e834bbcf5fd357eb9cf09a, AMD64 digestsha256:b5c02347debec6a8c3d3103bb4b4abb6f9489c22e928159d4cf86f14d3514e4d.SQLite backup passed
quick_check; the previous image is retained for rollback. Only the application container was recreated.docker inspect omniroute: expected revision, healthy, zero restarts.GET /api/health/ping:status: ok.POST /v1/responsesrequests togpt-6-astra-combo, using identical synthetic thread/turn metadata: HTTP 200 with output text in 2.179 s and 2.926 s.The quota/cooldown failure and recovery were reproduced in isolated tests. Production account and quota settings were not artificially changed for the smoke test; existing private conversations were not replayed.