Skip to content

fix(routing): keep native turns retryable during quota cooldowns - #15486

Merged
diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.52from
JxnLexn:dev/turn-quota-retry
Oct 10, 2026
Merged

diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.52from
JxnLexn:dev/turn-quota-retry

Conversation

@JxnLexn

@JxnLexn JxnLexn commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Problem and resulting behavior

A valid native Responses continuation could be terminated with HTTP 400 after its accounts became temporarily unavailable. The client was told to start a new turn even though the blocker was a quota limit or cooldown. Generic account availability misses were also treated as model-level failures merely because the provider supports per-model quota.

Preserve the turn binding through temporary unavailability:

  • Recognized active model quota/rate-limit locks return HTTP 429; transient service locks return HTTP 503. Retry-After reflects the internal cooldown, not a guaranteed upstream reset time.
  • Account quota, cooldown, capacity, and policy filters use the normal dispatch gates. A generic availability miss no longer invalidates a pinned model or triggers an unnecessary model switch.
  • Model-specific lock safeguards, alternate-model eligibility checks, pending-tool/opaque-state safety, and the single-resume limit remain enforced. Existing bounded short-lock waits remain intact.

The account catalog repair remains separate in #15487.

Regression coverage

The isolated SQLite reproduction creates an exhausted session-quota account and a sibling in a persisted overload cooldown, with no active model lock. Before the fix, the same-turn continuation returns HTTP 400. After the fix it returns retryable unavailability, retains the pin, and succeeds in the same turn once the cooldown expires.

Additional cases cover opaque continuation state, pending tool calls, plain input with a healthy alternate, quota-lock recovery, permanent/unknown lock reasons, and the maximum resume count. Auto-resume tests now expect retryable quota responses while retaining the no-model-switch assertions. Account fixtures are cleaned between scenarios so earlier accounts cannot accidentally satisfy sibling failover.

Validation

  • 81 focused routing, continuation, quota-policy, and account-selection tests passed on the upstream branch.
  • 78 corresponding tests passed on the fork integration branch.
  • 493 Vitest tests and core TypeScript checks passed on both branches.
  • Formatting/lint hooks passed for the changed files.
  • Full repository CI is not claimed green. Base-red inherited: 🔴 Release branch not green: release/v3.8.52 #15306. Current CI also reports generated-skills drift, a translation-manifest mismatch for docs/architecture/QUALITY_GATES.md, and an existing dependency audit failure before ESLint. This patch changes no dependencies or generated documentation.

Deployment verification

Deployed the successful GitHub-built image to LeonAPI: ghcr.io/jxnlexn/omniroute:sha-39f7a67, revision 39f7a6746b54f31bf6e834bbcf5fd357eb9cf09a, AMD64 digest sha256:b5c02347debec6a8c3d3103bb4b4abb6f9489c22e928159d4cf86f14d3514e4d.

SQLite backup passed quick_check; the previous image is retained for rollback. Only the application container was recreated.

  • docker inspect omniroute: expected revision, healthy, zero restarts.
  • Local and public GET /api/health/ping: status: ok.
  • Two minimal POST /v1/responses requests to gpt-6-astra-combo, using identical synthetic thread/turn metadata: HTTP 200 with output text in 2.179 s and 2.926 s.

The quota/cooldown failure and recovery were reproduced in isolated tests. Production account and quota settings were not artificially changed for the smoke test; existing private conversations were not replayed.

@JxnLexn
JxnLexn requested a review from diegosouzapw as a code owner October 4, 2026 00:04
JxnLexn and others added 5 commits October 5, 2026 14:14
# Conflicts:
#	tests/unit/native-codex-auto-resume.test.ts
…unt level

modelScopedOnly made an account-level block (quota policy, persisted
cooldown, isModelAvailable miss) keep the pin for every turn, so a plain
turn with a healthy alternate sat on 503/429 until the account came back
instead of auto-resuming (cheaper-inference#13180/cheaper-inference#13564). resolvePinnedTurnUnusable() keeps
the pin only when the turn cannot safely move (opaque state, pending tool
call, no healthy alternate) and otherwise hands it to the alternate.
Adds a changelog fragment for the new model_cooldown 429/503 contract.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw merged commit 8d00ab5 into cheaper-inference:release/v3.8.52 Oct 10, 2026
32 of 51 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks @JxnLexn — this is in the upcoming release.

diegosouzapw added a commit that referenced this pull request Oct 10, 2026
… tip

The base advanced 125 commits after the first measurement (#15926 grew
codex.ts to 1526, #15486 regrew combo.ts to 1211), so the 1519 ceiling
failed CI. Rebuilt from the tip baseline @7510677f2f with --update:
35 frozen ceilings lowered, 5 removed, 6 testFrozen lowered, none raised.
diegosouzapw added a commit that referenced this pull request Oct 10, 2026
…ore decompositions (rail G2) (#16070)

* chore(quality): ratchet file-size baseline down after the combo/chatCore decompositions (rail G2)

Run check:file-size --update on release/v3.8.52 @35ef635c. combo.ts drops out
of the baseline (4080 -> 1198, under the 1200 cap) and chatCore.ts tightens
6426 -> 3938 after #12746/#12811 and #14725. 29 more frozen ceilings lowered,
3 more entries removed as under cap, 6 testFrozen ceilings lowered. Down-only:
no ceiling raised, no entry added. Frozen file entries 67 -> 63.

* chore(quality): re-measure the file-size ratchet on the fresh release tip

The base advanced 125 commits after the first measurement (#15926 grew
codex.ts to 1526, #15486 regrew combo.ts to 1211), so the 1519 ceiling
failed CI. Rebuilt from the tip baseline @7510677f2f with --update:
35 frozen ceilings lowered, 5 removed, 6 testFrozen lowered, none raised.
shipsfromrio added a commit to shipsfromrio/OmniRoute that referenced this pull request Oct 14, 2026
check:mutation-test-coverage --strict fails a PR when a unit test that
imports a mutated module is missing from stryker.conf.json tap.testFiles,
because its mutant kills would not count. Both new test files import
accountFallback.ts (isModelLocked, getModelLockoutInfo), and this PR's
Quality Gates run listed them. They now sit in alphabetical order in the
rate-limit-* block, as cheaper-inference#15486 did for its own tests. The gate's other
entries on the release tip (12 tests across 5 modules) predate this
branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants