Skip to content

fix(sse): lock the model on a 429 whose RetryInfo wait exceeds the queue budget - #16525

Merged
diegosouzapw merged 15 commits into
cheaper-inference:release/v3.8.52from
shipsfromrio:fix/429-long-retry-lockout
Oct 18, 2026
Merged

diegosouzapw merged 15 commits into
cheaper-inference:release/v3.8.52from
shipsfromrio:fix/429-long-retry-lockout

Conversation

@shipsfromrio

@shipsfromrio shipsfromrio commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: #15306

Addresses the queue stall reported in #16442, for every RetryInfo wait longer than the queue
budget: the 1104 s and 3009 s waits there, and the 86399s Gemini sends for an exhausted daily
quota. The failure-path gaps #16442 also describes stay open; see Notes.

Problem

A 429 body that names its own retry froze that key's limiter for the whole wait, while the router
heard about it for 3 s. updateFromResponseBody empties the key's reservoir until the stated wait
ends. The failure path does record a model lockout for the 429, whatever the generic lockout
setting, but, as #16442 traces, only at the API-key base cooldown of 3 s, because it does not read
error.details[].retryDelay. So 3 s later the router picks the frozen key again. Every dispatch
sits out requestQueue.maxWaitMs (30 s by default), fails as a local 503
Request exceeded queue budget, and only then falls back. Ten frozen keys ahead of the next working
target burned the 300 s combo timeout without one upstream call.

Measured on one production combo on 3.8.51, 2026-10-06 → 2026-10-12:

  • 7,479 failed vs 915 successful combo requests, almost all Combo global timeout (300000ms).
  • Between 02:08 and 16:46 UTC on 2026-10-12, 4,136 of 4,790 attempts to gemini-3.6-flash inside
    the combo were local queue 503s (p50 30,085 ms); only 86 came back as upstream 429s.
  • Of 646 body-stated waits Gemini sent between 2026-10-09 and 2026-10-12, 488 were longer than 1 h,
    88 between 31 s and 1 h, and 70 at or under 30 s.

Fix

  • updateFromResponseBody: a 429 that carries a typed Google RetryInfo detail whose
    retryDelay is longer than the connection's maxWaitMs now locks the model on that connection.
    This applies when the key's limiter is already scoped to the model (getLimiterKey: gemini and
    github per model, antigravity per quota family, codex per model scope).
    • The lock uses lockModel, which keys those same scopes (getModelLockKey), so it covers
      what the pause covered. The limiter is left unpaused.
    • Antigravity is the one exception. Its limiter is per quota family, but the lock goes on the
      exact model (lockExactModel), like its other quota locks (lockModelIfPerModelQuota, the
      core-owned quota state). An exhausted Claude model then does not take the whole Claude family
      down on that connection for up to a day.
    • Both lock functions keep the longer of two locks. So when the failure path then records its own
      short lockout (the 3 s in fix(resilience): Gemini 429 RetryInfo (details-only) is dropped before model lockout — 3s lock while the limiter pauses 1104s, requests stall 30s in queue #16442, through recordModelLockoutFailure), the RetryInfo lock
      stands.
    • The wait is read by retryInfoDetailsMs, the reader behind the google_rpc_retry_info
      provenance (now exported from retryAfterJson.ts). That includes its @type check and its
      24 h cap.
    • The name the client asked for is locked too, when an alias resolved it to that model, because
      account selection (and the combo pre-dispatch gate, for targets pinned to the connection) checks
      the requested name.
      • It is locked exactly, with lockExactModel, whose key isModelLocked already checks.
        Through lockModel's quota-family keys, an alias that crosses an antigravity family or a
        codex scope would lock its own family for the whole wait. For example, a gemini-2.5-pro
        alias for claude-sonnet-4-5 would take gemini-2.5-flash down on that connection.
      • chatCore now passes its pre-alias model as a new optional argument. Its end-of-request
        call runs on both legs (applyProviderFailureClassification in the streaming and
        non-streaming handlers).
      • The provider pipeline's call is unchanged, because its requestedModel is already the
        resolved one.
  • Everything else keeps pacing the limiter exactly as before.
  • withRateLimit: when the limiter that the 429 path just rebuilt is paused, and the pause
    outlasts the caller's remaining queue time, the call is turned away at once. It gets the
    queue-timeout code (RATE_LIMIT_QUEUE_TIMEOUT, 503), with the same classification and combo
    fallback as before, instead of waiting that time out. This covers the prose, untyped and
    connection-wide cases above.
  • updateFromHeaders: the 429 log no longer says pausing for 60s (the first line of the log
    in fix(resilience): Gemini 429 RetryInfo (details-only) is dropped before model lockout — 3s lock while the limiter pauses 1104s, requests stall 30s in queue #16442). That path evicts the limiter and takes no pause. The log now prints the real
    retry-after, the header as sent when it does not parse to a positive wait, or says there was
    none. That note is built by describeRetryAfter in rateLimitManager/headers.ts, next to
    parseResetTime. Built inline, it took updateFromHeaders to a complexity of 16 (cognitive 18)
    against the limit of 15 in the repository's ESLint rules, a new finding for
    check:complexity-ratchets on a changed file.

The new logic lives in open-sse/services/rateLimitManager/bodyRetryPause.ts. The three test-quiet
log helpers move to rateLimitManager/log.ts unchanged, so the new module logs the same way and
rateLimitManager.ts stays within its frozen size: 1312 → 1319 lines by the check:file-size
count, ceiling 1329. In chatCore.ts the new argument adds a line, and the sibling
updateFromHeaders call moves onto its if line, as the file already does for other one-line
guards: 3935 → 3934 on the branch. release/v3.8.52 now sits exactly at the 3938 ceiling after
#16456, and the merge with it measures 3937, so file-size-baseline.json is untouched.

Why a model lockout

  • It is the signal both routers already read, independent of the opt-in generic lockout setting:

    • the combo pre-dispatch gate (open-sse/services/combo/executeTargetGates.ts, isModelLocked)
      for targets pinned to a connection;
    • account selection (src/sse/services/auth.ts, isModelLocked) for every request.

    The lock calls themselves (lockModel, lockExactModel) are unconditional.

  • It has the scope the pause had, or a narrower one on antigravity. Other models on the same key
    keep serving where the quota is per model, and so do other keys.

  • When every connection for the model is locked, account selection answers allRateLimited with
    a model-scoped cooldown ("getProviderCredentials reports allRateLimited when every account is
    model-locked" in tests/unit/sse-auth.test.ts pins this), and the cooldown-aware retry does not
    sit out a wait longer than maxRetryWaitMs. The request, or the combo target, fails at once
    instead of costing 30 s and a 503 per key.

  • It is visible (GET /api/rate-limits lists model lockouts) and operators can clear it.
    POST /api/resilience/reset clears model lockouts, and a successful request on the same name
    clears its lock (clearModelLock in src/sse/handlers/chat.ts), like every other model lock.
    The frozen reservoir showed up nowhere and no reset touched it. Only a rebuilt limiter ended it
    early: a restart, a settings or override change, a protection toggle, or a later 429 with
    headers on the same limiter, which the frozen queue itself kept from happening. That is also why
    the lock does not come with a pause: a reset would clear the lock and leave an invisible pause
    behind.

Why the fail-fast cannot turn away a request the queue would serve

  • The pause is recorded per limiter instance, and only on a limiter built for it. The recorded
    end is taken before the build, because Bottleneck starts the refresh clock in the datastore
    constructor (_lastReservoirRefresh) and the heartbeat patch keeps that anchor. So the recorded
    end never runs past the real refill, even when the build is slow. On an older limiter the refill
    can come sooner, so nothing is recorded there.
  • Any later update to the reservoir or its refresh schedule forgets the record. That covers
    header learning, settings and override changes, and a relaxed limiter never fails calls fast.
  • Guards cover all three. A short pause still holds the call and then serves it. A pause relaxed
    by later headers no longer fails calls fast. A limiter build that stalls for 2.5 s does not stop a
    call with 3 s of budget from being served by the refill 1.5 s later.

Tests

tests/unit/rate-limit-long-body-retry-lockout.test.ts (new, end to end through the manager, 13
cases; it clears RATE_LIMIT_MAX_WAIT_MS at load, because that variable moves the default 30 s
queue budget the cases assume):

# Case Base 7f3cbb046b This PR
1 A 429 with a daily-quota RetryInfo (86399s) locks the model on that connection, leaves the key's other models free, and the next call on the key is served at once ✖ no lock (isModelLocked is false) ✔
2 The lock never outlasts the 24 h retry-hint ceiling (999999s) ✖ no lock ✔
3 A 20 s RetryInfo fits the 30 s queue budget, so the limiter pauses; a caller with 5 s left is turned away at once ✖ waited 5015 ms ✔
4 With no model to lock, a long pause still turns callers away at once ✖ waited 5010 ms ✔
5 A prose hint never becomes a lock; its pause turns callers away at once ✖ waited 5011 ms ✔
6 A retryDelay outside a typed RetryInfo detail never becomes a lock; its pause turns callers away at once ✖ waited 5017 ms ✔
7 On a connection-wide limiter (openai) a long RetryInfo locks nothing and keeps the pause, so another model on the connection is turned away at once ✖ waited 5004 ms ✔
8 The requested (alias) name is locked with the resolved model ✖ not locked ✔
9 The sequence in #16442: a 1104s RetryInfo, then the failure path's own 3 s lockout (recordModelLockoutFailure); the lock still lasts the RetryInfo wait ✖ only the 3 s lock ("the lockout shrank to 3000ms") ✔
10 Guard: a short pause inside the budget holds the call and then serves it ✔ ✔
11 Guard: once the reservoir is relaxed, a recorded pause no longer fails calls fast ✔ ✔
12 Guard: the per-target deadline (abort signal) ends a wait behind a pause ✔ ✔
13 Guard: a limiter build that stalls 2.5 s does not turn away a call the refill serves 1.5 s later ✔ ✔

The base column is this file run against a checkout of 7f3cbb046b (9 fail, 4 pass); with this PR
all 13 pass. The four guards pass on both sides on purpose: they pin the behaviour the fix must not
break.

tests/unit/rate-limit-body-retry-pause.test.ts (new, 7 timer-free cases with an injected clock):

  • the fail-fast comparison;
  • what forgets a recorded pause;
  • the full lockout decision table: status, scope, budget, maxWaitMs = 0, prose, untyped and
    other-typed retryDelay, and the 24 h cap;
  • the requested-name lock;
  • the requested-name lock is exact: an antigravity alias across quota families and a codex alias
    across scopes leave their own family serving (through lockModel this case fails);
  • on antigravity and agy the model itself is locked exactly, so the rest of its quota family keeps
    serving (through lockModel this case fails);
  • retryInfoDelayMs.

It imports the new module, so it cannot run on the base.

tests/unit/ratelimitmanager-headers-split.test.ts (one new case) pins describeRetryAfter: a
parsed wait, a zero or unparsable header as sent, and no header. This case was not run on this
machine, CI runs it; its six inputs were checked by calling the function directly.

Both new test files import accountFallback.ts, a mutated module, so they are listed in
stryker.conf.json tap.testFiles, as check:mutation-test-coverage --strict requires. The
gate's other entries on 9ecddd15a1 (12 tests across 5 modules) predate this branch.

Also run locally (Windows 11, Node 24.19.0):

  • npm run typecheck:core: clean on the branch, on its merge with 47688ba5ea, and on its merge
    with 9ecddd15a1, the current release/v3.8.52 tip.
  • ESLint with config/quality/eslint-suppressions.json, and Prettier, on the changed sources and
    tests: clean. The pre-commit hooks passed on every commit (lint-staged, docs-sync,
    check:any-budget:t11, check:tracked-artifacts, check:ai-attribution).
  • node scripts/check/check-cycles.mjs --ratchet: 14 cycles, at the ceiling of 14.
  • npm run check:file-size, on the branch and on its merges with 47688ba5ea and 9ecddd15a1:
    the only violation is open-sse/services/combo/executeTargetAttempt.ts (1288 > 1284). refactor(combo): route strategy-specific branching through a typed strategy registry (rail 3.8.52 R0.4a) #16206
    grew it from 1284 without a baseline entry; this PR does not touch it. On the merge with
    9ecddd15a1, chatCore.ts measures 3937 (ceiling 3938) and rateLimitManager.ts 1319
    (ceiling 1329).
  • npm run check:docs-all: pass.
  • On the merge with 47688ba5ea, the 28 neighbouring unit files (list below) plus
    combo-model-lock-pre-dispatch-wait-16319, gemini-deprecated-model-lockout and
    combo-priority-all-circuit-open: 315 of 317 pass. One failure is rate-limit-learned-cap-13594,
    whose after() hook cannot delete its temp SQLite directory on Windows (EPERM), the same on the
    base. The other is a wall-clock setup step in rate-limit-manager, described below.
  • The same 31 files on the merge of de911917aa (the head before the describeRetryAfter commit)
    with 9ecddd15a1, which includes fix(sse): honor Google RPC ErrorInfo quota-reset metadata as an upstream retry hint #16529: 326 of 327 pass, including the cases fix(sse): honor Google RPC ErrorInfo quota-reset metadata as an upstream retry hint #16529 added to
    account-fallback-retry-after-json and classify429. The one failure is again the after()
    hook of rate-limit-learned-cap-13594 (EPERM). rate-limit-manager passed all 19 tests, then
    again kept its process alive until a 300 s cap.
  • rate-limit-manager-queue-bound, "timeout before dispatch (<80ms not 200ms)", is a wall-clock
    bound: a 30 ms budget must reject in under 80 ms. On this busy Windows machine it went over in
    three of this branch's long runs (120, 146 and 131 ms) and passed in the others, including the
    final one. To tell the diff from the load, a replica of that case (same imports and hooks,
    printing the measured time) ran 40 times on each side, alternating base and branch, each run in
    a fresh process: median 40 vs 39 ms, p90 47 vs 49 ms, and the base went over 80 ms once
    (144 ms), the branch never (max 71 ms). This change adds only a synchronous WeakMap lookup to
    that path.
  • combo-model-lock-pre-dispatch-wait-16319 (from fix(resilience): wait out a short model lock instead of failing a combo instantly (#16319) #16360) bounds its quota-lock case at 1000 ms of
    wall clock. It went over once (3629 ms) inside a 14-minute run of ten files on the merge with
    e2d5747872. Alone, alternating the merge with 47688ba5ea and that tip in fresh processes,
    three runs a side, all six passed: 37–151 ms on the merge, 53–232 ms on the tip.
  • rate-limit-manager, "idle-capacity watchdog preserves a real Bottleneck minTime wait", polls
    every 5 ms, for at most 1 s, for the job to sit in Bottleneck's 100 ms minTime wait. In the
    20-minute run above it never saw it (Bottleneck did not place the minTime-delayed job in RUNNING). It passed in this branch's four earlier long runs, and alone on both sides (1145 ms on
    the merge, 666 ms on the tip). This change adds no timer, and the only line it adds to
    withRateLimit returns at once when no body pause is recorded.
  • On this Windows machine, rate-limit-manager.test.ts keeps its process alive for minutes after
    all 19 tests pass, with or without this branch. Run alone, it hit a 240 s cap on the tip
    47688ba5ea and on the merge with it, and a 200 s cap on the merge with e2d5747872.
The 28 unit files

antigravity-429-quota-cooldown, bug-7940-gemini-retrydelay, chat-rate-limit-body-lock,
classify429, combo-model-lockout-honors-reset-1308, provider-execution-pipeline,
rate-limit-enhanced, rate-limit-learned-cap-13594, rate-limit-long-body-retry-lockout,
rate-limit-body-retry-pause, rate-limit-manager-queue-bound, rate-limit-manager,
rate-limit-overrides-restart, rate-limit-queue-context, rate-limit-remaining-budget,
rateLimitManager-idle-eviction, rateLimitManager-update-sequencing,
ratelimit-reservoir-refresh, ratelimitmanager-headers-split, muse-code-oauth,
upstream-status-restatement, account-fallback-retry-after-json, chatcore-stream-error-result,
chatcore-quota-share-consumption, chatcore-streaming-quota-share,
combo-quota-exhaustion-only-fallback, repro-7387-sticky-quota-exhausted,
rate-limit-abandoned-queued-job (all under tests/unit/, --test-concurrency=1).

Notes

  • Complementary to fix(resilience): wait out a short model lock instead of failing a combo instantly (#16319) #16360 (merged after this branch was cut), which waits out a short model
    lock in a combo.
  • fix(sse): honor Google RPC ErrorInfo quota-reset metadata as an upstream retry hint #16529 (also merged after this branch was cut):
    • Its compound durations (2h7m23s) reach both readers used here. The limiter pause
      (parseRetryAfterFromBody) and the lock (retryInfoDetailsMs) both parse through
      parseDelayString, so they see the same value. A typed RetryInfo with
      retryDelay: "2h7m23s", which the base read as 2 s, now locks the model for 2 h 7 min 23 s.
    • Its ErrorInfo quota-reset metadata does not reach this path, because
      parseRetryAfterFromBody reads details[] only for retryDelay fields. An ErrorInfo-only
      429 therefore takes neither the pause nor the lock from that detail; a prose hint in its message
      is handled as before. When a body carries both details, the lock lasts the RetryInfo wait.
    • A throwaway test on the merge with 9ecddd15a1 checked the compound and ErrorInfo cases
      (4 of 4 pass). It is not part of this PR, because the branch predates fix(sse): honor Google RPC ErrorInfo quota-reset metadata as an upstream retry hint #16529.
  • fix(resilience): Gemini 429 RetryInfo (details-only) is dropped before model lockout — 3s lock while the limiter pauses 1104s, requests stall 30s in queue #16442. This PR covers its stall for RetryInfo waits past the queue budget. For a typed
    RetryInfo, it also covers its point (b): a limiter pause longer than the model lockout should
    count for target eligibility. Still open there:
    • parseUpstreamError() and checkFallbackError() do not read error.details[].retryDelay.
      The failure path's own lockout therefore stays at the base cooldown for a details-only
      RetryInfo, and its log line still prints that base (3 s) next to this PR's 🔒 line. Past
      the queue budget, the lock above covers the key; inside it, the limiter paces the key.
    • The Gemini 429 block in checkFallbackError() returns before the operator providerErrorRules
      lookup, so no rule can lengthen a Gemini 429 lockout.
    • A synthesized queue-budget 503 is not recorded as a lockout (its point (a)). With this PR, that
      503 comes at once when the recorded pause outlasts the caller's queue time, instead of after
      maxWaitMs.
  • Not in this PR:
    • gemini-3.6-flash is still missing from open-sse/config/geminiRateLimits.json, so the RPD
      classifier cannot tell its daily quota from a per-minute limit.
    • A combo deadline reaches withRateLimit only as an abort signal, so the fail-fast compares
      against the queue budget, not against the time left in the combo.
    • A long RetryInfo on a non-429 status keeps the old pause.
    • The decision still runs only when parseRetryAfterFromBody finds a positive wait, as the
      pause always has. A body whose first retryDelay is invalid or zero therefore hides a later
      typed RetryInfo from both, on the base as here. Google's 429 bodies carry a single
      RetryInfo, so this stays as it was to keep the change to the reported defect.
    • The combo gate checks target.connectionId || "", so an unpinned target relies on account
      selection to skip the locked connection. This is existing behaviour for every model lock.
    • parseRetryAfterFromBody has no ceiling, while the typed reader caps at 24 h. A prose,
      untyped or connection-wide retryDelay longer than 24 h therefore still pauses the limiter for
      its full value, as on the base.
    • A background-task redirect reassigns model in chatCore before dispatch, so only the
      degraded model is locked, not the name the client sent. The next request on that name reaches
      the upstream and gets a 429 back, instead of waiting out the queue as on the base.
  • Changelog fragment: changelog.d/fixes/0000-429-long-body-retry-lockout.md, with the 0000-
    prefix the other fragments opened before their PR number use.

…udget

A 429 body that names its own retry (Gemini RetryInfo.retryDelay) froze
that key's limiter for the whole wait. Gemini answers an exhausted daily
quota with retryDelay "86399s", and nothing else recorded it: no model
lockout, no cooldown. The combo kept routing back to the frozen key, every
request sat out the full queue budget (maxWaitMs, 30s by default) and
failed as a local 503 "Request exceeded queue budget"; ten such keys
burned a 300s combo timeout without one upstream call.

- updateFromResponseBody: on a 429, a body retry longer than the
  connection's maxWaitMs now locks that model on the connection (capped
  at 24h) and leaves the limiter unpaused. The combo pre-dispatch gate and
  account selection already skip on isModelLocked(). Shorter waits keep
  pacing the limiter as before.
- withRateLimit: a limiter pause recorded on a freshly built limiter that
  outlasts the caller's remaining queue time now rejects at once with the
  queue-timeout code instead of waiting that time out. Any later reservoir
  change forgets the pause, so a relaxed limiter never turns calls away.
- updateFromHeaders: the 429 log no longer claims "pausing for 60s"; the
  header path resets the limiter and sets no pause.

The new logic lives in open-sse/services/rateLimitManager/bodyRetryPause.ts
so rateLimitManager.ts stays under its file-size ceiling.

Tests: tests/unit/rate-limit-long-body-retry-lockout.test.ts (4 cases
fail on the base, 3 guards pass on both).
Narrow the long body-retry lockout:

- Only a Google RetryInfo.retryDelay converts to a model lockout. Prose
  "retry after Ns" hints stay a limiter pause, so they never bypass the
  lockout maxCooldownMs cap the way authoritative hints may.
- Only a limiter already scoped to the model (gemini, github, antigravity
  family, codex scope) converts. A connection-wide limiter keeps pausing the
  whole connection: locking one model there would let the connection's other
  models through the paused key.
- Any update to the reservoir or its refresh schedule forgets a recorded
  pause, not only a reservoir reset, so a later refresh change never leaves a
  stale fail-fast behind.
- The lockout log moves into bodyRetryPause (logger injected, as the wedge
  watchdog already does) to keep rateLimitManager.ts within its frozen size.

Tests: the timer-free decisions move to rate-limit-body-retry-pause.test.ts
with an injected clock, and the end-to-end file gains the prose and
connection-wide cases with wider timing margins (5 s budget, 2.5 s "at once").
Three gaps in the long body-retry lockout:

- The lock was written under the resolved model only, while the combo
  pre-dispatch gate and account selection check the name the client asked
  for. With a custom alias the gate missed the lock and, now that the limiter
  is not paused, the request went upstream again. chatCore passes its
  pre-alias model to updateFromResponseBody, which locks it with the
  resolved one, as chatCore's own quota path already does.
- Any details[].retryDelay counted as RetryInfo. The lockout now reads the
  wait through retryInfoDetailsMs, the typed reader behind the
  google_rpc_retry_info provenance (24h cap included), so an untyped or
  differently typed retryDelay stays a pause.
- The recorded pause end was taken after the limiter was built, while
  Bottleneck starts the refresh clock at construction, so a slow build
  overstated the pause and the fail-fast could turn away a call the refill
  would serve. The time is now taken before the build.

The test-quiet log helpers move to rateLimitManager/log.ts so
bodyRetryPause logs directly and rateLimitManager.ts stays within its frozen
size with the new parameter.
…amily

The alias lock went through lockModel, whose keys group antigravity
models by quota family and codex models by scope. An alias that crosses
them locked its own family for the whole RetryInfo wait: a gemini-2.5-pro
alias for claude-sonnet-4-5 took gemini-2.5-flash down on that connection,
and a gpt-5.5 alias for the spark model took gpt-5.4 down. The requested
name is now locked with lockExactModel, the exact key isModelLocked
already checks, so the combo gate and account selection still skip the
alias and nothing else is locked.

The anchor guard now measures the refill from the limiter's construction
instead of requiring a fixed second of waiting after the stall, which a
slow runner could use up without anything being wrong.
…relock

After a 429, src/sse/services/auth.ts records its own model lockout at
the API-key base cooldown (3 s). That write goes through lockModel,
which keeps the longer lock, so the RetryInfo lock survives it. On the
base only the 3 s lock exists, which is the stall cheaper-inference#16442 reports: the
router picks the model again after 3 s and every request waits out the
queue budget. The new case replays that sequence and fails on the base
with "the lockout shrank to 3000ms".
…ts family

Antigravity's limiter is per quota family, so the scoped model reached
lockModel, which keys antigravity by quota family too: one exhausted
Claude model on a connection locked the whole Claude family there for
up to a day. Antigravity's other quota locks are exact
(lockModelIfPerModelQuota, the core-owned quota state), because a
confirmed exhaustion of one model must not lock its family, so the
RetryInfo lock now uses lockExactModel on antigravity and agy too.

The docs, the changelog fragment and the test messages now say that
the combo pre-dispatch gate sees the lock only for targets pinned to
the connection; account selection covers the rest. The end-to-end test
clears RATE_LIMIT_MAX_WAIT_MS, which moves the default 30 s queue
budget its cases assume.
The 429 log line this branch rewrote said "no retry-after header"
whenever the header did not parse to a positive wait, including a
literal "Retry-After: 0" or a value it could not read. It now says so
only when the header is absent, and otherwise prints the header as
sent.
The fragment was named 429-long-body-retry-lockout.md, which reads as
PR cheaper-inference#429 next to its <PR>-<slug> siblings. Fragments opened before their
PR number exists use the 0000- prefix (25 of them on this branch,
contributor PRs included), so this one does too.
Passing the requested model to updateFromResponseBody added a line to
open-sse/handlers/chatCore.ts. On the current release/v3.8.52 tip the file
sits exactly at its frozen 3938-line ceiling (cheaper-inference#16456 added three lines), so
the merge measured 3939 and check-file-size failed. The sibling
updateFromHeaders call now fits on its if line, like the single-line
`if (toolSummary) log?.debug?.(...)` guard in the same file. The branch
drops to 3934 lines and the merge to 3937, without touching
file-size-baseline.json.
The note this branch added to the 429 log line (the parsed wait, the
header as sent, or its absence) pushed updateFromHeaders past the
ceilings that check:complexity-ratchets enforces on a PR's changed
files: cyclomatic 16 and cognitive 18, against 15. Against the release
tip, rateLimitManager.ts went from 3 to 4 complexity findings and from
2 to 3 cognitive-complexity ones, which the new-code mode reports as a
regression. The note is now describeRetryAfter() in
rateLimitManager/headers.ts, the zero-import leaf that already holds
parseResetTime, and the log line reads the same. A unit test pins its
cases: a parsed wait, a zero or unparsable value as sent, and no header.
rateLimitManager.ts drops from 1322 to 1319 lines by the check:file-size
count.
check:mutation-test-coverage --strict fails a PR when a unit test that
imports a mutated module is missing from stryker.conf.json tap.testFiles,
because its mutant kills would not count. Both new test files import
accountFallback.ts (isModelLocked, getModelLockoutInfo), and this PR's
Quality Gates run listed them. They now sit in alphabetical order in the
rate-limit-* block, as cheaper-inference#15486 did for its own tests. The gate's other
entries on the release tip (12 tests across 5 modules) predate this
branch.
@shipsfromrio

Copy link
Copy Markdown
Contributor Author

CI on 8931826c82, tested as the merge with 9ecddd15a1 (CI 38489565761, Quality Gates 38489565904):

  • The 21 cases this PR adds or extends pass, each in one CI unit shard and one fast-path shard. complexity-ratchets now passes (describeRetryAfter, 09a145da6a).
  • mutation-test-coverage --strict still fails, on the same 12 unit tests it lists for 9ecddd15a1 alone (Quality Gates push run 38444167890). Eight were missing on 47688ba5ea; codex-reasoning-replay-retry, authz/local-only-bypass-spawn-denylist, request-logger-url-secret-masking and claude-thinking-continuity landed since. The two test files this PR adds are listed (8931826c82). 🔴 Release branch not green: release/v3.8.52 #15306 does not list this gate.
  • check:dashboard-typecheck prints the same report here and on the base push run: 196 errors, 86 baselined ones gone, and one new TS2339 in src/app/(dashboard)/dashboard/providers/[id]/components/ConnectionRow.tsx, a file this PR does not touch. The base run also fails file-size ("3 gate(s) failed: file-size mutation-test-coverage check:dashboard-typecheck"); here only the other two fail.
  • Every other failing job also fails on 9ecddd15a1's push runs (CI 38444167795, Quality Gates 38444167890): Lint, Docs Sync (Strict), Vitest, Integration Tests (1/2), Unit Tests (2/8) to (8/8), the four fast-path shards and Merge integrity. The unit and fast-path shards fail on the same ten test files on both sides (below). Unit Tests (5/8), (6/8) and (7/8) here run out of heap after the test summary, as (5/8) and (6/8) do on the base.
  • Build lost its runner six minutes in, during next build ("The runner has received a shutdown signal"). On the base push run it has not left the queue.
The ten failing unit test files, on both sides

codex-provider-overlay-12782, combo-dispatch-prelude, combo-scoring-inspector, messages-count-tokens-route, model-hidden-dashboard-modality-read, model-test-route, oauth-providers-config, quality-rail-gate-membership, route-explainability, server-owned-tool-loop-flag (all under tests/unit/).

#16528 (@skygunner, Fixes #16442) closes the failure-path gap listed in Notes > #16442. parseUpstreamError reads error.details[].retryDelay, and createErrorResult relays it as Retry-After, so markAccountUnavailable locks the model for the RetryInfo window instead of 3 s. The two PRs complement each other:

…ference#16528 RetryInfo lockout

cheaper-inference#16528 relays a details-only RetryInfo window into the model lockout, but
updateFromResponseBody still emptied the per-model reservoir for that window,
so a call reaching the key before it ended (after the resilience reset clears
the lockout) sat out the whole queue budget. The new case fails on the tip
without this PR and passes with it. Changelog and RESILIENCE_GUIDE now say
what this adds on top of cheaper-inference#16528.
@diegosouzapw
diegosouzapw merged commit c76ee07 into cheaper-inference:release/v3.8.52 Oct 18, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants