Skip to content

fix(combo): model lockout honors long upstream quota reset - #4429

Merged
diegosouzapw merged 2 commits into
release/v3.8.32from
fix/port-issue-1308-model-lockout-honors-reset
Jun 20, 2026
Merged

diegosouzapw merged 2 commits into
release/v3.8.32from
fix/port-issue-1308-model-lockout-honors-reset

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Summary

  • A combo per-model lockout capped at the short base cooldown (~minutes) and discarded the long upstream quota reset the proxy had already parsed (e.g. Antigravity Resets in 160h27m24s, a Retry-After header). The exhausted model was therefore retried again within minutes, hammering an upstream that won't recover for hours/days.

Root cause

checkFallbackError/parseRetryFromErrorText already extract the long reset into cooldownMs, and cooldownMs is in scope at both combo model-lockout call sites in open-sse/services/combo.ts — but those sites passed exactCooldownMs as 0 (exponential backoff) or baseCooldownMs, never the parsed reset. Separately, the Antigravity executor's own parseRetryFromErrorMessage regex /reset (?:after|in)/i did not match the plural Resets in … phrasing.

Fix

  • New exported pure helper selectLockoutCooldownMs(parsedCooldownMs, { baseCooldownMs, useExponentialBackoff }) in accountFallback.ts: returns the parsed reset when it exceeds the base cooldown, else preserves the previous behavior (0 → exponential backoff, or base when backoff is disabled). recordModelLockoutFailure already honors a positive exactCooldownMs exactly (bypassing backoff/cap).
  • Both combo lockout call sites now pass selectLockoutCooldownMs(cooldownMs, mlSettings).
  • Antigravity parseRetryFromErrorMessage regex reset → resets? so Resets in 160h27m24s matches.

Test plan

  • New regression test tests/unit/combo-model-lockout-honors-reset-1308.test.ts (helper 3 cases + end-to-end lockout ~160h via getModelLockoutInfo + central parseRetryFromErrorText + Antigravity parseRetryFromErrorMessage) — RED before, GREEN after (6/6)
  • Regression: antigravity-429-quota-cooldown + executor-antigravity — 31 pass / 0 fail
  • npx eslint — 0 errors · npm run typecheck:core — clean · npm run check:any-budget:t11 — PASS · npm run check:file-size — OK (baseline bumped with _rebaseline_2026_06_20_1308… justification)

Attribution

Thanks to @Ansh7473 for the original report.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

… 9router#1308)

A combo per-model lockout capped at the short base cooldown (~minutes) and
discarded the long upstream quota reset that checkFallbackError/
parseRetryFromErrorText had already parsed (e.g. Antigravity "Resets in
160h27m24s"), so an exhausted model kept being retried within minutes.

Add a pure helper selectLockoutCooldownMs(parsedCooldownMs, settings) that
returns the parsed reset as exactCooldownMs when it exceeds the base cooldown,
otherwise preserving the previous behavior (0 → exponential backoff, or base
when backoff is disabled). Both combo lockout call sites now use it. Also fix
the Antigravity executor's parseRetryFromErrorMessage regex (reset -> resets?)
so the plural "Resets in …" phrasing matches.

Reported-by: Ansh7473 (decolua/9router#1308)
@diegosouzapw
diegosouzapw force-pushed the fix/port-issue-1308-model-lockout-honors-reset branch from 9bcd0b5 to c6d964c Compare June 20, 2026 23:33
@diegosouzapw
diegosouzapw merged commit b0e9cd4 into release/v3.8.32 Jun 20, 2026
3 checks passed
@diegosouzapw
diegosouzapw deleted the fix/port-issue-1308-model-lockout-honors-reset branch June 21, 2026 12:33
tkgo11 pushed a commit to tkgo11/OmniRoute that referenced this pull request Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant