Skip to content

fix(usage): honor active rateLimitedUntil cooldown on provider-limits sync - #11313

Closed
pacocartones wants to merge 1 commit into
cheaper-inference:release/v3.8.50from
pacocartones:fix/quota-cooldown-cleared-on-sync-11277
Closed

pacocartones wants to merge 1 commit into
cheaper-inference:release/v3.8.50from
pacocartones:fix/quota-cooldown-cleared-on-sync-11277

Conversation

@pacocartones

Copy link
Copy Markdown
Contributor

Fixes #11277

Problem

maybeClearRecoveredQuotaState() gated every defensive guard behind lastErrorType === "quota_exhausted". Any other lastErrorType fell straight through to clearRecoveredProviderState and wiped a still-active rateLimitedUntil without ever consulting it.

lastErrorType is not normalized across writers. The provider test route persists diagnosis.type (upstream_rate_limited, network_error, ...), which never equals "quota_exhausted", while persisting a real rateLimitedUntil. Combined with hasUsableQuota() returning true on the first non-zero window — so a connection with a 5h window and a weekly window looks recovered while the weekly one is still at zero — a multi-day cooldown was cleared on every 5-10 minute sync, burning a real upstream call, taking the same 429, and cooling down again. That is the loop the reporter observed with a 146-hour cooldown.

Fix

Adds an exported hasActiveCooldown() and hoists the quota-window guards out of the lastErrorType branch so they apply whenever a cooldown is still in force, regardless of which writer set it.

hasActiveCooldown deliberately fails open on an unparseable timestamp: a corrupt value must never be able to pin a connection offline forever.

The change is additive — the quota_exhausted refinements keep their exact previous behaviour for the case where the cooldown has already expired.

Test

Added to the existing tests/unit/provider-limits-recovery.test.ts.

Verified failing on the base commit, 2 of 17:

✖ active cooldown survives a quota sync even when lastErrorType is not quota_exhausted (#11277)
✖ hasActiveCooldown fails open on unparseable and absent timestamps (#11277)
ℹ pass 15
ℹ fail 2

With the fix, 17/17 pass:

ℹ tests 17
ℹ pass 17
ℹ fail 0

Run with the repository's own unit runner (node --test via the test:unit script), not vitest — tests/unit/*.test.ts is covered by the former.

Branch is up to date with release/v3.8.50.

… sync

maybeClearRecoveredQuotaState() gated every defensive guard behind
lastErrorType === "quota_exhausted". Any other lastErrorType fell straight
through to clearRecoveredProviderState and wiped a still-active
rateLimitedUntil without ever consulting it.

lastErrorType is not normalized across writers: the provider test route
persists diagnosis.type (upstream_rate_limited, network_error, ...), which
never equals "quota_exhausted", while persisting a real rateLimitedUntil.
Combined with hasUsableQuota() returning true on the first non-zero window,
a multi-day cooldown was cleared on every 5-10 minute sync, burning a real
upstream call, taking the same 429 and cooling down again.

Adds exported hasActiveCooldown() and hoists the quota-window guards out of
the lastErrorType branch so they apply whenever a cooldown is still active.
hasActiveCooldown fails OPEN on an unparseable timestamp so a corrupt value
can never pin a connection offline forever. The change is additive: the
quota_exhausted refinements keep their exact previous behaviour.

Fixes cheaper-inference#11277
@pacocartones

Copy link
Copy Markdown
Contributor Author

Closing this — #11310 landed the same fix and #11277 is closed as completed.

Your version is the better shape: hoisting the guards out of the lastErrorType branch is the same conclusion, and gating on the error type rather than adding a separate cooldown predicate keeps the surface smaller.

No action needed here. Thanks for the quick turnaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(quota): active rateLimitedUntil cooldown is cleared by the provider-limits sync when lastErrorType is not "quota_exhausted"

1 participant