Repository navigation
feat(sse): per-provider sliding-window rate-limit fallback (FCC port, Fase 8.2) - #4146
Merged
Merged
Conversation
… Fase 8.2) Providers that do not send rate-limit headers never let the adaptive Bottleneck path learn a reservoir, so resolveRpm() leaves them effectively un-throttled. For such a provider with a documented fixed cap, an operator can declare a default and a sliding window enforces it proactively — burst-free, unlike Bottleneck's fixed-window reservoir which refills in one burst every interval. - open-sse/services/slidingWindowLimiter.ts: pure deque limiter (tryAcquire/ reset, injectable clock, MAX_KEYS guard). Allowed/blocked oracle; no queueing. - open-sse/services/providerDefaultRateLimit.ts: PROVIDER_DEFAULT_RATE_LIMITS (EMPTY by default → zero behavior change) + acquireProviderDefaultSlot() + abort-aware awaitProviderDefaultSlot() (bounded wait, converges, lets through after the budget so a misconfig can't hang a request). - Wired as a pre-schedule gate in withRateLimit() — Bottleneck still applies on top; this only adds a floor for header-less providers. Verification: Bottleneck already provides proactive fixed-window limiting from the global RPM / learned headers, so this is a COMPLEMENT (opt-in, header-less providers only), not a replacement. Empty map ships inert. Tests: 7 limiter unit (N+1 blocks, window slide, retryAfter, key isolation, no double-count) + 3 gate unit (cap enforcement, no-op when unconfigured, per-connection isolation). 52 existing rate-limit tests stay green (no-op path). rateLimitManager.ts +5 rebaselined (logic extracted, not inlined).
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
# Conflicts: # config/quality/file-size-baseline.json
Merged
tkgo11
pushed a commit
to tkgo11/OmniRoute
that referenced
this pull request
Sep 23, 2026
… Fase 8.2) (cheaper-inference#4146) Per-provider sliding-window rate-limit fallback (FCC port, Fase 8.2): opt-in (empty map = no-op), burst-free sliding window as a floor for header-less providers; Bottleneck still applies. Integrado em release/v3.8.29 (conflito de file-size-baseline resolvido por união com cheaper-inference#4145).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fase 8.2 — Sliding-window rate limit proativo (porte free-claude-code)
Verificação (trust-but-verify) — primeiro
Antes de implementar, verifiquei o
rateLimitManager: ele já usa Bottleneck comreservoir(fixed-window) + refresh 60s + queueing, alimentado pelo setting globalrequestsPerMinute/ overrides por-conexão / headers aprendidos. Ou seja, throttle proativo já existe para quem manda headers ou tem RPM configurado.O gap real: providers que não enviam headers de rate-limit nunca deixam o Bottleneck aprender um reservoir →
resolveRpm()retornaEFFECTIVELY_INFINITE→ ficam sem throttle proativo. Esta PR fecha exatamente isso, de forma complementar (não substitui o Bottleneck).O que faz
slidingWindowLimiter.ts(novo): limiter de deque puro (tryAcquire/reset, clock injetável p/ testes, guardMAX_KEYS). É um oráculo allowed/blocked, sem queueing interno (não duplica o Bottleneck) — o caller decide esperarretryAfterMs. Sliding window é burst-free, diferente do reservoir fixed-window do Bottleneck (que recarrega em rajada na virada da janela e pode disparar 429 upstream).providerDefaultRateLimit.ts(novo):PROVIDER_DEFAULT_RATE_LIMITS(vazio por padrão → zero mudança de comportamento) +acquireProviderDefaultSlot()+awaitProviderDefaultSlot()abort-aware (wait limitado, converge — cada espera libera ≥1 slot —, e libera após o budget para um misconfig nunca pendurar a request).withRateLimit(). O Bottleneck continua aplicando por cima; isto só adiciona um piso para providers header-less. Opt-in: um operador adiciona{ provider: {requests, windowMs} }ao mapa.Arquitetura / escopo
<cap);rateLimitManager.tssó +5 (import + 1 call + comentário) — rebaseline justificado, não inlined.anynovo (any-budget:t11 verde), sem ciclos.Testes (TDD, RED→GREEN)
retryAfterMs, isolamento de keys, tentativa bloqueada não consome slot.Validação
typecheck:coreverde,eslint0,check:cycles/check:file-size/check:any-budget:t11verdes, 10 novos + 52 regressão verdes.🤖 Generated with Claude Code