Skip to content

fix(proxies): make the 429 set-aside curve tunable via environment - #14655

Merged
diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.51from
maxmad64bis:fix/refusal-cooldown-tunable
Sep 24, 2026
Merged

diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.51from
maxmad64bis:fix/refusal-cooldown-tunable

Conversation

@maxmad64bis

@maxmad64bis maxmad64bis commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: #14547

Summary

Requests that hit a provider 429 keep serving the same egress for 2 minutes before backing off, even when the refusal persists 5-15 minutes. The set-aside curve (ip_quota_429: 120 s base, doubling, 3 600 s cap) is hardcoded, so operators cannot tune it to their provider. After this change the base and cap are readable from the environment with the current values as defaults, so unset keeps the current doubling curve unchanged.

Related Issues

No linked issue — hardening of the merged refusal-memory mechanism (#13578) with no new default.

Validation

  • Change type: routing
  • Focused tests and category gates from the golden path
  • npm run lint — ESLint on the touched files is clean; the full run is red on the base (🔴 Release branch not green: release/v3.8.51 #14547)
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

  • tests/unit/proxy-refusal-memory-quota-tuning.test.ts (new, 7 cases: default curve, bounded values, out-of-range/unreadable fallback with warning, max-below-base fallback, cap never zero).
  • Neighbors kept green: tests/unit/proxy-refusal-memory.test.ts (14 tests).

Coverage Notes

Unit tests cover the change (open-sse/utils/proxyRefusalMemory.ts): default periods without variables, bounded doubling with variables set, fallback to defaults with a single startup warning on invalid input, and the max-below-base fallback. No coverage movement in touched files beyond the new test.

Reviewer Notes

  • Out-of-range or unreadable values fall back to the defaults with one warning at startup, never a zero or infinite cap; when the cap resolves below the base it falls back to the default cap with a warning.
  • PROXY_QUOTA_429_BASE_MS (default 120000) and PROXY_QUOTA_429_MAX_MS (default 3600000) are read once at startup and bounded to 1000..3600000; lowering the cap also shortens streak forgetting since entries purge at twice the cap after expiry.
  • API Route Typecheck failure is inherited, not caused by this PR: the 4 errors name auggie.ts and projectCombo.ts, none touched here (git diff empty on those files), and trace back to the base-red branch state (🔴 Release branch not green: release/v3.8.51 #14547).

@maxmad64bis
maxmad64bis force-pushed the fix/refusal-cooldown-tunable branch from 0c03cd6 to 747cd44 Compare September 23, 2026 15:30
@maxmad64bis
maxmad64bis marked this pull request as ready for review September 23, 2026 15:34
@maxmad64bis
maxmad64bis force-pushed the fix/refusal-cooldown-tunable branch from 747cd44 to 7697144 Compare September 23, 2026 22:28
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Nicely done — making the 429 set-aside curve tunable via env instead of hardcoded, with proper
bounds checking and a warning+fallback on invalid input. The spawnSync-per-scenario test pattern
for a module-load-time env read is exactly right (can't otherwise re-test process.env mutations
in the same process). .env.example and ENVIRONMENT.md are both updated. Ready to merge.

@diegosouzapw
diegosouzapw merged commit 2dcfe53 into cheaper-inference:release/v3.8.51 Sep 24, 2026
9 of 16 checks passed
@maxmad64bis
maxmad64bis deleted the fix/refusal-cooldown-tunable branch September 24, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants