Repository navigation
feat(proxies): stop re-serving a pool member the provider just refused - #13595
Closed
maxmad64bis wants to merge 3 commits into
Closed
maxmad64bis wants to merge 3 commits into
maxmad64bis wants to merge 3 commits into
Conversation
added 2 commits
September 13, 2026 20:39
OmniRoute picks a proxy in two places, a registry pool (cheaper-inference#6365) and opencode's per-account rotation, and neither remembers a failure past the request that saw it. The TCP reachability probe already fails a request on a dead proxy, then the next pick can land on it again; a 429 on a proxied opencode account only cools that account for 5 to 60 seconds, so a proxy whose egress IP is out of quota is retried every minute and each request pays an extra round trip. A small per-process memory now sets such a proxy aside, keyed by its entry point (scheme, username, host, port; never the password): a refused probe for 1 minute, a 429 through it for 2 minutes, doubling on each repeat up to 10 and 60 minutes. A response through the proxy clears it. Pools skip set-aside members under every rotation strategy (the round-robin cursor moves past the member actually served, sticky writes nothing more), the per-connection cache stops re-serving such a member, and the opencode rotation skips set-aside proxied accounts. Accounts without a proxy are never concerned. No proxy status is written and nothing is persisted. When every candidate is set aside, selection is exactly what it was, so the cheaper-inference#6246 fail-closed contract holds. PROXY_SKIP_RECENTLY_FAILED=false (or 0, no, off) restores plain selection.
The proxy log status collapses every outcome into success/timeout/error, so a 429 (the provider limits this egress IP), a 403 (it refuses it) and a 500 (an outage) are the same line. result.status cannot fill the gap: several 429s are generated locally (rate limiter, quota sharing, backpressure) before any request leaves. The status is now captured where it is known for sure: the patched global fetch is wrapped once, and while a provider request is being dispatched (inside runWithCapture) the received status is written to the applied-proxy sink added in cheaper-inference#5217. A new dispatch invalidates the earlier status at entry, so a retry that never reaches the network (local refusal, start timeout) leaves no stale code behind; a throw during dispatch clears it, so a network error after a 429 on another proxy leaves no stale code either. Side fetches of the same request (usage sync, dashboard events) and background calls an executor leaves running after it returns (settled dispatch token) do not overwrite it. The status then travels with the applied proxy into safeLogEvents and lands in a nullable proxy_logs.upstream_status column: null means no response was received. The textual status column is untouched, so existing filters return the same rows. proxyFetch.ts grows by four lines at its single install site; its cap is raised with a note. chat.ts does not grow.
maxmad64bis
force-pushed
the
feat/pool-skips-ip-quota-429
branch
from
September 13, 2026 19:47
dd4f024 to
48e7c8a
Compare
This was referenced Sep 13, 2026
maxmad64bis
changed the title
Pools already skip a member that just failed, but only when the TCP probe refuses it: a proxy whose egress IP is out of quota still accepts connections, so the pool keeps serving it and every request pays a 429 before trying again. The status alone cannot tell, since several 429s are generated locally. The proxy log now records the status the provider actually returned. At the log call, that status is read from proxyInfo: a 429 from a provider whose quota is tied to the egress IP (the existing isEgressBucketedLockScope list) sets the proxy aside, and a 2xx from such a provider clears it; a 2xx from another provider only ends an unreachable period, since a global pool is shared. A missing status (local refusal, network error), an edge relay (the status is the relay's) and a direct request write nothing, and result.status is never read. The switch and durations are the existing ones.
maxmad64bis
force-pushed
the
feat/pool-skips-ip-quota-429
branch
from
September 13, 2026 19:55
48e7c8a to
95d1ed3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Proxy pools already skip a member after a failed attempt, but only when the TCP probe refuses the connection — a pool member that stays reachable yet keeps refusing requests is served again on the next pick, and every request pays the same failure before moving on. The proxy log now records the outcome the provider actually returned, and at the log call the pool reads it back from the log context: a refusal received through a member sets that member aside for the existing cooldown, and a later success through it clears it. No request is retried more than before and no traffic is added — the pool just stops re-serving a route it already knows is down.
Related Issues
Builds on #13578 and #13580. Inherits the #13580 / #13581 merge note (see Reviewer Notes). Related to #10629, #10920, #10654, #10880, #9611, #9736.
Validation
npm run lint— ESLint on the touched files is clean (exit 0); the full run was not playedTests Added Or Updated
tests/unit/proxy-outcome-memory.test.ts(new, 8 tests): helper unit tests — a refusal received through a pool member sets it aside, a success through it clears it, a success from a provider outside the refusal scope only ends an unreachable period, locally generated outcomes write nothing, switch off writes nothing.tests/unit/pool-refused-member-path.test.ts(new, 3 tests): path tests — a received refusal through a pool member makes the next pick skip it; a success recovers it; an out-of-scope provider's outcome doesn't clear a refusal set-aside.merge-applied-proxy-sink,apply-executor-proxy-info-5217,proxy-logs-upstream-status,proxy-pool-skips-refused-member— 36 pass, 0 fail across the 6 files.hard-session-lease-bypass-inventory) is inherited — it touches files this PR doesn't, and its diff vs the base is empty.Coverage Notes
RED then GREEN: helper suite failed with module-not-found before the helper existed, then passed 8/8 after; the first path test failed before the call was wired in, then all 36 passed across the 6 files. Gates green: file-size, api-typecheck (baseline-only delta, nothing in this PR's files), any-budget, tracked-artifacts, docs-sync, prettier, eslint on the touched files, complexity ratchets (0 new). Anti-leak greps empty. One commit, 5 files, +257/-0.
Reviewer Notes
The new helper is
noteProxyOutcome(provider, proxyInfo)insrc/sse/handlers/proxyOutcomeMemory.ts, called fromsafeLogEventsinsrc/sse/handlers/chatHelpers.ts:1056, just before the proxy log call, using that function's ownproxyInfoparameter —result.statusis never read, so locally generated outcomes can't accuse a member. This extends the skip mechanism from #13578 (one more trigger, same memory, same switch, same escalating cooldowns): a received 429 from a provider in the existing scope list (opencode, opencode-go, opencode-cli) sets the member aside; a 2xx from such a provider clears it; a 2xx from another provider only ends anunreachableperiod since a global pool is shared. Missing outcome, direct requests, edge relays, and relays without a usable key write nothing. Known nit (non-blocking, reviewer-flagged): the relay test's uppercaseVercelcase passes via the null-key guard rather than the relay guard (isRelayTypeis lowercase-only, consistent with the two other existing call sites) — happy to normalize or narrow the case on the next useful amend.config/quality/file-size-baseline.jsonconflict with feat(proxies): show how many egress IPs actually served a proxy pool #13581; the last commit itself adds no file-size change and touches no file that fix(proxies): keep the stored status when a write does not send one #13577, feat(proxies): show how many egress IPs actually served a proxy pool #13581 or fix(api): keep omitted fields on partial updates #13582 change.