Skip to content

fix(sse): re-select pool member per attempt after per-address 429 - #14659

Merged
diegosouzapw merged 5 commits into
cheaper-inference:release/v3.8.51from
maxmad64bis:fix/noauth-pool-reselect
Sep 25, 2026
Merged

diegosouzapw merged 5 commits into
cheaper-inference:release/v3.8.51from
maxmad64bis:fix/noauth-pool-reselect

Conversation

@maxmad64bis

@maxmad64bis maxmad64bis commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Base-red inherited: #14547

Summary

When several keyless accounts share one connection pool, every retry leaves through the same pool member, so each attempt after a 429 hits the same refused address on providers whose quota is bucketed by egress address. After this change a per-address 429 asks the connection pool for another member for the next attempt. Orders, never excludes: an exhausted pool keeps the current behavior, and a resolver answer for the member the request already egressed through is ignored silently. Opt-in via OPENCODE_POOL_RESELECT (default off); with the flag off the rotation is exactly the plain rotation.

Related Issues

Related to #14620 (merged: the shared capture context this PR reads the resolver through is now on the base)
Related to #13578 (merged refusal-memory mechanism this follows up per attempt)
Related to #13657 (stop/park 429 semantics this preserves)

Validation

Tests Added Or Updated

  • tests/unit/opencode-429-pool-reselect.test.ts (6 cases: re-selects after per-address 429, keeps the member for non-bucketed providers, null resolver keeps behavior, flag-off inert, same-member answer ignored without logging, mixed fleets keep dedicated proxies; 6/6 GREEN on the rebuilt head)
  • Replayed neighbors on the rebuilt head: feature-flags-settings + server-owned-tool-loop-flag 58/58

Coverage Notes

Unit tests cover the change (open-sse/executors/opencode.ts 429 arm): re-selection on the retry path, provider allowlist gate, null-exhausted fallback, flag-off path, same-member ignore, and the mixed-fleet guard. No coverage movement beyond the new test.

Reviewer Notes

  • Post-fix(sse): share applied proxy capture across duplicated fetch modules #14620 rebuild: the branch is now one commit (3705c946) on the release tip (7d23bcf8); the parent symbol (APPLIED_PROXY_CONTEXT_KEY / getAppliedProxyContext in open-sse/utils/proxyFetch.ts, merged via fix(sse): share applied proxy capture across duplicated fetch modules #14620) is on the base, and this PR consumes it through the new currentAppliedProxySink().reselectPoolMember reader. The stacked-on banner is dropped.
  • File-size rebaselined with operator approval (_rebaseline_2026_09_24_14659_pool_reselect_per_attempt_growth): own growth opencode.ts 1301->1355 (+54), proxyFetch.ts 1294->1311 (+17), chat.ts 2563->2564 (+1), chatHelpers.ts 1258->1303 (+45); additive, flag-off inert, covered by the new test. The chatcore-compression-integration test-size flag cites a file outside this PR.
  • Reds qualified as inherited (§12): check-api-typecheck cites only auggie.ts / projectCombo.ts, both outside this PR (diff on them is empty); check-env-doc-sync cites DEEP_HEALTH_CHECK_ENABLED, also outside this PR and present on the tip; check-mutation-test-coverage --strict and the full lint run stay red on the base (🔴 Release branch not green: release/v3.8.51 #14547).

Maintainer rework (merge-batch 2026-09-24)

  • fix(sse): share applied proxy capture across duplicated fetch modules #14620 (the stacked dependency) is merged and this branch was already rebased on it; merged the current release/v3.8.51 tip into the branch (real merge, contributor commit untouched).
  • Conflicts resolved keeping both sides: OPENCODE_POOL_RESELECT sits next to the new STREAM_READINESS_STALL_RETRY flag (definitions, wrapper, ENVIRONMENT.md), the flag-count assertions go to 79, and the per-attempt dispatch now goes through the tip's headersWaitDispatch while still running under account.proxy ?? reselectedProxy.
  • file-size-baseline.json: own growth re-measured after the merge (opencode.ts 1338→1392, proxyFetch.ts 1334→1351, chat.ts 2567→2568) under a dated annotation. eslint-suppressions.json: pruned two suppressions that no longer occur (tightening only; the pre-commit hook refused the merge otherwise).
  • Red→green: tests/unit/opencode-429-pool-reselect.test.ts fails 1/6 with the base opencode.ts (the re-selection case; the other five are guard cases that must keep today's behavior) and passes 6/6 on this head.
  • Focused suites (6 files, 98 tests) green except the known inherited flake opencode-proxy-refusal-memory.test.ts › "a connection without configured accounts never touches the memory", which also fails intermittently on the base tip. typecheck:core 0 errors, check:open-sse-typecheck 0 errors, eslint clean on touched files, check:env-doc-sync and check-file-size OK.

@maxmad64bis
maxmad64bis force-pushed the fix/noauth-pool-reselect branch 2 times, most recently from 82bb39e to 1a1fecb Compare September 23, 2026 15:30
@maxmad64bis
maxmad64bis marked this pull request as ready for review September 23, 2026 15:34
@maxmad64bis
maxmad64bis force-pushed the fix/noauth-pool-reselect branch from 1a1fecb to 4efe559 Compare September 23, 2026 15:39
@maxmad64bis
maxmad64bis marked this pull request as draft September 23, 2026 22:43
@maxmad64bis
maxmad64bis force-pushed the fix/noauth-pool-reselect branch from 4efe559 to 3d0baea Compare September 23, 2026 23:02
@maxmad64bis
maxmad64bis marked this pull request as ready for review September 23, 2026 23:31
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Good follow-up to #14655/#14657's per-address quota theme, from the account-rotation side this
time — re-selecting a live pool member (never forcing) after a per-address 429 when the account
has no proxy of its own, gated behind OPENCODE_POOL_RESELECT with the stop/park-ending exits
placed correctly before the reselection point. The 270-line test file with 6 RED-then-GREEN cases
looks solid. This is stacked on #14620 as noted — merging that one first, then this.

When a proxy-less account dispatches under an ambient pool context and
the provider buckets quota by egress address, a 429 asks the connection
pool for another member for the next attempt instead of retrying the
refused address. Orders, never excludes: an exhausted pool keeps the
current behavior. Opt-in via OPENCODE_POOL_RESELECT (default off); with
the flag off the rotation is exactly the plain rotation.
# Conflicts:
#	config/quality/file-size-baseline.json
#	docs/reference/ENVIRONMENT.md
#	open-sse/executors/opencode.ts
#	src/shared/constants/featureFlagDefinitions.ts
#	src/shared/utils/featureFlags.ts
#	tests/unit/feature-flags-settings.test.ts
#	tests/unit/server-owned-tool-loop-flag.test.ts
# Conflicts:
#	config/quality/file-size-baseline.json
#	open-sse/executors/opencode.ts
… totals

The catalog summary and the Network heading still said 78/20 after the
release merge; the registry now has 79 flags (21 in Network), which
tests/unit/feature-flags-doc-sync-static.test.ts asserts.
# Conflicts:
#	config/quality/file-size-baseline.json
#	open-sse/executors/opencode.ts
@diegosouzapw
diegosouzapw merged commit 3d8a9a2 into cheaper-inference:release/v3.8.51 Sep 25, 2026
3 of 7 checks passed
diegosouzapw pushed a commit that referenced this pull request Sep 25, 2026
)

Maintainer rework: reconciled with release/v3.8.51 after #14750/#14795/#14810/#14659 — migration 192 now follows 190/191 with no gap, the call_logs INSERT carries has_content/usage_provenance + added_wait_ms/added_wait_cause + the optional resilience_actions column and the #14474 id-collision retry, attempt logging keeps the fresh-UUID row key and reads the added wait late, opencode keeps both the served-account tracker and the park/throttle added-wait counters, the unused getAddedWaitPercentiles was dropped, and file-size-baseline.json was rebuilt from the tip with only this PR's own ceilings (opencode.ts, proxyFetch.ts, core.ts, RequestLoggerDetail.tsx) instead of rewinding unrelated entries. Tests: 133/133 focused (opencode-added-wait, applied-egress-key, egress-throttle, attempt-logging, call-log persistence/id-collision/provenance, resilience-actions); typecheck:core and check:open-sse-typecheck clean; check-file-size and check-migration-numbering OK. opencode-429-park-resume / opencode-429-pool-reselect fail identically on the pure release tip (inherited, not from this PR). Thank you @maxmad64bis!
diegosouzapw added a commit that referenced this pull request Sep 25, 2026
… merge (#14865)

Hotfix for a regression introduced by the #14659 rework merge: restores the dropped reselectedProxy declaration in OpencodeExecutor (tip failed tsc TS2304 and threw ReferenceError on proxy-less opencode dispatch). opencode-429-pool-reselect 0/6 on the tip -> 6/6; opencode-proxy-refusal-memory 12/12; open-sse typecheck and file-size green.
diegosouzapw added a commit to maxmad64bis/OmniRouteFork that referenced this pull request Sep 25, 2026
Reconcile with cheaper-inference#14688 (PROXY_SKIP_RECENTLY_FAILED now default on: the
flag-off functional case pins =false instead of deleting the variable) and
cheaper-inference#14659 (OPENCODE_POOL_RESELECT: registry and catalog totals move to 80
flags / Network 22).
diegosouzapw pushed a commit that referenced this pull request Sep 25, 2026
…#14657)

Adds PROXY_POOL_SHARED_EGRESS_ORDER (default off): for egress-bucketed providers, a pool member sharing a recently refused member's observed egress address ranks just below healthy members (order only, never excluded). Maintainer rework: reconciled with release/v3.8.51 (kept #14755's maybeEmitPoolExhausted and the #14688 default-on refusal memory; flag-off functional case pins PROXY_SKIP_RECENTLY_FAILED=false; registry/catalog totals 80 flags / Network 22 after #14659). Focused proxy pool/rotation/refusal/flag suites 305/305; typecheck:core and open-sse typecheck clean; file-size green; env-doc-sync red only on the inherited tip TAVILY_BASE_URL (#14442). Thank you @maxmad64bis!
fouadSalkini added a commit to fouadSalkini/OmniRoute that referenced this pull request Sep 26, 2026
… suppression

Merging the restored cheaper-inference#14771 suppressions file re-added the entry for
tests/unit/gemini-business-provider.test.ts. That file no longer exists and
the base already dropped the entry in cheaper-inference#14659. Re-apply the base's removal
so this branch differs from release/v3.8.51 only by the apiKeySelfService.ts
entry the stack owns.
@maxmad64bis
maxmad64bis deleted the fix/noauth-pool-reselect branch September 30, 2026 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants