Skip to content

fix(db): rotate proxy pools on the chat path like the registry does - #14044

Merged
diegosouzapw merged 1 commit into
release/v3.8.51from
fix/13575-proxy-pool-chat-path
Sep 18, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.51from
fix/13575-proxy-pool-chat-path

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Closes #13575

Root cause

resolveProxyForConnection (src/lib/db/settings.ts) caches the first pool-resolution result for a connection and, outside the narrow #13578 "member was just set aside" escape hatch, returns that same cached member on every subsequent chat request — regardless of the pool's rotation strategy (round-robin/sticky/random). resolveProxyForScopeFromRegistry, which every existing rotation test (#6365) calls directly, re-runs the strategy on every call and rotates correctly. The chat path is the only consumer whose freeze bypasses that.

Fix

Extend the cache-validity check with a second term alongside the existing isCachedPoolMemberSetAside escape hatch: a cached result is now also stale (falls through to the same cascade the direct registry callers use) whenever it came from a live scope pool (source: "registry") and the connection does not need a stable egress. Two populations are excluded and keep their pinned member exactly as before:

  • EGRESS_BUCKETED_LOCK_PROVIDERS (open-sse/config/providerErrorRules.ts) — opencode's free-tier quota is bucketed by egress IP; rotating would fragment one connection's quota across several IPs.
  • grok-web — its cf_clearance cookie is pinned to the IP/User-Agent/TLS fingerprint that earned it (src/shared/providers/webSessionCredentials.ts); rotating the egress turns every subsequent request into a Cloudflare 403.

No new rotation strategy is introduced — the pool's existing round-robin/sticky/random logic in src/lib/db/proxies/rotation.ts is unchanged; the chat path now simply reaches it the same way the registry-direct callers already do.

Regression test

tests/unit/proxy-pool-chat-path-rotation-13575.test.ts (new):

  • RED before the fix: resolveProxyForConnection returned the same host 6/6 times for a 3-member account-scope pool.
  • GREEN after: rotates across all 3 members; an opencode connection and a grok-web connection each stay pinned to a single member across 6 calls.
node --import tsx/esm --test tests/unit/proxy-pool-chat-path-rotation-13575.test.ts

tests/unit/proxy-pool-skips-refused-member.test.ts (updated): three of its assertions encoded the old frozen-cache contract as correct behavior (resolveProxyForConnection returning the exact same cached object on a second call with no set-aside event). Updated those three to the corrected always-rotates-except-pinned contract; every other case in that file, and every case in tests/unit/proxy-pool-rotation-6365.test.ts, passes unchanged.

Gates run

  • node --import tsx/esm --test tests/unit/proxy-pool-chat-path-rotation-13575.test.ts tests/unit/proxy-pool-skips-refused-member.test.ts tests/unit/proxy-pool-rotation-6365.test.ts — 23/23 pass
  • Full existing proxy-resolution suite (8385-perkey-proxy-global-toggle, combo-scope-proxy-dead-7149, db-proxies-crud, db-proxies-split, db-settings-crud, db-settings-extended, db-settings-split, fixes-p1, issue-13470-token-refresh-proxy-bypass, proxy-assigned-unavailable-6246, proxy-noauth-provider-6272, proxy-registry, proxy-resolution-status-filter, proxySubscription.service, repro-8995, resolve-proxy-family) — 196/196 pass
  • node scripts/check/check-file-size.mjs — no ✗ on touched files
  • node scripts/check/check-complexity.mjs — OK (2857 violations vs baseline 3218)
  • node scripts/check/check-cognitive-complexity.mjs — OK (1291 violations vs baseline 1437)
  • npm run typecheck:core — exit 0
  • npm run check:open-sse-typecheck — 0 errors (settings.ts imports from open-sse/config/providerErrorRules.ts)
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json on all changed files — 0 errors
  • node scripts/check/check-changelog-integrity.mjs — OK
  • changelog.d/fixes/13575-proxy-pool-chat-path-rotation.md added

⚠️ base-red inherited: #14004 — docs env/docs contract (fixed separately in #14022), chatHelpers file-size drift

…13575)

resolveProxyForConnection cached a scope pool's first resolution result for the
life of the per-connection cache, so a chat-path request never saw the pool's
round-robin/sticky/random strategy advance again — only the narrow #13578
set-aside escape hatch could break the freeze. resolveProxyForScopeFromRegistry
(used directly by every existing rotation test) always re-ran the strategy and
rotated correctly.

The cache now treats a registry-sourced pool result as due for re-resolution on
every call (falling through to the same cascade the direct registry callers
use), except for the two populations that need a stable egress across
requests: EGRESS_BUCKETED_LOCK_PROVIDERS (opencode's quota is bucketed by
egress IP) and grok-web (its cf_clearance cookie is pinned to the IP/UA/TLS
fingerprint that earned it).

Regression test: tests/unit/proxy-pool-chat-path-rotation-13575.test.ts, RED
before the fix (resolveProxyForConnection returned the same host 6/6 times for
a 3-member pool), GREEN after. Updated tests/unit/proxy-pool-skips-refused-member.test.ts's
three assertions that encoded the frozen-cache contract to the corrected
always-rotates-except-pinned contract; all other cases in that file and in
tests/unit/proxy-pool-rotation-6365.test.ts pass unchanged.
@diegosouzapw

Copy link
Copy Markdown
Owner Author

Validated in local merge-train /tmp/mt-train3b.log on 192.168.0.113 @ train tip 408e2128791696a18966681953136fc96aeb99b0 (32 PRs boarded): static gates green; full test:unit 40694 tests, 17 failing — every one reproduces on the pure release tip (base-red sweep list), zero new reds. Merged --admin per merge-gates §4/§7.

@diegosouzapw
diegosouzapw merged commit f3ab24b into release/v3.8.51 Sep 18, 2026
14 of 21 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…iegosouzapw#13575) (diegosouzapw#14044)

resolveProxyForConnection cached a scope pool's first resolution result for the
life of the per-connection cache, so a chat-path request never saw the pool's
round-robin/sticky/random strategy advance again — only the narrow diegosouzapw#13578
set-aside escape hatch could break the freeze. resolveProxyForScopeFromRegistry
(used directly by every existing rotation test) always re-ran the strategy and
rotated correctly.

The cache now treats a registry-sourced pool result as due for re-resolution on
every call (falling through to the same cascade the direct registry callers
use), except for the two populations that need a stable egress across
requests: EGRESS_BUCKETED_LOCK_PROVIDERS (opencode's quota is bucketed by
egress IP) and grok-web (its cf_clearance cookie is pinned to the IP/UA/TLS
fingerprint that earned it).

Regression test: tests/unit/proxy-pool-chat-path-rotation-13575.test.ts, RED
before the fix (resolveProxyForConnection returned the same host 6/6 times for
a 3-member pool), GREEN after. Updated tests/unit/proxy-pool-skips-refused-member.test.ts's
three assertions that encoded the frozen-cache contract to the corrected
always-rotates-except-pinned contract; all other cases in that file and in
tests/unit/proxy-pool-rotation-6365.test.ts pass unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(resilience): Proxy pools can't pick another member on the chat path: the per-connection proxy cache freezes the choice

1 participant