Skip to content

fix(proxy): allow concurrent proxy dispatcher streams - #4288

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.30from
wilsonicdev:codex/fix-proxy-dispatcher-concurrency
Jun 19, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.30from
wilsonicdev:codex/fix-proxy-dispatcher-concurrency

Conversation

@wilsonicdev

Copy link
Copy Markdown
Contributor

Summary

Fixes #4163 by allowing cached HTTP/SOCKS proxy dispatchers to open multiple concurrent upstream sockets. With pipelining: 0, long-lived SSE streams such as Codex /v1/responses can otherwise serialize through a single proxy connection when several requests share the same account-level proxy.

Changes:

  • adds OMNIROUTE_PROXY_DISPATCHER_CONNECTIONS with default 32 and cap 256
  • preserves the existing proxy hardening (keepAliveTimeout: 1, keepAliveMaxTimeout: 1, pipelining: 0)
  • documents the env var in .env.example and docs/reference/ENVIRONMENT.md
  • adds unit coverage for default, diagnostic single-connection override, and cap behavior

Validation

  • node --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test tests/unit/proxy-dispatcher-family.test.ts
  • npm run test:fixes
  • node --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test tests/unit/proxy-fetch.test.ts tests/unit/proxy-egress-isolation-bdd.test.ts tests/unit/proxyfetch-vercel-relay-2743.test.ts tests/unit/provider-limits-apikey-proxy-context.test.ts
  • npm run typecheck:core
  • npm run check:env-doc-sync

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the OMNIROUTE_PROXY_DISPATCHER_CONNECTIONS environment variable to configure the maximum concurrent sockets per cached HTTP/SOCKS proxy dispatcher, defaulting to 32 and capped at 256. It updates open-sse/utils/proxyDispatcher.ts to parse, validate, and apply this limit, and adds corresponding unit tests in tests/unit/proxy-dispatcher-family.test.ts along with documentation in .env.example and docs/reference/ENVIRONMENT.md. There are no review comments, and I have no additional feedback to provide as the implementation is clean and well-tested.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@diegosouzapw
diegosouzapw changed the base branch from main to release/v3.8.30 June 19, 2026 20:39
@diegosouzapw
diegosouzapw merged commit 133432b into diegosouzapw:release/v3.8.30 Jun 19, 2026
3 checks passed
@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks @wilsonicdev! 🙏 Merged into release/v3.8.30. Configurable per-proxy connection pooling (OMNIROUTE_PROXY_DISPATCHER_CONNECTIONS, default 32, capped at 256) is exactly what long-lived SSE streams like Codex /v1/responses need to avoid bottlenecking through a single cached proxy dispatcher — and it composes well with the dispatcher diagnostics we just shipped (#4281). Validated locally (11/11 dispatcher tests + typecheck + file-size gate green). Will ship in the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex /v1/responses throughput collapses under concurrent SSE requests through account HTTP proxy: OmniRoute keeps one upstream proxy socket

2 participants