test(e2e): pin the openai websocket passthrough prefixes

[mateo-berri]

TLDR

Problem this solves:

• The websocket passthrough fix shipped with no e2e coverage
• Both OpenAI prefixes could go back to refusing every upgrade
• Realtime and responses.connect clients would break unnoticed

How it solves it:

• Two live tests driving a real websocket client, not HTTP
• The realtime one reads OpenAI's own session.created frame back
• Proven red with the routes unregistered, green with them

User Flow

Before: a realtime or client.responses.connect() app cannot open a socket through the gateway at all, so that traffic has to route around it

1. The user mints a key for their voice app: POST https://litellm-domain/key/generate with {"models": []}, and gets back an sk-... key
2. They check the prefix over HTTP first: POST https://litellm-domain/openai_passthrough/v1/responses with Authorization: Bearer sk-... and {"model": "gpt-5.5", "input": "hi"}, and get a 200 carrying a resp_... id
3. They point the app's realtime session at the same prefix: websocket upgrade GET wss://litellm-domain/openai_passthrough/v1/realtime?model=gpt-realtime-2 with Authorization: Bearer sk-..., and get back HTTP/1.1 403 Forbidden. No socket opens and the response carries no message
4. They try the SDK's client.responses.connect() against the other prefix: websocket upgrade GET wss://litellm-domain/openai/v1/responses, and get the same bare HTTP/1.1 403 Forbidden
5. They retry with the master key, then with a key scoped to every model, and every upgrade comes back 403 in exactly the same way, with nothing in the response to say what was wrong
6. They give up on the gateway for this app and connect straight to wss://api.openai.com/v1/realtime with their own OpenAI key. Those sessions never appear in GET https://litellm-domain/spend/logs, and the key's budget and rate limits never apply to them

Any other user hits the identical 403 no matter what their key is allowed to reach, because the refusal never looks at the key

After: the same app opens the socket and talks to the model through the gateway

1. The user mints a key for their voice app: POST https://litellm-domain/key/generate with {"models": []}, and gets back an sk-... key
2. They check the prefix over HTTP first: POST https://litellm-domain/openai_passthrough/v1/responses with Authorization: Bearer sk-... and {"model": "gpt-5.5", "input": "hi"}, and get a 200 carrying a resp_... id
3. They point the app's realtime session at the same prefix: websocket upgrade GET wss://litellm-domain/openai_passthrough/v1/realtime?model=gpt-realtime-2 with Authorization: Bearer sk-..., and get HTTP/1.1 101 Switching Protocols. The first frame they read is OpenAI's own {"type": "session.created", ...}
4. They send and receive audio and text events over that socket for the rest of the session, against the same key their HTTP traffic uses
5. client.responses.connect() against wss://litellm-domain/openai/v1/responses also returns HTTP/1.1 101 Switching Protocols, and the socket stays open waiting for the app's first event
6. Their realtime traffic runs through the gateway, so it is billed to the same key rather than to a separate OpenAI credential the gateway never sees

Another user whose key is scoped to a subset of models is still refused the upgrade with a 403, so they cannot reach a model outside their allow-list over the socket either

Relevant issues

Pins the fix for #36088

Linear ticket

Part of LIT-5902. This covers the PT-03 row; the PT-10 row stays open on that ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

The fix being pinned (#36151) is already on the merge base, so a Before captured there would be green and would prove nothing. Before is therefore the PR tip with that fix's two route registrations removed, which is exactly what the file looked like before #36151 landed. That is the only difference between the two sides

Shared setup: a proxy from this branch on http://localhost:43127 against a local Postgres, gpt-5.5 and gemini-2.5-flash registered from config, keys minted with POST /key/generate. The realtime traffic hits real OpenAI

Before (9b27d7a, with the websocket routes unregistered)

1. Deleted the two @router.websocket lines at litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py:2152-2153 and rebooted the proxy
2. uv run pytest llm_translation/test_passthrough_e2e.py -k TestOpenAIPassthroughWebsocket -> 2 failed, 12 deselected in 0.73s, with AssertionError: /openai/v1/responses refused the websocket upgrade with HTTP 403; the prefix relays this route over HTTP but drops a responses.connect client before the socket opens
3. A raw client sees the same thing, so the failure is the route and not the test: connecting to ws://localhost:43127/openai_passthrough/v1/realtime?model=gpt-realtime-2 with an all-models key raises InvalidStatus: server rejected WebSocket connection: HTTP 403

After (9b27d7a, the PR tip)

1. Restored the file, rebooted the proxy, and re-ran the same command -> 2 passed, 12 deselected in 2.73s
2. The same raw client probe at this commit, across both prefixes and both key shapes:

all-models   /openai_passthrough/v1/realtime?model=gpt-realtime-2: HANDSHAKE ACCEPTED
    first frame type='session.created' keys=['event_id', 'session', 'type']
restricted   /openai_passthrough/v1/realtime?model=gpt-realtime-2: HANDSHAKE REJECTED http=403
all-models   /openai/v1/responses: HANDSHAKE ACCEPTED
restricted   /openai/v1/responses: HANDSHAKE REJECTED http=403
all-models   /openai_passthrough/v1/responses: HANDSHAKE ACCEPTED
restricted   /openai_passthrough/v1/responses: HANDSHAKE REJECTED http=403
no-such-prefix /nonexistent_prefix/v1/realtime: HANDSHAKE REJECTED http=403

3. make check -> PASS, which runs basedpyright tests/e2e -> 0 errors, 0 warnings, 0 notes and the raw HTTP client ban
4. uv run pytest llm_translation/realtime/ --collect-only -q -> 9 tests collected, so moving ws_base_url did not break the suite it came from
5. uv run pytest coverage_registry/test_collector.py -q -> 13 passed, and uv run python -m coverage_registry.collector prints no "marker(s) point at ids not in the registry" line, so both new covers ids resolve to the rows added here

Type

✅ Test

Caveats (if any)

• The responses leg asserts only that the upgrade was accepted
• /openai/v1/realtime is not testable, the app-level route claims it
• gpt-realtime-2 is a literal in the test module
• The ticket's stated 1008/1011 signal was wrong, corrected here
• buildkite/e2e-tests is red on three pre-existing failures outside this diff

OpenAI speaks first on realtime, which is why that test can assert a session.created frame and prove the socket was really relayed upstream. A responses.connect socket waits for the client, so there is no opening frame to read and the accepted handshake is the whole signal. It still catches the regression exactly, since that route answered 403 before the fix, but it does not prove the relay the way the realtime one does

The second prefix is exercised through /openai/v1/responses rather than /openai/v1/realtime, which issue #36088 also lists. The app-level realtime endpoint in proxy_server.py is registered before the passthrough router is included, so it wins that path and the passthrough route never sees it. Testing /openai/v1/realtime would assert the app-level endpoint instead, which is a different behavior

REALTIME_MODEL sits in the test module as a literal, the same way EMBEDDING_MODEL already does. The passthrough route relays the model name to OpenAI verbatim, so it has to be OpenAI's own name rather than a gateway alias, but an environment that needs a different realtime model has no way to say so

LIT-5902 states the signal is a handshake accepted and then closed with 1008 or 1011. That is not what happens. The websocket auth dependency and the model-restriction check both close before accept(), and Starlette turns a pre-accept close into an HTTP 403 handshake rejection, which is indistinguishable from the unregistered route's 403. The After probe above shows it directly: the restricted key gets the same 403 as the nonexistent prefix. The usable signal is an all-models key getting an accepted handshake, which is what these tests assert

buildkite/e2e-tests is red here, and none of it comes from this change. Editing e2e_config.py makes the gate select the entire e2e suite instead of a subset, so this is one of the few builds that runs these three tests at all, which is why every neighbouring build is green. Both new tests passed in that same run. The failures are test_unflagged_model_converts_system_reminder_and_succeeds in TestAzureFoundryMidConversationSystem, TestVertexMidConversationSystem, and TestBedrockInvokeMidConversationSystem, each of which has now failed 16 of 16 runs over the past seven days for a reliability of 0. All three wait up to 60 seconds for a provider prompt cache to become fully readable, and neither file they live in appears in this diff. They are already tracked as LIT-5921. The check is not required on litellm_internal_staging and every required check is green

QA runbook

• tests/e2e/llm_translation/test_passthrough_e2e.py::TestOpenAIPassthroughWebsocket::test_realtime_upgrade_reaches_openai_through_the_passthrough_prefix - a realtime client opens a socket on the dedicated prefix and reads OpenAI's opening session event

  • Mint an all-models key: curl -X POST http://localhost:4000/key/generate -H "Authorization: Bearer sk-1234" -d '{"models": []}' (needs OPENAI_API_KEY on the proxy)
  • Open a websocket to ws://localhost:4000/openai_passthrough/v1/realtime?model=gpt-realtime-2 with Authorization: Bearer <key>
  • Expect 101 Switching Protocols and a first frame whose type is session.created
  • Repeat with a key restricted to one model and expect a 403 handshake rejection instead, which is the shape the unregistered route also returns
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

• tests/e2e/llm_translation/test_passthrough_e2e.py::TestOpenAIPassthroughWebsocket::test_responses_upgrade_is_accepted_on_the_openai_prefix - a responses.connect client is allowed to open a socket on the /openai prefix

  • Using the same all-models key, open a websocket to ws://localhost:4000/openai/v1/responses
  • Expect 101 Switching Protocols and the socket to stay open with no frame arriving, since that route waits for the client to send first
  • Delete the two @router.websocket lines in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py, reboot, and expect both upgrades to return 403
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 9b27d7a passes /live-pr-risk

[greptile-apps]

Greptile Summary

The PR adds end-to-end coverage for websocket upgrades through both OpenAI passthrough prefixes and centralizes websocket URL construction.

• Adds coverage-registry entries for realtime and Responses websocket passthrough.
• Adds a reusable passthrough websocket client and handshake result models.
• Adds live tests for the realtime opening event and Responses upgrade acceptance.
• Moves the shared HTTP-to-websocket URL conversion into the E2E configuration module.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
tests/e2e/llm_translation/passthrough_client.py	Adds websocket connection support that records rejected upgrades and optional opening event types.
tests/e2e/llm_translation/test_passthrough_e2e.py	Adds regression tests for realtime relay and Responses websocket upgrade acceptance on the OpenAI passthrough prefixes.
tests/e2e/e2e_config.py	Centralizes conversion of the configured proxy URL to its websocket equivalent.
tests/e2e/llm_translation/realtime/realtime_client.py	Reuses the centralized websocket base URL helper without changing URL semantics.
tests/e2e/coverage_registry/llm_conversational.yaml	Registers coverage for the OpenAI Responses websocket passthrough path.
tests/e2e/coverage_registry/llm_nonconversational.yaml	Registers coverage for the OpenAI realtime websocket passthrough path.

_{Reviews (2): Last reviewed commit: "test(e2e): drop the REALTIME_MODEL comme..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ Mateo

---

Mateo seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9b27d7a. Configure here.}