test(e2e): include the first websocket frame in the passthrough realtime assertion

[yuneng-berri]

TLDR

Problem this solves:

• The realtime passthrough e2e assertion only printed the frame type, error
• Triaging the failure took three CI runs to learn which refusal fired

How it solves it:

• The websocket handshake helper keeps the first frame's text
• The assertion message prints that text next to the type

User Flow

Before: an engineer triaging a red litellm-e2e run cannot tell from the log why the realtime upgrade was refused

1. They open the failed e2e - full suite job for tests/e2e/llm_translation/test_passthrough_e2e.py::TestOpenAIPassthroughWebsocket::test_realtime_upgrade_reaches_openai_through_the_passthrough_prefix
2. The assertion reads the first frame was error and assert 'error' == 'session.created', nothing else
3. They have to rerun the suite with extra instrumentation to see whether the gateway refused the upgrade or OpenAI did

After: the same log line names the refusal

1. They open the same failed job
2. The assertion reads the first frame was error: {"type": "error", "error": {"type": "invalid_request_error", "message": "OpenAI websocket passthrough is disabled on this gateway. A proxy admin can turn it on by setting general_settings.enable_openai_websocket_passthrough to true."}}
3. They fix the stack config without another run

Relevant issues

The failure this surfaced is the one #39841 introduced for the e2e stack, fixed on the stack side in project-releaser

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Both runs are the litellm-e2e-pr pipeline with E2E_PATHS set to the passthrough file, against an ephemeral stack whose values do not opt into the websocket passthrough, so the test fails on both sides and the only difference is the assertion text

Before (7672399)

1. https://buildkite.com/berriai-1/litellm-e2e-pr/builds/251, job e2e - selected tests
2. Observed output:

FAILED llm_translation/test_passthrough_e2e.py::TestOpenAIPassthroughWebsocket::test_realtime_upgrade_reaches_openai_through_the_passthrough_prefix - AssertionError: the accepted socket never carried OpenAI's opening session event, so the upgrade was not relayed upstream; the first frame was error
assert 'error' == 'session.created'

After (0b0ed97)

1. https://buildkite.com/berriai-1/litellm-e2e-pr/builds/261, job e2e - selected tests
2. Observed output:

FAILED llm_translation/test_passthrough_e2e.py::TestOpenAIPassthroughWebsocket::test_realtime_upgrade_reaches_openai_through_the_passthrough_prefix - AssertionError: the accepted socket never carried OpenAI's opening session event, so the upgrade was not relayed upstream; the first frame was error: {"type": "error", "error": {"type": "invalid_request_error", "message": "OpenAI websocket passthrough is disabled on this gateway. A proxy admin can turn it on by setting general_settings.enable_openai_websocket_passthrough to true."}}

3. The same test at the same commit passes once the stack opts in: https://buildkite.com/berriai-1/litellm-e2e-pr/builds/263

Type

✅ Test

Caveats (if any)

Low

• The frame text is capped at 600 characters so a large session frame cannot flood the log

QA runbook

• tests/e2e/llm_translation/test_passthrough_e2e.py::TestOpenAIPassthroughWebsocket::test_realtime_upgrade_reaches_openai_through_the_passthrough_prefix - the assertion now shows the refusing frame, so a reviewer can see which side refused without a rerun
  • Start a proxy with an openai/gpt-realtime-2 deployment and general_settings.enable_openai_websocket_passthrough unset (needs OPENAI_API_KEY)
  • Connect a websocket client to ws://localhost:4000/openai_passthrough/v1/realtime?model=gpt-realtime-2 with Authorization: Bearer <virtual key> and read the first frame
  • Expect a frame of type error whose message says the passthrough is disabled on this gateway
  • Run the test against that proxy and expect its assertion message to quote that same frame
  • Set general_settings.enable_openai_websocket_passthrough: true, restart, reconnect, and expect the first frame to be session.created
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR improves the realtime passthrough E2E failure message by retaining a bounded copy of the first websocket frame

• Adds an optional first_frame field to the test handshake result
• Captures at most 600 characters from a successfully parsed first frame
• Includes the captured text when the expected session.created event is absent

Confidence Score: 5/5

The PR appears safe to merge and improves E2E failure diagnostics without changing the tested behavior

The added field is compatible with every caller, the assertion still requires session.created, and no changed-code failure remains

Important Files Changed

Filename	Overview
tests/e2e/llm_translation/passthrough_client.py	Adds a compatible optional handshake field and captures a bounded copy of the parsed first websocket frame
tests/e2e/llm_translation/test_passthrough_e2e.py	Extends the existing assertion diagnostic without weakening its expected event check

_{Reviews (1): Last reviewed commit: "test(e2e): include the first websocket f..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yuneng-berri]

Not needed: project-releaser #225 alone makes the test pass (build 263). Diagnostics-only change, dropping it.