fix(realtime): surface an upstream handshake refusal as an error event and policy close

[devin-ai-integration]

TLDR

Problem this solves:

• Upstream realtime handshake refusal (Azure 401) closes the client with 1006 and no event
• Every relay catches InvalidStatusCode, which websockets 15 no longer raises
• The Azure relay then swallows the failure and never closes the client socket

How it solves it:

• Catch websockets.exceptions.InvalidStatus in every realtime relay
• Send an error event naming the upstream HTTP status, then close with a mapped code
• 401/403 close 1008, 429 closes 1013, anything else 1011
• Azure generic failures now send a fixed Internal server error event and close 1011; the exception text stays in the proxy log
• Live e2e test provisions a bad-credential Azure deployment and asserts the new contract

User Flow

Before: a developer connecting to a realtime deployment whose Azure credential is rejected sees the socket vanish with no explanation

1. They run wscat -c "wss://litellm-domain/v1/realtime?model=azure-realtime" -H "Authorization: Bearer sk-..."
2. wscat prints Connected (press CTRL+C to quit)
3. Nothing arrives, then wscat prints Disconnected (code: 1006, reason: "")
4. The same key against POST https://litellm-domain/v1/chat/completions works, so they cannot tell whether the proxy, the key, or the provider is broken

After: the same connection tells them exactly what the upstream said and closes cleanly

1. They run wscat -c "wss://litellm-domain/v1/realtime?model=azure-realtime" -H "Authorization: Bearer sk-..."
2. wscat prints Connected (press CTRL+C to quit)
3. They receive {"type": "error", "error": {"type": "server_error", "message": "Upstream realtime handshake rejected with HTTP 401"}}
4. wscat prints Disconnected (code: 1008, reason: "Upstream realtime handshake rejected with HTTP 401")

Relevant issues

Reported by a customer (Pylon #8933)

Affected release

Linear ticket

Resolves LIT-7058

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live before and after with the reporter's client, wscat, against real Azure OpenAI (gpt-realtime, api_version: 2025-08-28, GA protocol) and real OpenAI gpt-realtime. Each leg is one proxy booted from the named commit with --num_workers 2 on a random free port, no database, model list from the config below. The checkouts share one venv through PYTHONPATH because the PR changes no dependency, and litellm.__file__ was checked to resolve inside each worktree before the boot. The author's video of the same before and after run, captured on cd9636c: https://berriaillm.slack.com/archives/C0C04J86WTC/p1790054146396189?thread_ts=1790039088.400839&cid=C0C04J86WTC

Config shared by every leg (AZURE_SWEDEN_API_BASE, AZURE_SWEDEN_API_KEY, OPENAI_API_KEY, and LITELLM_MASTER_KEY come from the environment):

model_list:
  - model_name: azure-realtime-refused
    litellm_params: {model: azure/gpt-realtime, api_base: os.environ/AZURE_SWEDEN_API_BASE, api_key: invalid-e2e-key, api_version: "2025-08-28", realtime_protocol: GA}
  - model_name: azure-realtime-valid
    litellm_params: {model: azure/gpt-realtime, api_base: os.environ/AZURE_SWEDEN_API_BASE, api_key: os.environ/AZURE_SWEDEN_API_KEY, api_version: "2025-08-28", realtime_protocol: GA}
  - model_name: openai-realtime-refused
    litellm_params: {model: openai/gpt-realtime, api_key: sk-invalid-e2e}
  - model_name: openai-realtime-valid
    litellm_params: {model: openai/gpt-realtime, api_key: os.environ/OPENAI_API_KEY}
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Client, run interactively in a tmux pane per model (wscat only prints its Disconnected line in interactive mode), pane captured 14 s after connecting:

npx --yes wscat -c 'ws://localhost:<port>/v1/realtime?model=<model_name>' -H 'Authorization: Bearer <master key>'

Before (a59ecfb, the merge base, port 22510, 2 workers)

azure-realtime-refused, Azure refuses the websocket upgrade with HTTP 401:

Connected (press CTRL+C to quit)
Disconnected (code: 1006, reason: "")

openai-realtime-refused, OpenAI accepts the upgrade and rejects the key in band (control):

Connected (press CTRL+C to quit)
< {"type": "error", "event_id": "event_EQyPppPmRRMf7k6PxEcpY", "error": {"type": "invalid_request_error", "code": "invalid_api_key", "message": "Incorrect API key provided: sk-inval**-e2e. You can find your API key at https://platform.openai.com/account/api-keys.", "param": null, "event_id": null}}
< {"type": "error", "error": {"type": "server_error", "message": "upstream websocket closed with code 3000: invalid_request_error.invalid_api_key"}}
Disconnected (code: 3000, reason: "invalid_request_error.invalid_api_key")

azure-realtime-valid and openai-realtime-valid (controls): session.created arrived (sess_EQyQ5gs7fBc8Ahn3kkg8T, sess_EQyQKumAfI21IPiIGheLA) and both sockets stayed open

After (cd9636c, the PR tip, port 49813, 2 workers)

azure-realtime-refused:

Connected (press CTRL+C to quit)
< {"type": "error", "error": {"type": "server_error", "message": "Upstream realtime handshake rejected with HTTP 401"}}
Disconnected (code: 1008, reason: "Upstream realtime handshake rejected with HTTP 401")

openai-realtime-refused (control): identical to before, the in-band OpenAI invalid_api_key event, the relayed server_error, then Disconnected (code: 3000, reason: "invalid_request_error.invalid_api_key")

azure-realtime-valid and openai-realtime-valid (controls): session.created arrived (sess_EQyQBiguK2SA0B6FyAwMv, sess_EQyQQBCXCTo43ydZ5WKKA) and both sockets stayed open

Merged into current main (d1afec12b9 = cd9636c merged into f275be5, port 37420, 2 workers)

Same four results as After: azure-realtime-refused got the error event and Disconnected (code: 1008, reason: "Upstream realtime handshake rejected with HTTP 401"), openai-realtime-refused still closed 3000 in band, and both valid deployments got session.created (sess_EQyUUxwESdjWJnwXX0b45, sess_EQyUkuIi1uNw0oxGeWw5Q)

Observations from the run:

• OpenAI accepts the upgrade and refuses in band, unchanged here
• Valid Azure and OpenAI sessions behave the same on both legs
• Azure 401 and OpenAI relay 403 refusals driven live

OpenAI relay, upstream refuses the upgrade (side leg at a59ecfb and cd9636c)

The OpenAI relay's new branch needs a host that refuses the HTTP upgrade, and OpenAI's realtime endpoint never does (it accepts a bad key and refuses in band, the control above). OpenAIRealtime._construct_url keeps the api_base host and sets the path to /v1/realtime, so a deployment with api_base: https://platform.openai.com dials wss://platform.openai.com/v1/realtime, which refuses the upgrade with HTTP 403. One extra proxy per commit, booted the same way as the legs above (--num_workers 2, no database, random free port), with this config:

model_list:
  - model_name: openai-realtime-refused-upgrade
    litellm_params: {model: openai/gpt-realtime, api_base: https://platform.openai.com, api_key: os.environ/OPENAI_API_KEY}
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Before (a59ecfb, port 56557): npx --yes wscat -c 'ws://localhost:56557/v1/realtime?model=openai-realtime-refused-upgrade' -H 'Authorization: Bearer <master key>'

Connected (press CTRL+C to quit)
Disconnected (code: 1011, reason: "Internal server error: server rejected WebSocket connection: HTTP 403")

No error event: the dead InvalidStatusCode branch is skipped and the generic except closes 1011 with the exception text as the reason

After (cd9636c, port 42468): npx --yes wscat -c 'ws://localhost:42468/v1/realtime?model=openai-realtime-refused-upgrade' -H 'Authorization: Bearer <master key>'

Connected (press CTRL+C to quit)
< {"type": "error", "error": {"type": "server_error", "message": "Upstream realtime handshake rejected with HTTP 403"}}
Disconnected (code: 1008, reason: "Upstream realtime handshake rejected with HTTP 403")

Type

🐛 Bug Fix
✅ Test

Caveats (if any)

No severe, high, or medium caveats at cd9636c

Low

Each of these stays as is because fixing it costs more than it returns: every new commit re-enters the bot, CI, and QA loop, and CircleCI runs the branch tip, so a fix that landed on main after this branch's merge base reaches the non-required jobs below only through the merge itself, while merging main in would reset the bot verdicts and the per-commit QA

• Close code mapping is deliberately small (1008 for 401/403, 1013 for 429, 1011 otherwise); a raw HTTP status is never a valid close code, which is why the old websocket.close(code=e.status_code) could not have worked even when the branch was reachable. A finer map (404, 5xx) would be a new client contract nobody has asked for
• The error event and close reason name only the HTTP status, never the upstream response body: the close reason is capped at 123 bytes and the upstream body is provider prose that can carry resource names, while the status is what a client can branch on. Forwarding the body would push provider text through a 123-byte truncation
• The Azure relay's generic except now sends a fixed Internal server error event before its 1011, while the OpenAI relay's generic except still closes 1011 with the redacted exception text in the reason and no event (pre-existing on both relays). Unifying the two generic paths is follow-up scope, since it widens the diff past the handshake refusal this PR fixes
• close_after_upstream_handshake_refusal swallows a failed send_text but does not wrap websocket.close; a close that raises lands in the proxy layer's existing generic except and finally, which already close the client socket, so wrapping it would only hide that failure
• The Azure InvalidStatus branch logs a full traceback (verbose_proxy_logger.exception) per refused handshake, one stack per bad-credential connection in the proxy log, the same way its pre-existing generic branch does. Cosmetic, and a .warning swap is a re-review round for a log line
• CloseCode is imported inside the helper's functions rather than at module level. Tidier at module level, not worth a re-review round
• The e2e test covers the Azure relay only. The OpenAI relay's refusal branch is covered by its unit tests and by the live side leg above, and an e2e leg for it would rest on a third-party host refusing a non-API path, which is a flake waiting to happen
• The scripts/type_check_gate.py and budget gates were not runnable in the authoring environment (no PyPI access to build the typecheck venv); basedpyright was run directly on the changed files with no new diagnostics, and CircleCI lint is green at the tip
• CircleCI unit at cd9636c (2201155, same on the first run 2201017) is red only on tests/unit/enterprise/enterprise_callbacks/test_secret_detection.py::test_scan_message_stays_linear_on_adversarial_credential_lines[assignment-flood] (wall-clock bound, 11.1 s against 10 s); main's own pipelines fail it the same way (90021, 90023, 90056), and test(unit): make bedrock collector and secret scan timing tests deterministic #42405 (691c0d6, merged 2026-09-22 16:30Z, after this branch's merge base) made that test deterministic on main
• CircleCI llm_translation_testing at cd9636c (2201156, the identical list on the first run 2201043) is red on 12 tests, none touching realtime: 10 in tests/llm_translation/test_fireworks_ai_translation.py (test_transform_inline_no_longer_added x5, test_document_inlining_example x2, test_global_disable_flag_no_longer_adds_transform_inline x2, test_global_disable_flag_with_transform_messages_helper), red on main since fix(registry): add MAI-Image-2.5-Pro pricing, fix Fireworks/Together entries, absorb verified open registry PRs, add Groq deprecation and Bedrock regional Qwen3 Next pricing #34941 (701c2b7, in this branch's merge base) flipped supports_vision on the cost-map row those tests assert against (90056, 90060); test_optional_params.py::test_drop_nested_params_add_prop_and_strict[hosted_vllm-my-vllm-model], fixed on main by fix(utils): stop a nested additional_drop_params entry from crashing openai-compatible calls #42492 (d47e72f, merged 2026-09-22 17:50Z, after this branch's merge base); and test_bedrock_moonshot.py::TestBedrockMoonshotInvoke::test_json_response_format_stream, red on the same main runs
• CircleCI local_testing_part1 at cd9636c (2201154) is red on tests/local_testing/test_get_model_info.py::test_get_model_info_bedrock_cross_region_capability_parity and ::test_get_model_info_bedrock_models; both went red on main with chore(prices): sync OpenRouter prices: 7 models #42261 (1fbfb46, in this branch's merge base) and main has failed them on every run since, 89979 through 90060, none touching realtime
• CircleCI e2e_ui_testing at cd9636c is red only on tests/mcp/mcpTools.spec.ts "MCP Tools tab lists the tools the upstream server advertises" on both runs (2201001: 1 failed, 153 passed; rerun 2201153: 1 failed, 151 passed, 2 flaky that passed on retry); main failed that same spec on 90021, 90023, and 90036, and test(e2e-ui): check the MCP Tools tab against the upstream's own tools/list #42397 (b96842f, merged after this branch's merge base) fixed it, so main's 90056 and 90060 pass
• CircleCI integration-cost (2201099: 23 test_case_bills_expected_cost cases over gpt-5.3-codex, gpt-5.5-pro, gpt-5.6, and azure gpt-5.6 streams billing 0.0) and integration-providers (2201098: test_fal_ai_video_wire::test_fal_h3_video_create_uses_canonical_body_and_status_path) at cd9636c fail the same lists as main's 90021 and 90023; fix(logging): price terminal Responses stream events from their inner response #42385 (3bbbf7f, merged after this branch's merge base) fixed both on main, so main's 90036, 90056, and 90060 pass both jobs, and LIT-8312 tracks the fal red
• Buildkite e2e-tests build 16377 at cd9636c (the triggered litellm-e2e-pr build 629) is red on 8 of 186 selected tests, none of them realtime: six test_xiaomi_mimo_e2e.py cases with an upstream 401 Invalid API Key, test_embeddings_endpoint_e2e.py::test_cohere_embeddings_returns_vector timing out, and test_ocr_rust_e2e.py::test_rust_ocr_response[mistral] rate limited with a 429; the PR's test_upstream_handshake_refusal_is_an_error_event_and_policy_close[azure-bad-key] passed in that build

/live-pr-risk at cd9636c

Breaking: none observed. Every path driven live (Azure refused, Azure valid, OpenAI refused in band, OpenAI valid, and the OpenAI relay against an upgrade-refusing host) answered the same on the merge base, the tip, and the tip merged into main, except the handshake refusals the PR targets: Azure 401 on all three legs and the OpenAI relay's 403 on the side leg

Backward incompatible: the client-visible outcome of an upstream handshake refusal changes from a bare 1006 with no frame (Azure relay) or a 1011 from the generic except (OpenAI relay, llm_http_handler realtime and responses websocket, proxy fallback, where the InvalidStatusCode branch was dead) to an error event plus 1008 for 401/403, 1013 for 429, 1011 otherwise. That change is the fix the linked ticket asks for. The Azure relay's generic failure now also sends a fixed Internal server error event before the 1011 it already sent; observable only on a non-handshake failure, not driven live here

Regression risk: llm_http_handler.async_realtime (the base_llm_http_handler route, vertex_ai included) and async_realtime_responses, the proxy_server.realtime_websocket_endpoint fallback (reachable only when a handler lets InvalidStatus escape, which none of the changed handlers do now), and the Azure generic-except error event are covered by the unit tests in the four mapped test files but were not driven live, because no provider in this rig refuses the upgrade on those routes (OpenAI's realtime endpoint accepts it and refuses in band; the OpenAI relay's own refusal branch was driven live through the side leg above)

Dependency graph, from the string dispatch in litellm/realtime_api/main.py: azure_realtime.async_realtime verified live on all three legs, openai_realtime.async_realtime verified live on the accepted-upgrade path (all three legs) and on the refused-upgrade path (side leg at the merge base and the tip), base_llm_http_handler.async_realtime and async_realtime_responses unit-tested, proxy_server.realtime_websocket_endpoint fallback unit-tested, _ProxyClientWebSocket.send_text (new duck-typed method the helper needs) verified live on the Azure leg, bedrock_realtime and xai_realtime untouched and carry no InvalidStatus branch. The legacy suites under tests/llm_translation/realtime/ run in CircleCI realtime_translation_testing, green at the tip. uv.lock and pyproject.toml unchanged

Not verified: Bedrock, xAI, and Vertex AI realtime refusals (no refusing deployment available on those routes), the responses websocket refusal, and the Azure generic-except path (needs a non-handshake failure)

QA runbook

• tests/e2e/llm_translation/realtime/test_realtime_e2e.py::test_upstream_handshake_refusal_is_an_error_event_and_policy_close[azure-bad-key] - a realtime deployment whose Azure credential is refused at the websocket handshake yields an error event and a 1008 close instead of a bare 1006
  • Ensure the proxy has AZURE_API_BASE set to a real Azure OpenAI resource that hosts a gpt-realtime deployment
  • POST http://localhost:4000/model/new with the master key and {"model_name": "azure-realtime-refused", "litellm_params": {"model": "azure/gpt-realtime", "api_key": "invalid-e2e-key", "api_version": "2025-08-28", "realtime_protocol": "GA"}}
  • Connect wscat -c "ws://localhost:4000/v1/realtime?model=azure-realtime-refused" -H "Authorization: Bearer <key>"
  • Expect the first frame to be {"type": "error", "error": {"type": "server_error", "message": "Upstream realtime handshake rejected with HTTP 401"}}
  • Expect wscat to then print Disconnected (code: 1008, reason: "Upstream realtime handshake rejected with HTTP 401")
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/5865afbb93d74388884fa3cf89f5ec0d
Open in Devin Desktop: https://app.devin.ai/desktop/session/5865afbb93d74388884fa3cf89f5ec0d?variant=devin

• cd9636c passes /live-pr-risk

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[mateo-berri]

bugbot run

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness or repository-rule violations.

Findings

1.  Test docstrings violate policy ▶

Summary

This PR improves realtime WebSocket failure handling when an upstream provider refuses the HTTP upgrade.

• Catches the current websockets.exceptions.InvalidStatus exception across realtime relay paths.
• Sends a structured client error before closing with an appropriate WebSocket close code.
• Prevents unexpected Azure connection errors from exposing exception details.
• Adds unit and live Azure coverage for handshake refusal behavior.

_{Reviews (3) · Last reviewed commit: "refactor(realtime): map handshake refusa..."}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_realtime_upstream_handshake_refusal (cd9636c) with main (5a76420)}

[greptile-apps]

Test docstrings violate policy

This explanatory docstring, and the matching one in tests/test_litellm/llms/openai/realtime/test_openai_realtime_handler.py:423-424, repeat behavior already stated by the test names. The repository's AGENTS.md directive permits comments only for necessary complex logic, tool directives, or TODO/FIXME notes. Remove both docstrings to satisfy this requirement before merging.

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[mateo-berri]

Fixed in 9c873e1: both docstrings are removed, the test names already say what is being checked

[cursor]

Stale Bugbot comment from a previous run.

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/azure/realtime/handler.py	69.23%	4 Missing ⚠️
litellm/llms/custom_httpx/llm_http_handler.py	60.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit cd9636c. Configure here.}

[mateo-berri]

@greptileai

[mateo-berri]

LGTM