Skip to content

fix(proxy): preserve HTTP operations when injecting WebSocket stubs i… - #27244

Closed
michelligabriele wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_fix_swagger_responses_websocket_clobber
Closed

fix(proxy): preserve HTTP operations when injecting WebSocket stubs i…#27244
michelligabriele wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_fix_swagger_responses_websocket_clobber

Conversation

@michelligabriele

Copy link
Copy Markdown
Collaborator

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Screenshots / Proof of Fix

Starting v1.82.x, POST /v1/responses and POST /responses disappeared from Swagger UI even though both routes still serve traffic.

Before the fix — Swagger UI's responses tag, with the two POST entries missing (only POST /openai/v1/responses is visible at the top, since it doesn't have a WebSocket twin):

before

After the fix — same view, both POST /v1/responses and POST /responses restored alongside POST /openai/v1/responses:
after

JSON-level proof from probing /openapi.json before and after:

--- presence of POST per path ---     (before fix)
  POST /v1/responses             NO
  POST /responses                NO
  POST /openai/v1/responses      YES

--- presence of POST per path ---     (after fix)
  POST /v1/responses             YES
  POST /responses                YES
  POST /openai/v1/responses      YES

The routes themselves were registered and serving traffic in both states — this was a documentation-layer bug, not a routing one.

Type

🐛 Bug Fix

Changes

What was wrong

get_openapi_schema() in litellm/proxy/proxy_server.py injects a synthetic GET stub for every WebSocket route so it shows up in Swagger UI. The injection was a wholesale dict assignment:

openapi_schema["paths"][base_path] = {
    "get": { ... websocket stub ... }
}

Before v1.82.x every WebSocket path was unique, so this never bit. v1.82.x added @router.websocket("/v1/responses") and @router.websocket("/responses") — paths that already had @router.post(...) registered. The wholesale assignment overwrote the existing path entry, dropping the POST operation from the served OpenAPI schema entirely. POST /openai/v1/responses stayed visible only because it has no WebSocket twin.

A vanilla FastAPI app with the same @app.post(...) + @app.websocket(...) pair on the same path produces a correct OpenAPI schema (FastAPI's own get_openapi() skips WebSocket routes by design), so the regression was entirely in LiteLLM's custom WebSocket-injection.

What this changes

  • Extract the WebSocket-stub loop into a module-level helper _inject_websocket_stubs_into_openapi_schema(openapi_schema, websocket_routes) so it can be unit-tested in isolation.
  • Replace the wholesale assignment with setdefault(base_path, {}) (gets the existing entry or creates an empty one — never clobbers) plus a if "get" not in path_entry guard (skips the synthetic stub if a real GET is already documented). This restores the missing POST entries and also closes the same trap for any future GET-vs-WebSocket path collision.
  • For paths with no HTTP twin (every WebSocket path before v1.82.x and the future norm), behavior is unchanged — setdefault({}) creates a fresh entry, then the same stub gets written.

Tests

New TestWebSocketStubInjection class in tests/test_litellm/proxy/test_openapi_schema_validation.py with 4 unit tests:

  1. test_websocket_stub_does_not_clobber_existing_post — when a WebSocket route shares a path with a POST, the POST survives and the GET stub is added alongside.
  2. test_websocket_stub_added_when_path_is_new — WebSocket-only paths still get a fresh {"get": stub} entry (no behavior change).
  3. test_websocket_stub_skipped_when_existing_get — a real GET takes precedence over the synthetic stub (closes the same trap for future GET collisions).
  4. test_responses_post_routes_registered_on_router — sanity check that /v1/responses, /responses, and /openai/v1/responses are still wired with POST on the responses router. Guards against silent removal at the source.

Files touched

  • litellm/proxy/proxy_server.py — extract helper + switch to setdefault merge
  • tests/test_litellm/proxy/test_openapi_schema_validation.py — add TestWebSocketStubInjection class

Verification

  • tests/test_litellm/proxy/test_openapi_schema_validation.py — 11 passed (7 existing + 4 new)
  • tests/test_litellm/proxy/test_lazy_openapi_snapshot.py and tests/mcp_tests/test_openapi_spec_path_url.py — passed (adjacent OpenAPI tests, no regressions)
  • make lint-ruff — clean
  • uv run black — clean
  • End-to-end against a running proxy: probing /openapi.json shows both missing POSTs flip from NO to YES; control endpoint and HTTP-route-alive checks unchanged; Swagger UI at /docs renders the restored entries under the responses tag (see screenshots above).

@greptile-apps

greptile-apps Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Fixes a documentation-layer regression (introduced in v1.82.x) where adding @router.websocket(...) on a path that already had @router.post(...) caused the POST to vanish from the OpenAPI schema. The root cause was a wholesale dict assignment that overwrote any existing path entry; the fix replaces it with dict.setdefault plus a "get" not in guard inside the new helper _inject_websocket_stubs_into_openapi_schema.

  • proxy_server.py: WebSocket-stub injection logic is extracted into a standalone helper. setdefault ensures existing HTTP operations (e.g. POST /v1/responses) are merged with, not replaced by, the synthetic GET stub; an existing real GET also takes precedence over the stub.
  • test_openapi_schema_validation.py: Four new unit tests cover the exact regression scenario, the no-twin path (unchanged behavior), the GET-wins guard, and a router-registration sanity check — all mock-only, no network calls.

Confidence Score: 5/5

Safe to merge — the change is confined to OpenAPI schema generation (a cold, documentation-only path) and cannot affect live request routing.

The fix is minimal and surgical: one wholesale dict assignment replaced by setdefault plus a guard, wrapped in a new testable helper. The bug only affected what appeared in /openapi.json and Swagger UI, never actual request dispatch. Four new unit tests directly exercise the regression case and all boundary conditions. No existing tests were weakened, no auth or request paths were touched.

No files require special attention.

Important Files Changed

Filename Overview
litellm/proxy/proxy_server.py Extracts WebSocket-stub injection into _inject_websocket_stubs_into_openapi_schema; replaces wholesale dict assignment with setdefault+if "get" not in guard so HTTP operations on shared paths are preserved
tests/test_litellm/proxy/test_openapi_schema_validation.py Adds 4 targeted unit tests for the new helper — no real network calls, covers clobber regression, WebSocket-only paths, GET precedence, and router registration sanity check

Reviews (1): Last reviewed commit: "fix(proxy): preserve HTTP operations whe..." | Re-trigger Greptile

@codecov

codecov Bot commented May 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@oss-pr-review-agent-shin

Copy link
Copy Markdown
Contributor

🤖 litellm-agent: Merged into staging branch litellm_agent_oss_staging_05_06_2026. Staging PR: #27256


Triage Summary
Refactors the OpenAPI schema generation in proxy_server.py to prevent WebSocket stub injection from silently dropping existing HTTP operations on shared paths. Extracts the stub-injection logic into a standalone helper (_inject_websocket_stubs_into_openapi_schema) that merges into existing path entries rather than replacing them, and skips adding a GET stub when a real GET is already present. Adds a regression test class covering the POST-preservation case and the skip-existing-GET case.

Merge Confidence: 5/5 ✅ READY
Ready to ship.

All checks green. Greptile 5/5, no blocking pattern findings, CircleCI passed.

oss-pr-review-agent-shin Bot added a commit that referenced this pull request May 6, 2026
oss-pr-review-agent-shin Bot added a commit that referenced this pull request May 6, 2026
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant