fix(proxy): gate the OpenAI websocket passthrough behind an explicit opt-in

[mateo-berri]

TLDR

Problem this solves:

• Passthrough WebSocket routes relay any frames under the proxy's OpenAI key
• Any unrestricted key can run any OpenAI model, unmetered, model logged "unknown"
• A key with no model list of its own, owned by a user (or team, project) that has one, is relayed too: the handshake only checks the model query string, and /openai/v1/responses has none
• Both /openai_passthrough/* and /openai/* WebSocket routes were on by default

How it solves it:

• The passthrough WebSocket routes are now off unless an admin opts in
• New general_settings.enable_openai_websocket_passthrough: true turns them back on, from YAML or through POST /config/field/update when store_model_in_db is on (the field is not on the Admin UI general settings page, which only lists the alerting fields); every pod picks the DB value up on its next config reload and a YAML value always wins
• Refused sockets get one error event naming the setting, then close 1008
• Once enabled, an identity with a model restriction at any enforced level (key, team, team membership, user, project) gets an error event naming the reason instead of a relay, on both passthrough routes, whether or not the URL names a model
• Without a database the gate reads the token's own key and team model lists, and with one an identity whose user row is missing (the master key on a fresh database) counts as having no user-level list, the same as HTTP auth, instead of failing the handshake; any other user lookup failure refuses the socket rather than relaying

User Flow

Before: a developer with an ordinary virtual key opens a WebSocket through the passthrough prefix and the gateway relays it to OpenAI on its own credential, for a model the gateway does not list, with no spend recorded

1. A proxy admin runs the proxy with a model_list holding gpt-5.6 and gpt-realtime-2.1-mini, both on the proxy's own OpenAI key, and no passthrough settings at all
2. A developer with a key that has no model restriction connects to wss://litellm-domain/openai_passthrough/v1/realtime?model=gpt-realtime-2.1 with Authorization: Bearer <their key>
3. The handshake succeeds and a session.created event comes back naming gpt-realtime-2.1, a model the proxy never listed
4. They send session.update, conversation.item.create, and response.create, and response.done arrives with the model's answer
5. They connect to wss://litellm-domain/openai/v1/responses the same way, send one response.create for gpt-5.6, and response.completed arrives with the answer
6. A second developer, whose key carries no model list but whose user record is restricted to gpt-5.6, is refused at the handshake on the realtime URL (its query string names a model) and relayed on wss://litellm-domain/openai/v1/responses exactly like step 5, because that handshake names no model
7. The admin opens https://litellm-domain/ui/?page=logs: the sessions show model unknown, spend $0.00, and zero tokens, and GET /key/info for those keys still reports spend 0
8. The admin tries to switch the relay off through POST /config/field/update and gets 400 Invalid field=enable_openai_websocket_passthrough passed in.; there is no switch
9. Any holder of any key on this proxy whose own model list is empty can run any OpenAI model the proxy's credential can reach, outside every budget and model allow-list

After: the same connections are refused with a message naming the setting until the admin opts in, the proxy's own realtime route keeps working, and every model restriction refuses the relay once it is on

1. A proxy admin runs the proxy with a model_list holding gpt-5.6 and gpt-realtime-2.1-mini, both on the proxy's own OpenAI key, and no passthrough settings at all
2. A developer with a key that has no model restriction connects to wss://litellm-domain/openai_passthrough/v1/realtime?model=gpt-realtime-2.1 with Authorization: Bearer <their key>
3. The handshake succeeds, the first event is {"type": "error", "error": {"type": "invalid_request_error", "message": "OpenAI websocket passthrough is disabled on this gateway. A proxy admin can turn it on by setting general_settings.enable_openai_websocket_passthrough to true."}}, and the socket closes with code 1008
4. wss://litellm-domain/openai/v1/responses answers with the same error event and close code
5. The developer switches to the proxy's own wss://litellm-domain/v1/realtime?model=gpt-realtime-2.1-mini, and the session runs end to end for that listed model
6. The admin who really wants the relay either adds general_settings: {enable_openai_websocket_passthrough: true} and restarts, or, with store_model_in_db: true, sends POST /config/field/update with {"field_name": "enable_openai_websocket_passthrough", "field_value": true, "config_type": "general_settings"} (the Admin UI general settings page does not list this field) and every pod picks it up within the config reload interval with no restart; the passthrough connections in steps 2 and 4 now run as before, and POST /config/field/delete turns them back off the same way
7. Once enabled, the second developer from Before step 6, and any key, team, team member, user, or project with a model list, gets {"type": "error", "error": {"type": "invalid_request_error", "message": "Keys with model restrictions cannot use OpenAI websocket passthrough, because this route relays frames to the provider without reading which model they ask for."}} and close 1008 on both passthrough routes, even when the restriction allows the model in the URL
8. Nobody can drive the proxy's OpenAI credential over the passthrough WebSocket routes unless an admin turned the relay on for that deployment, and then only with an identity that has no model restriction anywhere

Relevant issues

The WebSocket passthrough route was added for #36088 and shipped on by default in v1.99.0; this PR keeps that capability behind an explicit opt-in

Linear ticket

Resolves LIT-6849

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup for every run below. Real OpenAI key in the environment, real spend. Every proxy runs with --num_workers 2 against a Postgres database; the DB opt-in run boots two separate proxy processes on two ports sharing that one database. PORT is whichever free port that run used

# config.yaml (the After runs that opt in add one line under general_settings, shown in each case)
model_list:
  - model_name: gpt-5.6
    litellm_params:
      model: openai/gpt-5.6
      api_key: os.environ/OPENAI_API_KEY
  - model_name: gpt-realtime-2.1-mini
    litellm_params:
      model: openai/gpt-realtime-2.1-mini
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

python litellm/proxy/proxy_cli.py --config config.yaml --port $PORT --num_workers 2 --detailed_debug

H=(-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json")
gen() { curl -s "http://127.0.0.1:$PORT/key/generate" "${H[@]}" -d "$1" | jq -r .key; }
UNRESTRICTED=$(gen '{"key_alias":"ws-unrestricted"}')
RESTRICTED_GPT56=$(gen '{"key_alias":"ws-restricted-gpt56","models":["gpt-5.6"]}')
RESTRICTED_RT=$(gen '{"key_alias":"ws-restricted-realtime","models":["gpt-realtime-2.1"]}')

# a key with no model list of its own, owned by an internal user who is then restricted to gpt-5.6
curl -s "http://127.0.0.1:$PORT/user/new" "${H[@]}" -d '{"user_id":"lit6849-user-gpt56","auto_create_key":false}'
USER_RESTRICTED=$(gen '{"key_alias":"ws-user-restricted","user_id":"lit6849-user-gpt56"}')
curl -s "http://127.0.0.1:$PORT/user/update" "${H[@]}" -d '{"user_id":"lit6849-user-gpt56","models":["gpt-5.6"]}'

ws_client.py is the client used for every WebSocket leg: it connects with Authorization: Bearer <key>, sends each --send frame, prints every event type it gets back (full text for error, session.*, and response.completed), and prints the close code and reason when the server closes

ws_client.py

import argparse, asyncio, json, sys
import websockets


def closed_code(e):
    rcvd = getattr(e, "rcvd", None)
    return (rcvd.code, rcvd.reason) if rcvd is not None else (None, "")


async def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("--url", required=True)
    ap.add_argument("--key", required=True)
    ap.add_argument("--send", action="append", default=[])
    ap.add_argument("--frames", type=int, default=20)
    ap.add_argument("--timeout", type=float, default=25.0)
    args = ap.parse_args()

    try:
        ws = await websockets.connect(args.url, additional_headers={"Authorization": f"Bearer {args.key}"})
    except websockets.exceptions.InvalidStatus as e:
        print(f"HANDSHAKE REJECTED: HTTP {e.response.status_code}")
        return 1

    print("CONNECTED")
    etype = None
    try:
        for payload in args.send:
            await ws.send(payload)
            print(f"SENT -> {payload}")
        seen = 0
        while seen < args.frames:
            raw = await asyncio.wait_for(ws.recv(), timeout=args.timeout)
            obj = json.loads(raw)
            etype = obj.get("type", "?")
            if etype in ("response.output_text.done", "response.text.done"):
                print(f"RECV <- {etype} text={obj.get('text')!r}")
            elif etype in ("error", "response.done", "response.completed", "session.created", "session.updated"):
                print(f"RECV <- {etype} {json.dumps(obj)[:600]}")
            else:
                print(f"RECV <- {etype}")
            seen += 1
            if etype in ("response.done", "response.completed", "error"):
                break
        if etype == "error":
            try:
                await asyncio.wait_for(ws.recv(), timeout=5)
            except websockets.exceptions.ConnectionClosed as e:
                code, reason = closed_code(e)
                print(f"CLOSED code={code} reason={reason!r}")
    except websockets.exceptions.ConnectionClosed as e:
        code, reason = closed_code(e)
        print(f"CLOSED code={code} reason={reason!r}")
    finally:
        await ws.close()
    return 0


sys.exit(asyncio.run(main()))

REALTIME='/openai_passthrough/v1/realtime?model=gpt-realtime-2.1'
RESPONSES='/openai/v1/responses'
MANAGED='/v1/realtime?model=gpt-realtime-2.1-mini'
realtime_leg() { python ws_client.py --url "ws://127.0.0.1:$PORT$1" --key "$2" \
  --send '{"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}' \
  --send '{"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Reply with exactly LIT6849 and nothing else."}]}}' \
  --send '{"type":"response.create"}'; }
responses_leg() { python ws_client.py --url "ws://127.0.0.1:$PORT$1" --key "$2" \
  --send '{"type":"response.create","model":"gpt-5.6","input":"Reply with exactly LIT6849 and nothing else."}'; }

Before (59d42d3)

An unrestricted key drives a realtime model that is not in model_list through the passthrough

1. realtime_leg "$REALTIME" "$UNRESTRICTED"
2. The proxy accepts the socket and relays it. gpt-realtime-2.1 is not in model_list, and the answer comes back anyway:

CONNECTED
SENT -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
SENT -> {"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Reply with exactly LIT6849 and nothing else."}]}}
SENT -> {"type":"response.create"}
RECV <- session.created {"type": "session.created", "event_id": "event_<redacted>", "session": {"type": "realtime", "object": "realtime.session", "id": "sess_<redacted>", "model": "gpt-realtime-2.1",  ...
RECV <- session.updated {"type": "session.updated", ... "model": "gpt-realtime-2.1", "output_modalities": ["text"], ...
RECV <- conversation.item.added
RECV <- conversation.item.done
RECV <- response.created
RECV <- response.output_item.added
RECV <- conversation.item.added
RECV <- response.content_part.added
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.done text='LIT6849'
RECV <- response.content_part.done

The same key drives the Responses API over the /openai/v1/responses WebSocket passthrough

1. responses_leg "$RESPONSES" "$UNRESTRICTED"
2. Relayed too, and the response completes:

CONNECTED
SENT -> {"type":"response.create","model":"gpt-5.6","input":"Reply with exactly LIT6849 and nothing else."}
RECV <- response.created
RECV <- response.in_progress
RECV <- response.output_item.added
RECV <- response.content_part.added
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.done text='LIT6849'
RECV <- response.content_part.done
RECV <- response.output_item.done
RECV <- response.completed {"type": "response.completed", "response": {"id": "resp_<redacted>", "object": "response", "created_at": 1788569689, "status": "completed", ... "model": "gpt-5.6-sol", ...

Keys with a model list are refused at the handshake, whichever model they name

1. realtime_leg "$REALTIME" "$RESTRICTED_GPT56"
2. HANDSHAKE REJECTED: HTTP 403
3. realtime_leg "$REALTIME" "$RESTRICTED_RT"
4. HANDSHAKE REJECTED: HTTP 403 (the key allows exactly the model it asked for, and it is still refused, because the route cannot read which model the frames name)

The proxy's own /v1/realtime route, for reference

1. realtime_leg "$MANAGED" "$UNRESTRICTED"
2. Works, with gpt-realtime-2.1-mini from model_list: ... RECV <- response.output_text.done text='LIT6849' ...

What the proxy recorded for the passthrough sessions

1. curl -s "http://127.0.0.1:$PORT/key/info" -H "Authorization: Bearer $UNRESTRICTED" | jq ".info.spend, .info.models"
2. 0.0 and []
3. curl -s "http://127.0.0.1:$PORT/spend/logs" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | jq -c ".[] | {model, spend, total_tokens, api_base}"
4. Two realtime turns and one Responses turn ran against the provider under the proxy's own key, and none of them cost the caller anything:

{"model":"","spend":0.0,"total_tokens":0,"api_base":""}
{"model":"openai/gpt-realtime-2.1-mini","spend":0.0,"total_tokens":0,"api_base":""}
{"model":"gpt-realtime-2.1","spend":0.0,"total_tokens":0,"api_base":""}
{"model":"unknown","spend":0.0,"total_tokens":0,"api_base":"wss://api.openai.com/v1/responses"}

A key with no model list of its own, owned by a user restricted to gpt-5.6, is relayed on the Responses WebSocket

1. curl -s "http://127.0.0.1:$PORT/user/info?user_id=lit6849-user-gpt56" "${H[@]}" | jq .user_info.models prints ["gpt-5.6"]
2. curl -s "http://127.0.0.1:$PORT/key/info" -H "Authorization: Bearer $USER_RESTRICTED" | jq '{models: .info.models, user_id: .info.user_id, team_id: .info.team_id}' prints {"models": [], "user_id": "lit6849-user-gpt56", "team_id": null}
3. realtime_leg "$REALTIME" "$USER_RESTRICTED" gets HANDSHAKE REJECTED: HTTP 403: the query string names a model the user may not call, so the handshake auth catches this one
4. responses_leg "$RESPONSES" "$USER_RESTRICTED"
5. That handshake names no model, nothing reads the user's list, and the frames are relayed under the proxy's credential:

CONNECTED
SENT -> {"type":"response.create","model":"gpt-5.6","input":"Reply with exactly LIT6849 and nothing else."}
RECV <- response.created
RECV <- response.in_progress
RECV <- response.output_item.added
RECV <- response.content_part.added
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.done text='LIT6849'
RECV <- response.content_part.done
RECV <- response.output_item.done
RECV <- response.completed {"type": "response.completed", "response": {"id": "resp_<redacted>", "object": "response", "created_at": 1788572617, "status": "completed", ...

The admin has no switch

1. curl -s -X POST "http://127.0.0.1:$PORT/config/field/update" "${H[@]}" -d '{"field_name":"enable_openai_websocket_passthrough","field_value":false,"config_type":"general_settings"}'
2. {"detail":{"error":"Invalid field=enable_openai_websocket_passthrough passed in."}} with HTTP 400

After (5c80e30)

The default-config legs, the enabled-config legs and the master key probe ran at 5c80e30, the tip that merges litellm_internal_staging on top of aca1c54. The refusal frames and the relayed sessions below came out byte for byte the same as the a0b2e7f run (ids aside). The DB opt-in legs ran at f846388 and the no-database leg at 2a7fc8d; the commits since only change what happens when a user lookup fails and how the refusal frame is built, and the default and enabled legs at 5c80e30 cover both

Default config, nothing added: both passthrough prefixes refuse with a readable reason

1. realtime_leg "$REALTIME" "$UNRESTRICTED"
2. The handshake still completes (so browser and SDK clients can read the reason), then the proxy sends one error event naming the setting and closes with 1008. Nothing reaches OpenAI:

CONNECTED
SEND REFUSED (socket already closing) -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
RECV <- error {"type": "error", "error": {"type": "invalid_request_error", "message": "OpenAI websocket passthrough is disabled on this gateway. A proxy admin can turn it on by setting general_setting ...
CLOSED code=1008 reason='OpenAI websocket passthrough is disabled'

3. responses_leg "$RESPONSES" "$UNRESTRICTED"
4. Same on the /openai/ prefix:

CONNECTED
SEND REFUSED (socket already closing) -> {"type":"response.create","model":"gpt-5.6","input":"Reply with exactly LIT6849 and nothing else."}
RECV <- error {"type": "error", "error": {"type": "invalid_request_error", "message": "OpenAI websocket passthrough is disabled on this gateway. A proxy admin can turn it on by setting general_setting ...
CLOSED code=1008 reason='OpenAI websocket passthrough is disabled'

Default config: the proxy's own /v1/realtime route is untouched

1. realtime_leg "$MANAGED" "$UNRESTRICTED"
2. ... RECV <- session.created {... "model": "gpt-realtime-2.1-mini", ...} ... RECV <- response.output_text.done text='LIT6849' ... RECV <- response.done {"type": "response.done", ... "status": "completed", ...

Opted in from YAML: the relay behaves exactly as before for unrestricted keys

1. Add the one line and restart the proxy:

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  enable_openai_websocket_passthrough: true

2. realtime_leg "$REALTIME" "$UNRESTRICTED"
3. Relayed, same as the Before run:

CONNECTED
SENT -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
SENT -> {"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Reply with exactly LIT6849 and nothing else."}]}}
SENT -> {"type":"response.create"}
RECV <- session.created {"type": "session.created", "event_id": "event_<redacted>", "session": {"type": "realtime", "object": "realtime.session", "id": "sess_<redacted>", "model": "gpt-realtime-2.1",  ...
RECV <- session.updated {"type": "session.updated", ... "model": "gpt-realtime-2.1", "output_modalities": ["text"], ...
RECV <- conversation.item.added
RECV <- conversation.item.done
RECV <- response.created
RECV <- response.output_item.added
RECV <- conversation.item.added
RECV <- response.content_part.added
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.done text='LIT6849'
RECV <- response.content_part.done
RECV <- conversation.item.done
RECV <- response.output_item.done
RECV <- response.done {"type": "response.done", "event_id": "event_<redacted>", "response": {"object": "realtime.response", "id": "resp_<redacted>", "status": "completed", ...

4. responses_leg "$RESPONSES" "$UNRESTRICTED"
5. ... RECV <- response.output_text.done text='LIT6849' ... RECV <- response.completed {"type": "response.completed", "response": {"id": "resp_<redacted>", "object": "response", ... "status": "completed", ...

Opted in: keys with a model list are still refused, now with a reason instead of a bare 403

1. realtime_leg "$REALTIME" "$RESTRICTED_RT" (the key allows exactly the model in the query string)
2. Refused after the handshake with the model-restriction reason, because the relay cannot check the model the frames name:

CONNECTED
SEND REFUSED (socket already closing) -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
RECV <- error {"type": "error", "error": {"type": "invalid_request_error", "message": "Keys with model restrictions cannot use OpenAI websocket passthrough, because this route relays frames to the pro ...
CLOSED code=1008 reason='Keys with model restrictions cannot use OpenAI websocket passthrough'

3. realtime_leg "$REALTIME" "$RESTRICTED_GPT56"
4. HANDSHAKE REJECTED: HTTP 403 (unchanged: the key's model list excludes the model in the query string, so the handshake auth refuses it before the route runs)

Opted in: the user-restricted identity from the Before run is refused on both routes

1. Same identity as before: /user/info prints ["gpt-5.6"], /key/info prints {"models": [], "user_id": "lit6849-user-gpt56", "team_id": null}
2. realtime_leg "$REALTIME" "$USER_RESTRICTED" still gets HANDSHAKE REJECTED: HTTP 403
3. responses_leg "$RESPONSES" "$USER_RESTRICTED"
4. The route now resolves the restriction the handshake could not see (the user's model list) and refuses instead of relaying:

CONNECTED
SEND REFUSED (socket already closing) -> {"type":"response.create","model":"gpt-5.6","input":"Reply with exactly LIT6849 and nothing else."}
RECV <- error {"type": "error", "error": {"type": "invalid_request_error", "message": "Keys with model restrictions cannot use OpenAI websocket passthrough, because this route relays frames to the pro ...
CLOSED code=1008 reason='Keys with model restrictions cannot use OpenAI websocket passthrough'

Opted in through the API: two proxy processes, one database, no restart

Both processes boot from the shared config plus store_model_in_db: true and no enable_openai_websocket_passthrough line, --num_workers 2 each, on ports $PA and $PB, sharing one Postgres database

1. PORT=$PA realtime_leg "$REALTIME" "$UNRESTRICTED" gets the disabled error event and CLOSED code=1008 reason='OpenAI websocket passthrough is disabled'
2. curl -s -X POST "http://127.0.0.1:$PA/config/field/update" "${H[@]}" -d '{"field_name":"enable_openai_websocket_passthrough","field_value":true,"config_type":"general_settings"}'
3. {"param_name":"general_settings","param_value":{"enable_openai_websocket_passthrough":true},"last_run_at":null,"reload_revision":0} with HTTP 200
4. After the config reload interval (45 s here), PORT=$PB realtime_leg "$REALTIME" "$UNRESTRICTED" on the other process:

CONNECTED
SENT -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
SENT -> {"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Reply with exactly LIT6849 and nothing else."}]}}
SENT -> {"type":"response.create"}
RECV <- session.created {"type": "session.created", "event_id": "event_<redacted>", "session": {"type": "realtime", "object": "realtime.session", "id": "sess_<redacted>", "model": "gpt-realtime-2.1",  ...
RECV <- session.updated {"type": "session.updated", ... "model": "gpt-realtime-2.1", "output_modalities": ["text"], ...
RECV <- conversation.item.added
RECV <- conversation.item.done
RECV <- response.created
RECV <- response.output_item.added
RECV <- conversation.item.added
RECV <- response.content_part.added
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.done text='LIT6849'
RECV <- response.content_part.done
RECV <- conversation.item.done
RECV <- response.output_item.done
RECV <- response.done {"type": "response.done", "event_id": "event_<redacted>", "response": {"object": "realtime.response", "id": "resp_<redacted>", "status": "completed", ...

5. curl -s -X POST "http://127.0.0.1:$PB/config/field/delete" "${H[@]}" -d '{"field_name":"enable_openai_websocket_passthrough","config_type":"general_settings"}'
6. {"param_name":"general_settings","param_value":{},"last_run_at":null,"reload_revision":0} with HTTP 200
7. After the reload interval, PORT=$PA realtime_leg "$REALTIME" "$UNRESTRICTED" on the first process is refused again:

CONNECTED
SEND REFUSED (socket already closing) -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
RECV <- error {"type": "error", "error": {"type": "invalid_request_error", "message": "OpenAI websocket passthrough is disabled on this gateway. A proxy admin can turn it on by setting general_setting ...
CLOSED code=1008 reason='OpenAI websocket passthrough is disabled'

Opted in, no database attached: a custom-auth identity that carries only a team model list

Custom auth hands back an identity with no key model list and a team model list of one model, and the proxy runs without DATABASE_URL

1. realtime_leg "$REALTIME" "$NODB_TEAM", then responses_leg /openai/v1/responses "$NODB_TEAM"
2. On the merge base both handshakes are rejected with HTTP 403 (the old check read the token's lists before accepting). At 2a7fc8d both are accepted, get the error event, and close 1008; the same output on both routes. Ran at 2a7fc8d, and the two commits after it only change the database-backed lookups, which a proxy without a database never runs:

CONNECTED
SEND REFUSED (socket already closing) -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
RECV <- error {"type": "error", "error": {"type": "invalid_request_error", "message": "Keys with model restrictions cannot use OpenAI websocket passthrough, because this route relays frames to the pro ...
CLOSED code=1008 reason='Keys with model restrictions cannot use OpenAI websocket passthrough'

Opted in: the master key on the passthrough realtime route

1. realtime_leg "$REALTIME" "$LITELLM_MASTER_KEY"
2. Relayed on the merge base. At 2a7fc8d the handshake was rejected with HTTP 500, because the admin's user row does not exist on a fresh database and the gate treated the failed lookup as fatal (found by the dependency sweep). At 5c80e30 it relays again:

CONNECTED
SENT -> {"type":"session.update","session":{"type":"realtime","output_modalities":["text"]}}
SENT -> {"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Reply with exactly LIT6849 and nothing else."}]}}
SENT -> {"type":"response.create"}
RECV <- session.created {"type": "session.created", "event_id": "event_<redacted>", "session": {"type": "realtime", "object": "realtime.session", "id": "sess_<redacted>", "model": "gpt-realtime-2.1",  ...
RECV <- session.updated {"type": "session.updated", "event_id": "event_<redacted>", "session": {"type": "realtime", "object": "realtime.session", "id": "sess_<redacted>", "model": "gpt-realtime-2.1",  ...
RECV <- conversation.item.added
RECV <- conversation.item.done
RECV <- response.created
RECV <- response.output_item.added
RECV <- conversation.item.added
RECV <- response.content_part.added
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.delta
RECV <- response.output_text.done text='LIT6849'
RECV <- response.content_part.done
RECV <- conversation.item.done
RECV <- response.output_item.done
RECV <- response.done {"type": "response.done", "event_id": "event_<redacted>", "response": {"object": "realtime.response", "id": "resp_<redacted>", "status": "completed", "status_details": null, "out ...

Opted in: what the proxy recorded

Captured during the a2d5215 run; 7351911 changes admission only, not logging

1. curl -s "http://127.0.0.1:$PORT/spend/logs" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | jq -c ".[] | {model, spend, total_tokens, api_base}"
2. The relayed passthrough sessions still land as $0 rows with 0 tokens, while the proxy's own realtime route rows carry usage. That gap is what LIT-7014 tracks, and it is why the route stays off unless an admin opts in:

{"model":"gpt-realtime-2.1","spend":0.0,"total_tokens":0,"api_base":""}
{"model":"unknown","spend":0.0,"total_tokens":0,"api_base":"wss://api.openai.com/v1/responses"}
{"model":"openai/gpt-realtime-2.1-mini","spend":0.0003534,"total_tokens":194,"api_base":""}

Observations from the run:

• Handshake auth already refuses restricted identities when the URL names a model; this PR adds the routes that do not
• Relayed passthrough sessions still log $0 and model unknown; this PR leaves that alone (LIT-7014)
• A DB opt-in reaches the other process within one reload interval; no restart needed

Type

🐛 Bug Fix

Caveats (if any)

Severe

• Deliberate behavior change: passthrough WebSocket callers are refused until the admin opts in
  • Who breaks: anyone on /openai_passthrough/<ws path> or /openai/<ws path> over WebSocket since v1.99.0
  • Turn back on with general_settings.enable_openai_websocket_passthrough: true in YAML, or through POST /config/field/update when store_model_in_db is on (not from the Admin UI general settings page, which only lists the alerting fields); the error event says so
  • Why off by default: the relay runs on the proxy's own credential and cannot enforce a key's model list, budget, or guardrails
  • The proxy's own /v1/realtime, /openai/v1/realtime, /realtime, and /v1/responses WebSocket routes are unchanged
• Once enabled, identities that used to be relayed on /openai/v1/responses despite a team, membership, user, or project model list are now refused there too
  • Who breaks: only callers who were getting past a model restriction that every HTTP route already enforces
  • Turn back on by giving that identity no model restriction at any level, which is what the relay needs to be safe

Medium

• Once enabled, a user lookup that fails for any reason other than a missing row (say the database is unreachable) refuses the socket, where HTTP routes carry on with the limits recorded on the token
  • Chosen because the relay cannot enforce anything after the handshake, so an unknown restriction has to close it
• Once enabled, an identity whose team_id has no team row fails the handshake with 404
  • Same answer HTTP model-access enforcement gives for a provably missing team; custom-auth deployments that skip common checks over HTTP meet it here
• Once enabled, frames from unrestricted keys are still relayed unread: no per-model access checks, spend stays $0
  • Follow-up tracked in LIT-7014
• The refusal is sent after the handshake, not as an HTTP status, so clients see a close code
  • Chosen so browser and SDK clients get a readable reason; a 403 handshake gives them nothing
• A YAML enable_openai_websocket_passthrough value always wins over the DB one, like every other DB-overlaid general setting
  • An admin who set it in YAML cannot flip it from the API; the same rule already applies to the other overlaid keys

Low

• A pre-accept 403 on the realtime passthrough route (the key cannot use the model in the query string) logs an ASGI RuntimeError after the close, on the merge base and on this branch alike
• The earlier tip a0b2e7f inherited three reds from litellm_internal_staging (integrations / Run tests on the datadog span test, frontend-lint on the dashboard's local/no-large-inline-object-arg budget, and CircleCI proxy_store_model_in_db_tests::test_chat_completion_bad_model_with_spend_logs on staging's scheduled pipelines). 5c80e30 merges the staging fixes for the first two, and both are green at this tip. The merge also tripped two lint ceilings staging had ratcheted down in the meantime (LIT002 on the two dict literals of the refusal frame, and RUF100 on a # noqa: UP037 that stopped being needed once Literal is imported directly), fixed in aca1c54 by building the frame from a TypedDict and dropping the suffix. At the moment auto-merge fired, 26 of the 29 CircleCI jobs had passed at 5c80e30 and using_litellm_on_windows, e2e_ui_testing and proxy_multi_instance_tests were still running; none of the three is a required context

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• a0b2e7f passes /live-pr-risk (every dependent path is listed under Screenshots / Proof of Fix and Caveats; the one CircleCI red at this tip is staging's own, per the Low caveat above)

• 5c80e30 passes /live-pr-risk (aca1c54 only changes how the refusal frame is built, from two dict literals to a TypedDict-annotated literal, and drops a # noqa: UP037 that no longer suppressed anything; the frame's JSON bytes on the wire are identical, which the default and enabled legs at 5c80e30 observe on both passthrough prefixes and for the model-restricted refusal, so every client reading that frame sees what it saw at a0b2e7f; the base merge itself had no conflicts)

---

Note

High Risk
Changes default proxy behavior and auth for a credential-relay path that bypasses normal model and spend enforcement; misconfiguration or the opt-in flag could expose the provider key or break existing WebSocket clients.

Overview
OpenAI WebSocket passthrough is off by default and only serves /openai/* and /openai_passthrough/* when general_settings.enable_openai_websocket_passthrough is true (YAML or DB via store_model_in_db; YAML overrides DB). Refused connections accept the handshake, send one OpenAI-style error frame, then close with code 1008.

When enabled, the route blocks any identity with a model restriction at any enforced level (key, team, team membership, user, project)—not just fields on the token—via new enforced_model_allowlists(). Missing user rows are treated as unrestricted (e.g. master key on a fresh DB); provable absence uses UserNotFoundError instead of a bare exception.

Managed /v1/realtime and other non-passthrough WebSocket paths are unchanged.

^{Reviewed by Cursor Bugbot for commit 5c80e30. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_gate_openai_ws_passthrough (5c80e30) with litellm_internal_staging (29ac88e)}

[greptile-apps]

Greptile Summary

This PR adds explicit administrative opt-in and effective model-scope admission checks for OpenAI WebSocket passthrough routes.

• Keeps passthrough disabled by default and returns a readable WebSocket error before closing refused connections.
• Supports YAML and database-backed configuration while preserving YAML precedence.
• Refuses identities carrying model restrictions at key, team, membership, user, or project scope.
• Distinguishes a genuinely missing user row from database or cache failures so admission does not fail open.
• Adds focused tests and generated schema support for the setting.

Confidence Score: 5/5

The PR appears safe to merge; no new actionable issue or outstanding previous finding remains.

The database configuration overlay now activates the opt-in when YAML does not override it, and lookup failures remain fail-closed while genuinely absent user rows are handled explicitly. The other previous findings were resolved after correct disputes or were manually resolved.

Important Files Changed

Filename	Overview
litellm/proxy/auth/auth_checks.py	Resolves all enforced model allowlists and distinguishes missing users from unreadable user records.
litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py	Gates OpenAI WebSocket passthrough and sends typed error frames for disabled or restricted connections.
litellm/proxy/proxy_server.py	Applies the database-backed passthrough setting while retaining YAML precedence.
litellm/proxy/_types.py	Adds the optional general setting to the proxy configuration model.
tests/test_litellm/proxy/test_openai_ws_passthrough_routes.py	Covers disabled, enabled, and model-restricted WebSocket admission behavior.
tests/test_litellm/proxy/auth/test_auth_checks.py	Covers effective allowlist resolution and missing-versus-failed user lookup behavior.

_{Reviews (7): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile}

[greptile-apps]

Database opt-in is ignored

The new field allows /config/update to persist this setting and report a successful configuration update, but _update_general_settings never copies it from the database into the live general_settings mapping. An administrator who enables passthrough through the supported management API will therefore continue receiving disabled errors on every connection. The setting needs to participate in database-overlay reconciliation with the intended YAML precedence.

Knowledge Base Used: Proxy configuration and deployment

[codecov]

Codecov Report

❌ Patch coverage is 96.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ass_through_endpoints/llm_passthrough_endpoints.py	94.23%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Team restrictions skipped without database
  • The no-prisma branch of enforced_model_allowlists now also returns the token's team_models list, so JWT/no-DB identities with a team model allowlist stay restricted on the OpenAI websocket passthrough.

Or push these changes by commenting:

@cursor push 8f67fe512f

Preview (8f67fe512f)

diff --git a/litellm/proxy/auth/auth_checks.py b/litellm/proxy/auth/auth_checks.py
--- a/litellm/proxy/auth/auth_checks.py
+++ b/litellm/proxy/auth/auth_checks.py
@@ -4164,7 +4164,7 @@
     """One model allowlist per level that ``common_checks`` enforces on a request from this identity."""
     key_models: Final = _resolve_key_models_for_auth_check(valid_token=valid_token)
     if prisma_client is None:
-        return (key_models,)
+        return (key_models, tuple(valid_token.team_models or ()))
     team_object: Final = (
         None
         if valid_token.team_id is None

diff --git a/tests/test_litellm/proxy/auth/test_auth_checks.py b/tests/test_litellm/proxy/auth/test_auth_checks.py
--- a/tests/test_litellm/proxy/auth/test_auth_checks.py
+++ b/tests/test_litellm/proxy/auth/test_auth_checks.py
@@ -7397,6 +7397,18 @@
         user_api_key_cache=cache,
         proxy_logging_obj=proxy_logging_obj,
     )
+    without_database_team_only = await enforced_model_allowlists(
+        valid_token=UserAPIKeyAuth(
+            token="hashed-fake",
+            models=[],
+            team_models=["gpt-4o"],
+            user_id="user-fake",
+            team_id="team-fake",
+        ),
+        prisma_client=None,
+        user_api_key_cache=cache,
+        proxy_logging_obj=proxy_logging_obj,
+    )
 
     assert [list(scope) for scope in team_scoped] == [
         ["gpt-4o", "gpt-4o-mini"],
@@ -7406,4 +7418,5 @@
         ["gpt-4.1"],
     ]
     assert [list(scope) for scope in personal] == [[], [], [], ["o3"], []]
-    assert [list(scope) for scope in without_database] == [["gpt-4o"]]
+    assert [list(scope) for scope in without_database] == [["gpt-4o"], []]
+    assert [list(scope) for scope in without_database_team_only] == [[], ["gpt-4o"]]

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 5c80e30. Configure here.}