fix(passthrough): resolve vertex live credentials from db model deployments

[mateo-berri]

TLDR

Problem this solves:

• Live passthrough ignores Vertex credentials stored in the DB
• Sessions die with no close code or reason
• Google's own rejection reaches the client as a clean 1000
• Bare model ids and gateway aliases are rejected by Vertex

How it solves it:

• Resolve credentials from pass-through-enabled router deployments
• Close 1011 naming both ways to configure credentials
• Relay the upstream close code and reason through
• Rewrite the setup frame's model to Vertex's resource path

User Flow

Before: a developer whose Live app connects through the gateway is dropped the moment the session starts, with nothing saying why

1. The proxy admin saves the Vertex service-account credential with POST https://litellm-domain/credentials, sending vertex_project, vertex_location and the key JSON, and gets {"success":true} back
2. The admin registers the Live model with POST https://litellm-domain/model/new, sending litellm_credential_name and use_in_pass_through: true, and gets 200 with the new model id
3. The developer checks GET https://litellm-domain/v1/models, sees the model listed, and takes it as ready to use
4. The developer's app opens a WebSocket to wss://litellm-domain/vertex_ai/live with its virtual key in the Authorization header, and the handshake returns 101
5. The app sends its first setup frame naming the model
6. Seconds later the socket closes with no session ever started: 1011 "Vertex AI authentication failed" where the gateway host carries no Google credentials of its own, or a bare 1000 with an empty reason where it does, which reads exactly like a normal end of session
7. The developer tries a model Vertex will not serve and gets that same empty 1000, so nothing separates a bad model from broken credentials from a finished session
8. The one thing that yields a working session is appending ?vertex_project= and ?vertex_location= to the URL, which the Live SDK never sends

After: the same app gets a real Live session on the credential the admin already saved, and anything Vertex turns down comes back in Google's own words

1. The proxy admin saves the Vertex service-account credential with POST https://litellm-domain/credentials, sending vertex_project, vertex_location and the key JSON, and gets {"success":true} back
2. The admin registers the Live model with POST https://litellm-domain/model/new, sending litellm_credential_name and use_in_pass_through: true, and gets 200 with the new model id
3. The developer checks GET https://litellm-domain/v1/models, sees the model listed, and takes it as ready to use
4. The developer's app opens a WebSocket to wss://litellm-domain/vertex_ai/live with its virtual key in the Authorization header, and the handshake returns 101
5. The app sends its first setup frame naming the model
6. The gateway answers setupComplete with a session id, and the app's first turn comes back as model text followed by turnComplete carrying real token counts
7. The developer tries a model Vertex will not serve and the socket closes 1008 in Google's own words, "Publisher model projects/.../not-a-real-model was not found or ", so the mistake is obvious
8. Naming the model as a bare id, as the gateway's own model name, or as the full projects/.../publishers/google/models/... path all work the same, with no query parameters on the URL

Relevant issues

Linear ticket

Resolves LIT-5868

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup, run once for both legs. $PROJECT stands in for a real GCP project id and sa.json for its service-account key

1. Write the config the customer runs, which stores everything in the DB and sets no default_vertex_config:

cat > config_a.yaml <<'YAML'
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  store_model_in_db: true
litellm_settings:
  set_verbose: true
YAML

2. Boot the proxy on a free port, with no GOOGLE_APPLICATION_CREDENTIALS, VERTEXAI_* or DEFAULT_VERTEXAI_* anywhere in its environment:

export PORT=38211 LITELLM_MASTER_KEY=sk-1234 PROJECT=<gcp project id>
export DATABASE_URL=postgresql://<user>@127.0.0.1:5432/litellm_lit5868 STORE_MODEL_IN_DB=True
python litellm/proxy/proxy_cli.py --config config_a.yaml --port $PORT --detailed_debug --use_v2_migration_resolver

3. Store the Vertex credential, register the Live model against it for pass-through, and mint a virtual key:

curl -s -X POST http://127.0.0.1:$PORT/credentials -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
  -d "{\"credential_name\":\"GCP global\",\"credential_info\":{},\"credential_values\":{\"vertex_project\":\"$PROJECT\",\"vertex_location\":\"global\",\"vertex_credentials\":$(jq -Rs . < sa.json)}}"
{"success":true}

curl -s -X POST http://127.0.0.1:$PORT/model/new -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
  -d '{"model_name":"gemini-live","litellm_params":{"model":"vertex_ai/gemini-live-2.5-flash","litellm_credential_name":"GCP global","use_in_pass_through":true}}'

export KEY=$(curl -s -X POST http://127.0.0.1:$PORT/key/generate -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d '{}' | jq -r .key)

curl -s http://127.0.0.1:$PORT/v1/models -H "Authorization: Bearer $KEY"
{"data":[{"id":"gemini-live","object":"model","created":1677610602,"owned_by":"openai"}],"object":"list"}

4. The real client is the Google Live SDK, which the customer's agent framework drives. Pointing its base_url at the gateway with no project or location puts it in the API-gateway mode it documents in live.py: it takes the URL verbatim, sends the model name exactly as the caller typed it, and authenticates with the custom header. It only speaks wss://, so a self-signed TLS terminator sits in front of the local proxy and the client trusts that certificate:

openssl req -x509 -newkey rsa:2048 -keyout tls_key.pem -out tls_cert.pem -days 2 -nodes \
  -subj "/CN=127.0.0.1" -addext "subjectAltName=IP:127.0.0.1"
python tls_front.py 38212 $PORT   # TLS on 38212, plain TCP to the proxy

cat > adk_live_probe.py <<'PY'
import asyncio
import os
import ssl
import sys

import websockets.exceptions
from google import genai
from google.genai import types

PORT = os.environ["PORT"]
KEY = os.environ["KEY"]
MODEL = sys.argv[1] if len(sys.argv) > 1 else "gemini-live-2.5-flash"


async def main() -> None:
    client = genai.Client(
        vertexai=True,
        http_options=types.HttpOptions(
            base_url=f"wss://127.0.0.1:{PORT}/vertex_ai/live",
            headers={"Authorization": f"Bearer {KEY}"},
            async_client_args={"ssl": ssl.create_default_context(cafile=os.environ["CAFILE"])},
        ),
    )
    api = client._api_client
    print(f"CLIENT vertexai=True project={api.project} location={api.location} url={api.custom_base_url}")
    print(f"CLIENT model argument {MODEL!r}")
    config = types.LiveConnectConfig(response_modalities=["TEXT"])
    try:
        async with client.aio.live.connect(model=MODEL, config=config) as session:
            print("SESSION OPEN (setupComplete received)")
            await session.send_client_content(
                turns=types.Content(role="user", parts=[types.Part(text="Say hello in one word")]),
                turn_complete=True,
            )
            print("SENT clientContent")
            async for message in session.receive():
                sc = message.server_content
                if sc is not None and sc.model_turn is not None:
                    text = "".join(p.text or "" for p in (sc.model_turn.parts or []))
                    if text:
                        print(f"RECV model text {text!r}")
                if message.usage_metadata is not None:
                    print(f"RECV usage tokens={message.usage_metadata.total_token_count}")
                if sc is not None and sc.turn_complete:
                    print("RECV turnComplete")
                    return
    except websockets.exceptions.ConnectionClosed as exc:
        rcvd = exc.rcvd
        code = rcvd.code if rcvd else None
        reason = repr(rcvd.reason) if rcvd else None
        print(f"CLOSED code={code} reason={reason}")
    except Exception as exc:
        print(f"ERROR {type(exc).__name__}: {exc}")


asyncio.run(main())
PY

5. The SDK only ever sends one shape of model name, so a second probe drives the raw socket to cover the other shapes a client can send. It opens the socket, sends one setup frame with the model given on the command line, then one turn, and prints every frame or the close it gets back:

cat > live_probe.py <<'PY'
import asyncio
import json
import os
import sys

import websockets

URL = f"ws://127.0.0.1:{os.environ['PORT']}/vertex_ai/live"


async def main() -> None:
    setup = json.loads(sys.argv[1])
    async with websockets.connect(URL, additional_headers={"Authorization": f"Bearer {os.environ['KEY']}"}) as ws:
        print("UPGRADE OK (101)")
        await ws.send(json.dumps({"setup": setup}))
        print(f"SENT {json.dumps({'setup': setup})}")
        while True:
            try:
                frame = json.loads(await asyncio.wait_for(ws.recv(), timeout=20))
            except websockets.exceptions.ConnectionClosed as e:
                print(f"CLOSED code={e.rcvd.code} reason={e.rcvd.reason!r}")
                return
            except asyncio.TimeoutError:
                print("TIMEOUT waiting for a frame")
                return
            if "setupComplete" in frame:
                print(f"RECV {json.dumps(frame)}")
                await ws.send(json.dumps({"clientContent": {"turns": [{"role": "user", "parts": [{"text": "Say hello in one word"}]}], "turnComplete": True}}))
                print("SENT clientContent")
                continue
            server_content = frame.get("serverContent", {})
            text = "".join(p.get("text", "") for p in server_content.get("modelTurn", {}).get("parts", []))
            if text:
                print(f"RECV model text {text!r}")
            if server_content.get("turnComplete"):
                print(f"RECV turnComplete, usage {json.dumps(frame.get('usageMetadata', {}))}")
                return


asyncio.run(main())
PY

The last case needs credential resolution to succeed so the failure comes from Vertex rather than the gateway, so it runs against a second config that adds the documented default_vertex_config:

cat > config_b.yaml <<'YAML'
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  store_model_in_db: true
litellm_settings:
  set_verbose: true
default_vertex_config:
  vertex_project: "<gcp project id>"
  vertex_location: "global"
  vertex_credentials: "sa.json"
YAML

Before (5290150)

Google Live SDK, the client the customer runs

1. PORT=38212 CAFILE=tls_cert.pem python adk_live_probe.py gemini-live-2.5-flash
2. The SDK reaches the gateway and the session is killed before it ever starts:

CLIENT vertexai=True project=None location=None url=wss://127.0.0.1:38212/vertex_ai/live
CLIENT model argument 'gemini-live-2.5-flash'
ERROR APIError: 1011 None. Vertex AI authentication failed

3. grep -c "GCP global" proxy.log prints 0: the stored credential is never consulted

Bare Vertex model id

1. python live_probe.py '{"model":"gemini-live-2.5-flash","generationConfig":{"responseModalities":["TEXT"]}}'
2. The upgrade succeeds and the session then dies with nothing from Vertex:

UPGRADE OK (101)
SENT {"setup": {"model": "gemini-live-2.5-flash", "generationConfig": {"responseModalities": ["TEXT"]}}}
CLOSED code=1011 reason='Vertex AI authentication failed'

Gateway model name

1. python live_probe.py '{"model":"gemini-live","generationConfig":{"responseModalities":["TEXT"]}}'
2. Same close:

UPGRADE OK (101)
SENT {"setup": {"model": "gemini-live", "generationConfig": {"responseModalities": ["TEXT"]}}}
CLOSED code=1011 reason='Vertex AI authentication failed'

Full Vertex resource path

1. python live_probe.py "{\"model\":\"projects/$PROJECT/locations/global/publishers/google/models/gemini-live-2.5-flash\",\"generationConfig\":{\"responseModalities\":[\"TEXT\"]}}"
2. Same close, so no way of naming the model helps:

UPGRADE OK (101)
SENT {"setup": {"model": "projects/$PROJECT/locations/global/publishers/google/models/gemini-live-2.5-flash", "generationConfig": {"responseModalities": ["TEXT"]}}}
CLOSED code=1011 reason='Vertex AI authentication failed'

Model Vertex does not serve

1. Restart the proxy on config_b.yaml, so credentials resolve and the rejection comes from Google
2. python live_probe.py "{\"model\":\"projects/$PROJECT/locations/global/publishers/google/models/not-a-real-model\",\"generationConfig\":{\"responseModalities\":[\"TEXT\"]}}"
3. Google's 1008 and its message are dropped and the client is told the session simply ended:

UPGRADE OK (101)
SENT {"setup": {"model": "projects/$PROJECT/locations/global/publishers/google/models/not-a-real-model", "generationConfig": {"responseModalities": ["TEXT"]}}}
CLOSED code=1000 reason=''

After (4f04e59)

Google Live SDK, the client the customer runs

1. PORT=38212 CAFILE=tls_cert.pem python adk_live_probe.py gemini-live-2.5-flash
2. The SDK gets a real Live session on the credential the admin saved, and a full turn comes back:

CLIENT vertexai=True project=None location=None url=wss://127.0.0.1:38212/vertex_ai/live
CLIENT model argument 'gemini-live-2.5-flash'
SESSION OPEN (setupComplete received)
SENT clientContent
RECV model text 'Hello'
RECV usage tokens=6
RECV turnComplete

3. PORT=38212 CAFILE=tls_cert.pem python adk_live_probe.py gemini-live does the same on the gateway's own model name:

CLIENT vertexai=True project=None location=None url=wss://127.0.0.1:38212/vertex_ai/live
CLIENT model argument 'gemini-live'
SESSION OPEN (setupComplete received)
SENT clientContent
RECV model text 'Hello'
RECV usage tokens=6
RECV turnComplete

Bare Vertex model id

1. python live_probe.py '{"model":"gemini-live-2.5-flash","generationConfig":{"responseModalities":["TEXT"]}}'
2. The session runs on the stored credential and completes a real turn:

UPGRADE OK (101)
SENT {"setup": {"model": "gemini-live-2.5-flash", "generationConfig": {"responseModalities": ["TEXT"]}}}
RECV {"setupComplete": {"sessionId": "9bb0932e-5bb5-4236-bd0b-6f8533f672d2"}}
SENT clientContent
RECV model text 'Hello'
RECV turnComplete, usage {"promptTokenCount": 5, "candidatesTokenCount": 1, "totalTokenCount": 6, "promptTokensDetails": [{"modality": "TEXT", "tokenCount": 5}], "candidatesTokensDetails": [{"modality": "TEXT", "tokenCount": 1}]}

Gateway model name

1. python live_probe.py '{"model":"gemini-live","generationConfig":{"responseModalities":["TEXT"]}}'
2. The gateway's own model name resolves to the deployment behind it and the turn completes:

UPGRADE OK (101)
SENT {"setup": {"model": "gemini-live", "generationConfig": {"responseModalities": ["TEXT"]}}}
RECV {"setupComplete": {"sessionId": "4b229967-601f-4435-9160-a702dcdc0c71"}}
SENT clientContent
RECV model text 'Hello'
RECV turnComplete, usage {"promptTokenCount": 5, "candidatesTokenCount": 1, "totalTokenCount": 6, "promptTokensDetails": [{"modality": "TEXT", "tokenCount": 5}], "candidatesTokensDetails": [{"modality": "TEXT", "tokenCount": 1}]}

Full Vertex resource path

1. python live_probe.py "{\"model\":\"projects/$PROJECT/locations/global/publishers/google/models/gemini-live-2.5-flash\",\"generationConfig\":{\"responseModalities\":[\"TEXT\"]}}"
2. A client that already sends the full path is passed through untouched:

UPGRADE OK (101)
SENT {"setup": {"model": "projects/$PROJECT/locations/global/publishers/google/models/gemini-live-2.5-flash", "generationConfig": {"responseModalities": ["TEXT"]}}}
RECV {"setupComplete": {"sessionId": "54969701-94e1-46aa-a3a9-1d9f8a245ce0"}}
SENT clientContent
RECV model text 'Hello'
RECV turnComplete, usage {"promptTokenCount": 5, "candidatesTokenCount": 1, "totalTokenCount": 6, "promptTokensDetails": [{"modality": "TEXT", "tokenCount": 5}], "candidatesTokensDetails": [{"modality": "TEXT", "tokenCount": 1}]}

Model Vertex does not serve

1. Restart the proxy on config_b.yaml, so credentials resolve and the rejection comes from Google
2. python live_probe.py "{\"model\":\"projects/$PROJECT/locations/global/publishers/google/models/not-a-real-model\",\"generationConfig\":{\"responseModalities\":[\"TEXT\"]}}"
3. Google's own close code and message reach the client, cut only by the 123-byte limit the WebSocket close frame imposes, exactly as a direct connection to Vertex reports it:

UPGRADE OK (101)
SENT {"setup": {"model": "projects/$PROJECT/locations/global/publishers/google/models/not-a-real-model", "generationConfig": {"responseModalities": ["TEXT"]}}}
CLOSED code=1008 reason='Publisher model `projects/$PROJECT/locations/global/publishers/google/models/not-a-real-model` was not found or '

Type

🐛 Bug Fix

Caveats (if any)

• default_vertex_config and the DEFAULT_VERTEXAI_* env vars still outrank the DB entries, so a global default keeps winning where one is set
• Needs ?model=, default_vertex_config, or one project, location, and credential across the pass-through deployments, since it will not guess between them
• Close reasons are cut to the 123-byte WebSocket limit
• The close relay is shared, so the OpenAI WebSocket passthrough surfaces upstream close codes too, matching a direct connection

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 4f04e59 passes /live-pr-risk

[greptile-apps]

Greptile Summary

The PR enables Vertex Live passthrough to resolve DB-backed deployment credentials, normalizes setup-frame model names into Vertex resource paths, and relays meaningful upstream WebSocket close details.

• Adds credential resolution from pass-through-enabled router deployments while refusing ambiguous no-model selections.
• Rewrites bare model IDs and gateway aliases using the resolved Vertex project and location.
• Preserves valid upstream close codes and UTF-8-safe close reasons within the WebSocket payload limit.
• Adds focused tests for credential identity, model rewriting, and close propagation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/pass_through_endpoints/passthrough_endpoint_router.py	Resolves Vertex credentials from eligible deployments and now rejects ambiguity across projects, locations, and service-account identities.
litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py	Integrates deployment credential resolution and setup-model resource-path rewriting into the Vertex Live route.
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py	Adds optional setup-frame rewriting and relays valid upstream WebSocket close information without double-closing the client.
litellm/constants.py	Defines the RFC 6455 close-reason byte limit used by the shared WebSocket relay.
tests/test_litellm/proxy/pass_through_endpoints/test_llm_pass_through_endpoints.py	Covers DB credential use, model normalization, default precedence, and actionable authentication failures.
tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py	Covers setup-frame forwarding and valid, invalid, normal, and truncated upstream close behavior.
tests/test_litellm/proxy/pass_through_endpoints/test_passthrough_endpoint_router.py	Covers named and inline credentials, model hints, dict credentials, and ambiguous deployment identities.

_{Reviews (4): Last reviewed commit: "fix: harden vertex live passthrough agai..." | Re-trigger Greptile}

[veria-ai]

Low: Upstream resource details disclosed

An authenticated caller can submit an invalid setup model and receive the provider's raw close reason, which can contain the configured Vertex project and full upstream resource path. Keep the raw reason in server logs and return a stable client-safe message instead.

Suggested change

	reason=_truncated_close_reason(upstream_close.reason),
	reason="Upstream WebSocket closed the connection",

[mateo-berri]

Relaying Google's reason is the fix here. The same project and path already reach callers through the HTTP vertex_ai passthrough and chat errors

[veria-ai]

Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.

[veria-ai]

PR overview

This pull request updates passthrough handling so Vertex live credentials are resolved from database-backed model deployments. The change affects the Vertex live connection setup and upstream WebSocket handling.

One low-impact disclosure remains open: an authenticated caller using an invalid setup model can receive the provider’s raw WebSocket close reason, potentially revealing the configured Vertex project and full upstream resource path. No issues have yet been addressed, but the exposure is limited to authenticated callers and upstream resource metadata.

Open issues (1)

• Low: Upstream resource details disclosed — litellm/proxy/pass_through_endpoints/pass_through_endpoints.py:2269

Fixed/addressed: 0 · PR risk: 4/10

[codecov]

Codecov Report

❌ Patch coverage is 92.70073% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...y/pass_through_endpoints/pass_through_endpoints.py	83.33%	8 Missing ⚠️
...ass_through_endpoints/llm_passthrough_endpoints.py	96.15%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Setup rewrite keeps model prefixes
  • Extended _resolve_alias_to_upstream_model to also strip a leading 'models/' (Gemini Live SDK) and any '/' prefix (LiteLLM id) in addition to 'publishers/google/models/', so every common client naming lands as a bare id under 'publishers/google/models/{id}'.
• ✅ Fixed: Dict Vertex credentials become unusable
  • Added _get_vertex_credentials_value so named-credential lookups accept dicts (validated via TypeAdapter[dict[str,str]]) and dropped the str() wrapper in the live websocket path so a dict service-account is forwarded unchanged to _ensure_access_token_async.

Or push these changes by commenting:

@cursor push 33ff3afa16

Preview (33ff3afa16)

diff --git a/litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py b/litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py
--- a/litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py
+++ b/litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py
@@ -2438,30 +2438,37 @@
         aliased: Final = _resolve_alias_to_upstream_model(setup_model, llm_router)
         return (
             f"projects/{vertex_project}/locations/{vertex_location}/"
-            f"{VERTEX_PUBLISHER_MODEL_PREFIX}{aliased.removeprefix(VERTEX_PUBLISHER_MODEL_PREFIX)}"
+            f"{VERTEX_PUBLISHER_MODEL_PREFIX}{aliased}"
         )
 
     return rewrite
 
 
 def _resolve_alias_to_upstream_model(setup_model: str, llm_router: "Router | None") -> str:
-    if llm_router is None:
-        return setup_model
-    upstream: Final = next(
-        (
-            deployment["litellm_params"]["model"]
-            for deployment in (llm_router.get_model_list() or ())
-            if deployment.get("model_name") == setup_model
-        ),
-        None,
+    """
+    Return the bare Vertex model id for ``setup_model``, resolving router aliases and stripping any
+    ``publishers/google/models/``, ``models/`` (Gemini Live SDK), or ``<provider>/`` (LiteLLM id) prefix
+    so the caller can safely paste it into ``publishers/google/models/{id}``
+    """
+    upstream: Final = (
+        next(
+            (
+                deployment["litellm_params"]["model"]
+                for deployment in (llm_router.get_model_list() or ())
+                if deployment.get("model_name") == setup_model
+            ),
+            None,
+        )
+        if llm_router is not None
+        else None
     )
-    if upstream is None:
-        return setup_model
+    resolved: Final = upstream if upstream is not None else setup_model
+    without_publisher: Final = resolved.removeprefix(VERTEX_PUBLISHER_MODEL_PREFIX).removeprefix("models/")
     try:
-        _, provider, _, _ = litellm.get_llm_provider(model=upstream)
+        _, provider, _, _ = litellm.get_llm_provider(model=without_publisher)
     except litellm.exceptions.BadRequestError:
-        return upstream
-    return upstream.removeprefix(f"{provider}/")
+        return without_publisher
+    return without_publisher.removeprefix(f"{provider}/")
 
 
 async def vertex_ai_live_websocket_passthrough(
@@ -2500,9 +2507,7 @@
         vertex_credentials_config.vertex_location if vertex_credentials_config is not None else None
     )
     credentials_value: Final = (
-        str(vertex_credentials_config.vertex_credentials)
-        if vertex_credentials_config is not None and vertex_credentials_config.vertex_credentials is not None
-        else None
+        vertex_credentials_config.vertex_credentials if vertex_credentials_config is not None else None
     )
 
     try:

diff --git a/litellm/proxy/pass_through_endpoints/passthrough_endpoint_router.py b/litellm/proxy/pass_through_endpoints/passthrough_endpoint_router.py
--- a/litellm/proxy/pass_through_endpoints/passthrough_endpoint_router.py
+++ b/litellm/proxy/pass_through_endpoints/passthrough_endpoint_router.py
@@ -1,6 +1,8 @@
 from collections.abc import Callable
 from typing import TYPE_CHECKING, Final
 
+from pydantic import TypeAdapter
+
 import litellm
 from litellm._logging import verbose_router_logger
 from litellm.integrations.vector_store_integrations.vector_store_pre_call_hook import (
@@ -27,6 +29,18 @@
     return value if isinstance(value, str) else None
 
 
+def _get_vertex_credentials_value(
+    values: dict[str, object] | None,
+) -> VERTEX_CREDENTIALS_TYPES | None:
+    """Vertex service-account credentials can be stored as a JSON string or a parsed dict; keep either shape"""
+    value: Final = values.get("vertex_credentials") if values is not None else None
+    if isinstance(value, str):
+        return value
+    if isinstance(value, dict):
+        return TypeAdapter(dict[str, str]).validate_python(value)
+    return None
+
+
 class PassthroughEndpointRouter:
     """
     Use this class to Get credentials for pass-through endpoints
@@ -172,9 +186,9 @@
         vertex_location: Final = _get_str_value(credential_values, "vertex_location") or litellm_params.get(
             "vertex_location"
         )
-        vertex_credentials: Final = _get_str_value(credential_values, "vertex_credentials") or litellm_params.get(
-            "vertex_credentials"
-        )
+        vertex_credentials: Final = _get_vertex_credentials_value(
+            credential_values
+        ) or litellm_params.get("vertex_credentials")
         if vertex_project is None or vertex_location is None:
             return None
         return VertexPassThroughCredentials(

diff --git a/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py b/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
--- a/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
+++ b/tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
@@ -5161,7 +5161,12 @@
 @pytest.mark.asyncio
 @pytest.mark.parametrize(
     "setup_model",
-    ["gemini-live-2.5-flash", "publishers/google/models/gemini-live-2.5-flash"],
+    [
+        "gemini-live-2.5-flash",
+        "publishers/google/models/gemini-live-2.5-flash",
+        "models/gemini-live-2.5-flash",
+        "vertex_ai/gemini-live-2.5-flash",
+    ],
 )
 async def test_websocket_passthrough_rewrites_setup_model_to_full_resource(setup_model):
     sent_frame = await _run_setup_rewrite_passthrough(setup_model, llm_router=None)

diff --git a/tests/test_litellm/proxy/pass_through_endpoints/test_passthrough_endpoint_router.py b/tests/test_litellm/proxy/pass_through_endpoints/test_passthrough_endpoint_router.py
--- a/tests/test_litellm/proxy/pass_through_endpoints/test_passthrough_endpoint_router.py
+++ b/tests/test_litellm/proxy/pass_through_endpoints/test_passthrough_endpoint_router.py
@@ -237,6 +237,56 @@
     assert resolved.vertex_credentials == '{"type": "service_account", "project_id": "proj-inline"}'
 
 
+def test_vertex_deployment_preserves_dict_credentials_from_named_credential():
+    service_account = {"type": "service_account", "project_id": "proj-db"}
+    CredentialAccessor.upsert_credentials(
+        [
+            _vertex_credential(
+                "cred_gcp_dict",
+                {
+                    "vertex_project": "proj-db",
+                    "vertex_location": "global",
+                    "vertex_credentials": service_account,
+                },
+            )
+        ]
+    )
+    llm_router = litellm.Router(
+        model_list=[
+            _vertex_deployment(
+                "gemini-live", "vertex_ai/gemini-live-2.5-flash", litellm_credential_name="cred_gcp_dict"
+            )
+        ]
+    )
+    passthrough_router = _passthrough_router(llm_router)
+
+    resolved = passthrough_router.get_vertex_credentials_from_router_deployments(model=None)
+
+    assert resolved is not None
+    assert resolved.vertex_credentials == service_account
+
+
+def test_vertex_deployment_preserves_dict_credentials_from_inline_litellm_params():
+    service_account = {"type": "service_account", "project_id": "proj-inline"}
+    llm_router = litellm.Router(
+        model_list=[
+            _vertex_deployment(
+                "gemini-live",
+                "vertex_ai/gemini-live-2.5-flash",
+                vertex_project="proj-inline",
+                vertex_location="us-east4",
+                vertex_credentials=service_account,
+            )
+        ]
+    )
+    passthrough_router = _passthrough_router(llm_router)
+
+    resolved = passthrough_router.get_vertex_credentials_from_router_deployments(model=None)
+
+    assert resolved is not None
+    assert resolved.vertex_credentials == service_account
+
+
 def _two_vertex_deployments_router() -> litellm.Router:
     return litellm.Router(
         model_list=[

_{You can send follow-ups to the cloud agent here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_vertex_live_db_credentials (4f04e59) with litellm_internal_staging (6fcdea0)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (952c6d5) during the generation of this report, so 6fcdea0 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4f04e59. Configure here.}