fix: support served_model_name for Baseten dedicated deployments

[AlexKer]

Summary

• Baseten dedicated deployments use an 8-char deployment ID for URL routing, but the
  vLLM server may expect a different model name in the request body (e.g.
  baseten-hosted/zai-org/GLM-5 vs 1234abcd)
• Previously there was no way to get both the correct URL and the correct request body
  model name — users would either get routing errors or 404s
• Added served_model_name param in litellm_params that overrides the model name sent in
  the request body while keeping the deployment ID for URL construction

Test plan

• Unit tests for transform_request with and without served_model_name
• Existing routing and param mapping tests still pass
• Manual test with a Baseten dedicated deployment using a custom served_model_name

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 17, 2026 7:56pm

[CLAassistant]

All committers have signed the CLA.

[AlexKer]

@greptileai

[greptile-apps]

Greptile Summary

This PR adds a served_model_name litellm param that lets users decouple the Baseten deployment ID (used for URL routing) from the model name sent in the request body — solving a real pain point for dedicated deployments where the vLLM server is configured with a custom served-model-name.

Key changes:

• BasetenConfig.transform_request is overridden to substitute served_model_name (from litellm_params) for the deployment-ID model name before the request body is built.
• served_model_name is declared in both GenericLiteLLMParams (Pydantic model) and LiteLLMParamsTypedDict (TypedDict), giving users IDE autocompletion and type-checking support.
• Documentation is updated with a dedicated "Dedicated Deployment" proxy example.
• Unit tests cover the new override logic, the passthrough case, and URL routing — all without real network calls.

One naming-convention concern: every other provider-specific field in router.py is prefixed with the provider name (e.g. vertex_project, aws_region_name). served_model_name carries a ## BASETEN ## label but has no baseten_ prefix, which breaks the established convention and risks silent conflicts if another vLLM-based provider adds a field with the same name in the future.

Confidence Score: 4/5

• Safe to merge with a minor naming convention concern; no logic bugs or breaking changes identified.
• Implementation is correct and well-tested. The transform_request override is clean, async requests are handled correctly via the parent's else branch (which delegates to self.transform_request), and served_model_name does not leak into optional_params. The only issue is that the new field is named served_model_name without a baseten_ prefix, inconsistent with every other provider-specific param in the same file.
• litellm/types/router.py — the served_model_name field naming convention.

Important Files Changed

Filename	Overview
litellm/llms/baseten/chat.py	Adds transform_request override that reads served_model_name from litellm_params and substitutes it for the deployment-ID model name before delegating to the parent. Logic is correct and isolated to this provider.
litellm/types/router.py	Adds served_model_name to GenericLiteLLMParams and LiteLLMParamsTypedDict. Functionally correct but the field name lacks a baseten_ prefix, inconsistent with every other provider-specific param in this file.
tests/test_litellm/llms/baseten/chat/test_baseten_completions.py	New unit tests cover the happy path (with and without served_model_name) and API-base routing. All tests are pure mocks/unit tests — no real network calls.
docs/my-website/docs/providers/baseten.md	Documentation updated with a new "Dedicated Deployment" proxy section explaining served_model_name usage. Existing Model API example is preserved.

Sequence Diagram

sequenceDiagram
    participant User
    participant LiteLLMProxy
    participant BasetenConfig
    participant OpenAIGPTConfig
    participant BasetenAPI

    User->>LiteLLMProxy: chat.completions.create(model="baseten-model")
    LiteLLMProxy->>BasetenConfig: transform_request(model="wd1lndkw", litellm_params={served_model_name: "baseten-hosted/zai-org/GLM-5"})
    BasetenConfig->>BasetenConfig: served_model_name = litellm_params.get("served_model_name")
    BasetenConfig->>BasetenConfig: model = "baseten-hosted/zai-org/GLM-5"
    BasetenConfig->>OpenAIGPTConfig: super().transform_request(model="baseten-hosted/zai-org/GLM-5", ...)
    OpenAIGPTConfig-->>BasetenConfig: {model: "baseten-hosted/zai-org/GLM-5", messages: [...]}
    BasetenConfig->>BasetenAPI: POST https://model-wd1lndkw.api.baseten.co/.../v1 body={model: "baseten-hosted/zai-org/GLM-5"}
    BasetenAPI-->>User: chat completion response

Loading

_{Last reviewed commit: c461ab2}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing AlexKer:baseten-update (c461ab2) with main (ef9cc33)}

[greptile-apps]

Missing provider prefix on served_model_name

All other provider-specific fields in GenericLiteLLMParams use a provider prefix (e.g. vertex_project, aws_access_key_id, watsonx_region_name). served_model_name carries a ## BASETEN ## comment indicating it is Baseten-specific, yet it has no baseten_ prefix.

This creates two risks:

1. Another provider (e.g. any future vLLM-based integration) could independently define served_model_name with slightly different semantics, causing silent conflicts.
2. Users configuring non-Baseten deployments may accidentally set served_model_name and observe unexpected request-body overrides because BasetenConfig.transform_request checks for this name without verifying the provider.

Consider renaming to baseten_served_model_name (matching the existing convention) and updating the comment in LiteLLMParamsTypedDict (line 351) and baseten/chat.py (line 97) accordingly.

Suggested change

	served_model_name: Optional[str] = None # override model name in request body (e.g. Baseten dedicated deployments)
	served_model_name: Optional[str] = None # override model name in request body (e.g. Baseten dedicated deployments)