feat(messages): passthrough /v1/messages to native endpoints via supported_endpoints

[mateo-berri]

Relevant issues

Generalizes and supersedes #28745, which did the same thing but only for hosted_vllm and only via a disable_anthropic_translation env-var/litellm_params toggle

Linear ticket

Resolves LIT-3750

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

End-to-end on a live DB-less proxy hitting the real DeepSeek Anthropic-compatible endpoint (no mocks). The deployment is provider openai with model_info.supported_endpoints including /v1/messages. The same Anthropic /v1/messages payload (with a cache_control system block, an Anthropic-only feature) is sent to the proxy on both commits. On the base commit the openai provider translates /v1/messages and POSTs to a non-existent https://api.deepseek.com/anthropic/responses, returning 404. On the PR head the raw Anthropic payload is forwarded untranslated to https://api.deepseek.com/anthropic/v1/messages, returning a native Anthropic message response with Anthropic-style usage (including cache_creation_input_tokens)

Config used (DB-less, master_key: sk-1234):

model_list:
  - model_name: deepseek-anthropic-passthrough
    litellm_params:
      model: openai/deepseek-chat
      api_base: https://api.deepseek.com/anthropic
      api_key: os.environ/DEEPSEEK_API_KEY
    model_info:
      supported_endpoints: ["/v1/chat/completions", "/v1/messages"]

Before (base 26ee5dd59)

curl -sS -D - http://localhost:PORT/v1/messages \
  -H "Authorization: Bearer sk-1234" \
  -H "content-type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "deepseek-anthropic-passthrough",
    "max_tokens": 64,
    "system": [{"type":"text","text":"You are concise","cache_control":{"type":"ephemeral"}}],
    "messages": [{"role":"user","content":"Say hi in exactly three words"}]
  }'

HTTP/1.1 404 Not Found
{"error":{"message":"litellm.NotFoundError: NotFoundError: OpenAIException - . Received Model Group=deepseek-anthropic-passthrough\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"404"}}

Proxy debug log for this request shows the translated outbound call going to https://api.deepseek.com/anthropic/responses, which does not exist, hence the 404

After (PR head ab174609)

Same command, same payload:

curl -sS -D - http://localhost:PORT/v1/messages \
  -H "Authorization: Bearer sk-1234" \
  -H "content-type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "deepseek-anthropic-passthrough",
    "max_tokens": 64,
    "system": [{"type":"text","text":"You are concise","cache_control":{"type":"ephemeral"}}],
    "messages": [{"role":"user","content":"Say hi in exactly three words"}]
  }'

HTTP/1.1 200 OK
{"id":"0eb61974-efda-454e-a3f0-3b1f4ad40f88","type":"message","role":"assistant","model":"deepseek-anthropic-passthrough","content":[{"type":"text","text":"Hi, how are you?"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":13,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":6,"service_tier":"standard"}}

The body is genuine native Anthropic format ("type":"message", "role":"assistant", content, stop_reason, and Anthropic-style usage with cache_creation_input_tokens / cache_read_input_tokens / service_tier), not an OpenAI chat.completion. The proxy debug log for this request confirms the call was handled via litellm.anthropic_messages(...) and POSTed to https://api.deepseek.com/anthropic/v1/messages with the cache_control system block forwarded untranslated

Type

🆕 New Feature

Changes

Today the unified /v1/messages proxy endpoint always translates an inbound Anthropic request down to /v1/chat/completions (or the Responses API for openai) whenever the deployment's provider has no native Anthropic-messages config. That translation silently drops Anthropic-only features such as cache_control and thinking. A customer runs OpenAI-compatible model servers (self-hosted vLLM, DeepSeek's Anthropic-compatible endpoint, and similar) that also natively expose the Anthropic /v1/messages API, and wants LiteLLM to forward the raw Anthropic payload through untranslated, while keeping the provider as openai so /v1/chat/completions to the same deployment stays native

This adds a per-endpoint, per-deployment opt-in. Declare supported_endpoints under model_info and include "/v1/messages":

model_list:
  - model_name: my-vllm-model
    litellm_params:
      model: openai/some-model
      api_base: https://host/v1
      api_key: os.environ/SOME_KEY
    model_info:
      supported_endpoints: ["/v1/chat/completions", "/v1/messages"]

When /v1/messages is present, the gate routes to a new generic, provider-agnostic OpenAILikeAnthropicMessagesConfig (a subclass of the native AnthropicMessagesConfig, mirroring the existing DeepSeek pattern) that POSTs the Anthropic-shaped body to {api_base}/v1/messages with Authorization: Bearer {api_key}, a default anthropic-version, and content-type: application/json, inheriting the proven Anthropic request transformation, response parsing, and streaming from the parent native config. Because it delegates request shaping to the parent, it also forwards anthropic-beta headers for Anthropic features like context management and fast mode (merging with any caller-supplied anthropic-beta) rather than silently dropping them, which is the same class of feature-loss the translation path causes. Without the opt-in, behavior is unchanged and the request is still translated. /v1/chat/completions to the same deployment is untouched

Plumbing: the router already puts each deployment's model_info (which preserves supported_endpoints via ModelInfo's extra="allow") onto kwargs["model_info"], which flows through the @client async wrapper into anthropic_messages_handler. The gate reads it there and selects the passthrough config before the translation fallback, so the documented model_info path works end to end with no new hidden litellm_params field

Files

• litellm/llms/openai_like/messages/transformation.py (new): OpenAILikeAnthropicMessagesConfig
• litellm/llms/anthropic/experimental_pass_through/messages/handler.py: _deployment_passes_through_anthropic_messages helper and the gate branch that selects the passthrough config when opted in
• litellm/llms/base_llm/anthropic_messages/transformation.py: should_filter_anthropic_beta_headers() hook (defaults to True) so configs can opt out of provider-based anthropic-beta filtering; the passthrough config overrides it to False
• litellm/llms/custom_httpx/llm_http_handler.py: gate the post-validate update_headers_with_filtered_beta call on that hook so the native passthrough does not strip anthropic-beta (the deployment routes as openai, which has no beta mapping and would otherwise drop every value)
• tests/test_litellm/llms/openai_like/messages/test_openai_like_anthropic_messages_transformation.py (new): URL building for api_base variants, Anthropic-shape payload preservation, case-insensitive Bearer/version/content-type header handling (including standard-cased caller headers and a caller-supplied x-api-key), and anthropic-beta forwarding/merging for context management and fast mode
• tests/test_litellm/.../test_anthropic_experimental_pass_through_messages_handler.py: gate regression tests (passthrough when opted in, translate otherwise). These fail on the base commit and pass after

Header defaults (Authorization: Bearer {api_key}, anthropic-version, content-type) are injected only when the caller did not already supply them, matched case-insensitively so a standard-cased Authorization / Anthropic-Version / Content-Type is not duplicated, and caller headers are returned untouched (no mutation of the inbound dict). A caller-supplied anthropic-beta (any casing) is preserved and merged with the betas the parent config adds for the requested Anthropic features

Docs live in the separate litellm-docs repo, not under docs/my-website/docs here: BerriAI/litellm-docs#436

---

Note

Medium Risk
Changes routing and outbound HTTP for the unified /v1/messages path on OpenAI-compatible deployments; misconfiguration could send requests to the wrong URL or bypass translation unexpectedly, though the opt-in is explicit and default behavior is preserved.

Overview
Deployments can opt in to forwarding inbound Anthropic /v1/messages without translating to chat/completions or the Responses API by setting model_info.supported_endpoints to include "/v1/messages" (alongside existing endpoints like /v1/chat/completions). When that flag is present and no native provider Anthropic-messages config exists, routing uses a new OpenAILikeAnthropicMessagesConfig that POSTs the Anthropic-shaped body to {api_base}/v1/messages, with default auth/version headers only when the caller did not supply them.

Passthrough configs disable provider-based anthropic-beta filtering via a new should_filter_anthropic_beta_headers() hook (default True on the base config; False for passthrough), so beta headers and Anthropic-only payload fields (e.g. cache_control, thinking) are not stripped on the openai provider path. Without the opt-in, behavior is unchanged: OpenAI-labeled deployments still go through translation.

Regression tests cover the handler gate (opt-in vs translate) and the new transformation (URL building, payload shape, headers, beta forwarding).

^{Reviewed by Cursor Bugbot for commit 4c1ce65. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR adds an opt-in path for forwarding Anthropic /v1/messages requests to native openai-like upstreams. The main changes are:

• Adds a model_info.supported_endpoints gate for /v1/messages passthrough
• Introduces OpenAILikeAnthropicMessagesConfig for native Anthropic-shaped requests
• Preserves caller headers and Anthropic beta headers on the passthrough path
• Keeps the existing translation behavior when the deployment is not opted in
• Adds tests for routing, URL handling, payload shape, header defaults, and beta forwarding

Confidence Score: 4/5

The change is reasonably safe to merge because the new behavior is gated by explicit per-deployment configuration and existing translation behavior is preserved when the opt-in is absent.

The implementation is focused and covered by tests for the new routing gate, payload preservation, URL construction, header defaults, and beta forwarding. Remaining risk is mainly around live upstream compatibility for openai-like providers that opt into the native messages path.

No specific files require changes before merge.

T-Rex Logs

What T-Rex did

• Ran the messages passthrough tests and observed before and after states, with opted_in returning 200 OK and the upstream request initially POST to /v1/responses in the base state, followed by a 200 OK POST to /v1/messages in the head state.
• Ran the messages headers tests and observed before- and after-state details: the base commit could not import litellm.llms.openai_like.messages, while the head commit returns /v1/messages URLs for all api_base variants, injects authorization, anthropic-version, and content-type defaults, preserves caller headers, leaves the inbound dict unchanged, returns should_filter: false, and keeps the anthropic-beta flag on the handler-like post-filter path.

_{Ran code and verified through T-Rex}

_{Reviews (8): Last reviewed commit: "chore: remove accidentally committed loc..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Missing anthropic-beta header injection
  • validate_anthropic_messages_environment now calls _update_headers_with_anthropic_beta after merging defaults, so opted-in features (context management, fast mode, structured outputs, advisor, tool search) get the required anthropic-beta values auto-injected.
• ✅ Fixed: Passthrough skips parent request prep
  • Removed the bespoke transform_anthropic_messages_request override so the passthrough config inherits AnthropicMessagesConfig's request prep (advisor stripping, reasoning_effort mapping, legacy thinking translation, context_management normalization).

_{You can send follow-ups to the cloud agent here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Case-sensitive anthropic-beta merge
  • Normalized any case variant of the anthropic-beta header key to lowercase before invoking _update_headers_with_anthropic_beta so existing caller beta flags are merged rather than duplicated.

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 107b468. Configure here.}

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Local proxy debug logs committed
  • Removed the accidentally committed proxy_after.log and proxy_before.log files from the repo root.
• ✅ Fixed: Local QA config committed
  • Removed the accidentally committed qa_config.yaml from the repo root.

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4c1ce65. Configure here.}