feat(messages): route native Anthropic /messages through Rust behind LITELLM_RUST env var

[devin-ai-integration]

Relevant issues

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proof at commit a461146af8. Local proxy started with LITELLM_RUST=1 against a native anthropic/claude-haiku-4-5 deployment (dev_config.yaml), then:

curl -sD - http://localhost:4000/v1/messages -H 'x-api-key: sk-1234' \
  -H 'anthropic-version: 2023-06-01' -H 'content-type: application/json' \
  -d '{"model":"anthropic-haiku-4-5","max_tokens":64,"messages":[{"role":"user","content":"Say hi in 3 words."}]}'

Response headers (Rust path taken + cost still computed by the Python logging/cost path):

HTTP/1.1 200 OK
x-litellm-rust: true
x-litellm-response-cost: 5.5e-05
x-litellm-model-name: anthropic/claude-haiku-4-5

Body:

{"content":[{"text":"Hey there, friend!","type":"text"}],"id":"msg_011CdABh2scKsoaVZo5w2nKK",
 "model":"anthropic-haiku-4-5","role":"assistant","stop_reason":"end_turn",
 "type":"message","usage":{"input_tokens":15,"output_tokens":8}}

Proxy --detailed_debug log confirms the upstream is native https://api.anthropic.com/v1/messages and the response carries _hidden_params.additional_headers = {'x-litellm-rust': 'true'}. With LITELLM_RUST unset the same request takes the existing Python path unchanged. This was also verified end to end by driving the real Claude Code CLI (interactive REPL: code + plan tasks) against the LITELLM_RUST=1 proxy, with the resulting spend showing up in the LiteLLM UI Logs (call_type=anthropic_messages)

Type

🆕 New Feature

Changes

Extends the existing azure_ai Rust /messages path (#33616) to the native anthropic provider, gated by an env var so rollout stays off by default.

• Rust ai-gateway: register anthropic -> ANTHROPIC_MESSAGES_CONFIG in the messages_provider_config map. The core config (native /v1/messages, x-api-key, anthropic-version header, identity transforms) already existed; this makes it reachable from the HTTP host. Provider resolution, prepare, and handler stay provider-generic per PROVIDER_CODING_STANDARDS.md.

  match provider {
      "azure_ai" => Some(&AZURE_ANTHROPIC_MESSAGES_CONFIG),
      "anthropic" => Some(&ANTHROPIC_MESSAGES_CONFIG),   // added
      _ => None,
  }

• Python gate (BaseLLMHTTPHandler._maybe_rust_anthropic_messages): accept custom_llm_provider in {azure_ai, anthropic}, and enable the Rust path when the LITELLM_RUST env var is truthy (1/true/yes/on) OR the existing litellm_params["rust"] is True. The env var is namespaced LITELLM_RUST (not a bare RUST) to avoid colliding with unrelated CI/deployment env. Fallback to the Python path is unchanged (bridge missing, bridge error, disabled). Logging/cost is untouched: Rust returns the response dict with usage, so StandardLoggingPayload + cost calculation still run — spend continues to be tracked (see the x-litellm-response-cost header above).

  if custom_llm_provider not in ("azure_ai", "anthropic"):
      return None
  if litellm_params.get("rust") is not True and not BaseLLMHTTPHandler._rust_env_enabled():
      return None

• Tests: Rust ai-gateway gains a native-anthropic round-trip test (asserts POST /v1/messages, x-api-key, anthropic-version) and the provider-map test now expects anthropic to resolve (unsupported-provider coverage moved to openai). Python test_rust_bridge_messages.py gains cases for native-anthropic routing, LITELLM_RUST env enable, LITELLM_RUST=0 no-op, and unsupported-provider skip.

Validation: cargo fmt --check, cargo clippy --workspace --all-targets --locked -D warnings, cargo test --workspace --locked (all green); pytest tests/test_litellm/anthropic_interface/test_rust_bridge_messages.py (19 passed); make pre-commit clean.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/db90b4ee508f4e81b5c19568c03cd157
Requested by: @ishaan-berri

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR extends the existing Rust-backed /messages gateway (previously only for azure_ai) to the native anthropic provider. On the Rust side it's a single match arm pointing to a pre-existing ANTHROPIC_MESSAGES_CONFIG. On the Python side a new _rust_env_enabled() helper reads a RUST env var, and the provider guard is widened to accept both "azure_ai" and "anthropic". Fallback to the Python path on any bridge error or missing bridge is preserved.

• The Rust change is minimal: the native-anthropic ANTHROPIC_MESSAGES_CONFIG already existed in litellm-core; this PR makes it reachable from the HTTP gateway and adds a round-trip integration test.
• The Python gate now accepts both "azure_ai" and "anthropic", meaning setting RUST=1 enables the Rust path for both providers simultaneously.
• Two pre-existing tests are now sensitive to the RUST env var being set externally, and the generic one-word name RUST risks accidental activation in CI or deployment environments.

Confidence Score: 4/5

Safe to merge; the Rust path is off by default and falls back to Python on any error, so existing traffic is not disrupted.

The core routing logic is a one-line Rust match arm backed by a pre-existing config, and the Python gate preserves the existing fallback contract. The two concerns — the generic RUST env var name and the pre-existing tests that do not isolate themselves from that var — are non-blocking in a typical CI environment but could cause surprise in environments where RUST happens to be set.

litellm/llms/custom_httpx/llm_http_handler.py for the env var name; tests/test_litellm/anthropic_interface/test_rust_bridge_messages.py for the two tests that do not unset RUST.

Important Files Changed

Filename	Overview
litellm-rust/crates/ai-gateway/src/messages/common_utils.rs	Adds anthropic to the provider match in messages_provider_config, wiring it to the pre-existing ANTHROPIC_MESSAGES_CONFIG; minimal and correct.
litellm-rust/crates/ai-gateway/src/messages/tests.rs	Existing unsupported-provider test updated from anthropic to openai (correct since anthropic is now supported); new round-trip test validates path, x-api-key, and anthropic-version headers.
litellm/llms/custom_httpx/llm_http_handler.py	Adds _rust_env_enabled() (reads RUST env var) and expands _maybe_rust_anthropic_messages to accept both azure_ai and anthropic; the generic env var name and its interaction with pre-existing tests deserve attention.
tests/test_litellm/anthropic_interface/test_rust_bridge_messages.py	New tests for native-anthropic routing and RUST env var are well-structured; however, two pre-existing tests do not unset the new RUST env var and would fail if it is set externally.

Comments Outside Diff (1)

1. tests/test_litellm/anthropic_interface/test_rust_bridge_messages.py, line 265-284 (link)

   Pre-existing tests are now sensitive to the new RUST env var

   test_gate_skips_rust_when_flag_absent and test_gate_skips_rust_when_flag_false both use ExplodingAsyncMessages and assert the bridge is never called — but neither unsets RUST via monkeypatch. If the test runner's environment has RUST=1 (or any truthy value), _rust_env_enabled() returns True, the gate no longer short-circuits, and both tests will hit AssertionError("bridge must not be called"). The autouse _reset_rust_flag fixture only resets the litellm bridge state, not the process environment. Adding monkeypatch.delenv("RUST", raising=False) at the top of both tests would make them robust against the new env var.

_{Reviews (1): Last reviewed commit: "feat(messages): route native Anthropic /..." | Re-trigger Greptile}

[greptile-apps]

The env var name RUST is very short and generic. While the Rust toolchain itself does not export RUST=1, the name is easy to collide with unrelated CI scripts or deployment automation that happen to export a RUST variable for other reasons. A namespaced name like LITELLM_RUST keeps the intent unambiguous and avoids accidental activation in environments not specifically targeting this feature.

Suggested change

	@staticmethod
	def _rust_env_enabled() -> bool:
	return os.getenv("RUST", "").strip().lower() in {"1", "true", "yes", "on"}
	@staticmethod
	def _rust_env_enabled() -> bool:
	return os.getenv("LITELLM_RUST", "").strip().lower() in {"1", "true", "yes", "on"}

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[devin-ai-integration]

Agreed, renamed to LITELLM_RUST to keep it namespaced and avoid accidental activation from unrelated env. Done in a461146 (also updated the doc-flag exclusion, the tests, and the PR title/description)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_rust_anthropic_messages (a461146) with litellm_internal_staging (e238e89)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (3f9b71c) during the generation of this report, so e238e89 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[devin-ai-integration]

Verified: Claude Code CLI -> LiteLLM Rust core (RUST=1) end to end

Drove the real @anthropic-ai/claude-code CLI v2.1.197 against a DB-backed proxy started with RUST=1 (native Anthropic deployments, --detailed_debug), real Anthropic API, spend on a virtual key

• x-litellm-rust: true on /v1/messages from the RUST proxy; the same request against a proxy without RUST returns 200 with no such header (real discriminator, no mocks)
• Real code task: Claude Code wrote fib.py that executes to [0, 1, 1, 2, 3, 5, 8, 13]
• Plan mode (--permission-mode plan): ExitPlanMode called, result.subtype=success, full plan produced, no source files written
• Proxy debug log shows native api.anthropic.com/v1/messages upstream plus the x-litellm-rust hidden param
• UI Logs show nonzero spend: call_type=anthropic_messages, model=anthropic/claude-sonnet-4-5, User-Agent: claude-cli ($0.4045 across 21 anthropic_messages rows via /spend/logs)

Rust vs Python header discriminator

UI spend for Rust-path Claude Code traffic

Code task + plan mode

Note: Claude Code emits occasional sub-requests with a role: system message that Anthropic 400s; this is identical on Rust and Python paths (not introduced here) and the CLI recovers