feat: add support for anthropic_messages call type in prompt caching

[nuernber]

Relevant issues

Fixes #19226

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🐛 Bug Fix

Changes

Bug Fix: Allow anthropic_messages call type in PromptCachingDeploymentCheck

Problem:
Anthropic messages were never being cached in the PromptCachingCache due to the call type filter in PromptCachingDeploymentCheck.async_log_success_event (lines 65-68). The check only allowed completion and acompletion call types, causing anthropic_messages to be filtered out and return early without caching the model_id.

Solution:
Added CallTypes.anthropic_messages.value to the allowed call types check on line 68.

Changes:

• Modified litellm/router_utils/pre_call_checks/prompt_caching_deployment_check.py:
  • Line 68: Added and call_type != CallTypes.anthropic_messages.value to allow anthropic_messages call type
• Added test tests/router_unit_tests/test_prompt_caching_deployment_check.py:
  • Regression test that verifies anthropic_messages call type gets cached
  • Test creates a mock payload with anthropic_messages call type and verifies the model_id is stored in the cache

Impact:
Anthropic messages will now properly benefit from prompt caching routing, allowing requests with the same cacheable content to route to the same deployment for better cache hit rates.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
litellm	Ready	Preview, Comment	Jan 20, 2026 8:25pm

[ghost]

move to test_litellm/test_router.py - this will ensure it runs on the mock tests on github

[nuernber]

how does that look?

[ghost]

@nuernber can you contribute a simple .md guide for showing how to setup prompt caching routing for claude code w/ an example of success (how you validated it works)?

it seems like something a lot of users would benefit from.

Just need to put it in this folder -

litellm/docs/my-website/docs/tutorials/claude_mcp.md

Line 4 in 37c014c

# Use Claude Code with MCPs

[nuernber]

@nuernber can you contribute a simple .md guide for showing how to setup prompt caching routing for claude code w/ an example of success (how you validated it works)?

it seems like something a lot of users would benefit from.

Just need to put it in this folder -

litellm/docs/my-website/docs/tutorials/claude_mcp.md

Line 4 in 37c014c

# Use Claude Code with MCPs

Blocked by #19238 right now.

[nuernber]

See https://github.com/BerriAI/litellm/pull/19233/files#diff-e23b162642dad573338e891433a6fa6c85565c16379757c65a57f9a90572a69e

BEFORE (notice both API calls use different model id's and both incur cache writes):

1st call:

2nd call:

AFTER (notice both API calls use the same model id and the 2nd call has a cache read):

1st call:

2nd call:

[nuernber]

@nuernber can you contribute a simple .md guide for showing how to setup prompt caching routing for claude code w/ an example of success (how you validated it works)?

it seems like something a lot of users would benefit from.

Just need to put it in this folder -

litellm/docs/my-website/docs/tutorials/claude_mcp.md

Line 4 in 37c014c

# Use Claude Code with MCPs

Just added this, and see my comment above as well.

[nuernber]

@krrishdholakia how does the PR look now? Good to merge? It'd be great to get this fixed so it's included in the next LiteLLM release!

[ghost]

@jquinter can you review and let me know if this is good to merge

[jquinter]

@jquinter can you review and let me know if this is good to merge

@krrishdholakia Good to go: test passed locally (I confirm it), and errors here in the CI/CD are due to

• linter: an unrelated component (integrations/opentelemetry.py)
• MCP tests: 1 failing test, unrelated and already identified (issue repeating on all ci/cd pipelines)
• Mock tests: 13 failing tests, all unrelated to this PR, and mostly related to transformations in other components and calling real api instead of using mocks due a parallelization issue (also identified and working on it).

[nuernber]

@krrishdholakia, @jquinter can we merge this please?

[jquinter]

@greptile-apps can you review this?

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes a bug where anthropic_messages call types were being filtered out by PromptCachingDeploymentCheck, preventing Anthropic messages from benefiting from prompt cache routing.

Key Changes:

• Added CallTypes.anthropic_messages.value to the allowed call types in PromptCachingDeploymentCheck.async_log_success_event() (line 64)
• Updated debug message to reflect the new allowed call type
• Added comprehensive regression test to verify anthropic_messages calls are properly cached
• Added documentation tutorial for Claude Code prompt cache routing setup

Impact:
Anthropic messages will now properly route to the same deployment for cache hits, improving cache efficiency and reducing costs for Claude Code and other Anthropic API users with load balancing enabled.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The change is a straightforward bug fix that adds a single call type to an existing allowlist. The fix is well-tested with a comprehensive regression test, follows existing patterns in the codebase (same pattern used in litellm/utils.py and other files), includes helpful documentation, and has no performance or architectural concerns.
• No files require special attention

Important Files Changed

Filename	Overview
litellm/router_utils/pre_call_checks/prompt_caching_deployment_check.py	Added anthropic_messages to allowed call types for prompt caching - straightforward bug fix
tests/test_litellm/test_router.py	Added comprehensive regression test for anthropic_messages call type caching

Sequence Diagram

sequenceDiagram
    participant Client as Client/Claude Code
    participant Router as LiteLLM Router
    participant PreCheck as PromptCachingDeploymentCheck
    participant Cache as PromptCachingCache
    participant Deployment as Model Deployment
    
    Note over Client,Deployment: First Request with Prompt Caching
    Client->>Router: anthropic_messages() with cache_control
    Router->>PreCheck: async_filter_deployments()
    PreCheck->>Cache: async_get_model_id(messages)
    Cache-->>PreCheck: None (first time)
    PreCheck-->>Router: All healthy deployments
    Router->>Deployment: Send request to selected deployment
    Deployment-->>Router: Response (cache write tokens)
    Router->>PreCheck: async_log_success_event()
    Note over PreCheck: Check call_type == anthropic_messages ✓
    PreCheck->>Cache: async_add_model_id(model_id, messages)
    Cache-->>PreCheck: Cached
    Router-->>Client: Response
    
    Note over Client,Deployment: Subsequent Request (Same Prompt)
    Client->>Router: anthropic_messages() with cache_control
    Router->>PreCheck: async_filter_deployments()
    PreCheck->>Cache: async_get_model_id(messages)
    Cache-->>PreCheck: model_id (cached)
    Note over PreCheck: Filter to same deployment
    PreCheck-->>Router: [Single deployment with model_id]
    Router->>Deployment: Send to SAME deployment
    Deployment-->>Router: Response (cache read tokens)
    Router->>PreCheck: async_log_success_event()
    PreCheck->>Cache: async_add_model_id(model_id, messages)
    Router-->>Client: Response (with cache hit)

Loading

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}