[Feat] Add support for cache_control_injection_points for Anthropic API, Bedrock API

[ishaan-jaff]

[Feat] Add support for cache_control_injection_points for Anthropic API, Bedrock API

Usage with litellm python SDK

response = await litellm.acompletion(
            model="bedrock/anthropic.claude-3-5-haiku-20241022-v1:0",
            messages=[
                {
                    "role": "assistant",
                    "content": [
                        {
                            "type": "text",
                            "text": "You are an AI assistant tasked with analyzing legal documents.",
                        },
                    ],
                },
                {
                    "role": "user",
                    "content": "what are the key terms and conditions in this agreement? <very_long_text>",
                },
            ],
            cache_control_injection_points=[
                {
                    "location": "message",
                    "role": "user",
                },
            ],
        )

Spec for cache_control_injection_points

Field	Type	Description
location	Literal["message"]	Type for message-level injection points
role	Optional[Literal["user", "system", "assistant"]]	Optional: target by role (user, system, assistant)
index	Optional[int]	Optional: target by specific index
control	Optional[ChatCompletionCachedContent]	Content to inject

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on (make test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 15, 2025 0:53am