fix(bedrock,azure_ai): strip scope from cache_control for Anthropic messages

[Sameerlite]

Summary

Bedrock and Azure AI Foundry's Anthropic endpoints do not support the scope field in cache_control (e.g., "global" for cross-request caching). Requests with cache_control: { type: "ephemeral", scope: "global" } were failing with:

system.0.cache_control.ephemeral.scope: Extra inputs are not permitted

Changes

Bedrock

• Extend _remove_ttl_from_cache_control to also remove scope from cache_control
• Process both system and messages content blocks (previously only messages)
• Add test for scope removal

Azure AI

• Add _remove_scope_from_cache_control and override transform_anthropic_messages_request
• Strip scope from cache_control in both system and messages before sending
• Add test for scope removal

Testing

• pytest tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py -v
• pytest tests/test_litellm/llms/azure_ai/claude/test_azure_anthropic_messages_transformation.py -v

Made with Cursor

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 5, 2026 5:21am

[greptile-apps]

Greptile Summary

Strips the unsupported scope field from cache_control before sending requests to Bedrock and Azure AI Foundry Anthropic endpoints, fixing 400 errors when users specify cache_control: { type: "ephemeral", scope: "global" }. The Bedrock change also extends system-block processing (previously only messages were processed). Both providers add dedicated tests.

• Bedrock: _remove_ttl_from_cache_control refactored to also remove scope and process system content blocks (not just messages). Supports TTL retention for Claude 4.5 models.
• Azure AI: New _remove_scope_from_cache_control method and transform_anthropic_messages_request override to strip scope from both system and messages.
• The system/messages traversal logic is duplicated between the two providers — could be consolidated into the base class.
• The Bedrock method name _remove_ttl_from_cache_control is now stale since it also removes scope and processes system blocks.

Confidence Score: 4/5

• This PR is safe to merge — it only strips unsupported fields before sending to provider APIs, with no behavioral changes for supported configurations.
• The changes are straightforward field stripping with correct logic, tests covering both system and message blocks, and no backwards-incompatible behavior. Minor naming and duplication concerns don't affect correctness.
• No files require special attention — both implementation files have minor style concerns but no functional issues.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py	Extends _remove_ttl_from_cache_control to also strip scope and process system blocks; logic is correct but method name is now misleading.
litellm/llms/azure_ai/anthropic/messages_transformation.py	Adds _remove_scope_from_cache_control and overrides transform_anthropic_messages_request to strip scope; logic is correct but duplicates Bedrock traversal pattern.
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py	Adds mock-only test for scope removal from both system and messages blocks; no network calls.
tests/test_litellm/llms/azure_ai/claude/test_azure_anthropic_messages_transformation.py	Adds mock-only test for scope removal via transform_anthropic_messages_request end-to-end; no network calls.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["User sends request with<br/>cache_control: {type: ephemeral, scope: global}"] --> B{Provider?}
    B -->|Bedrock| C["_remove_ttl_from_cache_control()"]
    B -->|Azure AI| D["_remove_scope_from_cache_control()"]
    C --> E["Strip 'scope' from system blocks"]
    C --> F["Strip 'scope' from message blocks"]
    C --> G["Strip 'ttl' (non-4.5 models)"]
    D --> H["Strip 'scope' from system blocks"]
    D --> I["Strip 'scope' from message blocks"]
    E & F & G --> J["Send sanitized request to Bedrock API"]
    H & I --> K["Send sanitized request to Azure AI API"]

Loading

_{Last reviewed commit: 482bc93}

Comments Outside Diff (1)

1. undefined, line undefined (link)

   Method name is now misleading

   This method now removes both ttl and scope from cache_control, and processes both system and messages blocks — but its name _remove_ttl_from_cache_control only references TTL. Consider renaming to something like _sanitize_cache_control to accurately reflect the broader scope of what it does. The stale name will confuse future contributors who need to understand what fields are being stripped.

   Note: this would also require updating the call site at line 409 and the test references.

[greptile-apps]

Duplicated sanitization logic across providers

The system/messages iteration pattern in _remove_scope_from_cache_control is nearly identical to the one in Bedrock's _remove_ttl_from_cache_control. Consider extracting the shared traversal logic (iterating over system + message content blocks and calling a sanitizer on each cache_control dict) into a shared utility in the base class BaseAnthropicMessagesConfig, with each provider only supplying the sanitization function. This would reduce duplication and make it easier to add future field stripping in one place.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}