fix(responses): propagate message cache_control through Responses->Chat transform (LIT-2594)

[oss-agent-shin]

Summary

cache_control_injection_points configured on a Responses API request was silently dropped before the messages reached the provider transform, so prompt caching on the Responses path was a no-op for both Anthropic and Bedrock Converse Anthropic models.

Root cause

AnthropicCacheControlHook.get_chat_completion_prompt correctly attaches cache_control to each input message in chat-completion form (when the input items have a role field). But the next step in the Responses pipeline -- LiteLLMCompletionResponsesConfig._transform_responses_api_input_item_to_chat_completion_message -- constructed:

GenericChatCompletionMessage(role=..., content=...)

without forwarding any other fields, so the message-level cache_control was dropped on the floor. anthropic_messages_pt and AnthropicConfig.translate_system_message both read top-level cache_control on string-content messages, but they never saw it.

Fix

In the else-branch of _transform_responses_api_input_item_to_chat_completion_message, after building the GenericChatCompletionMessage, propagate cache_control from the input item onto the resulting message dict when present. Existing behaviour for tool-call / function-call input items (which have their own transforms and do not carry message-level cache_control) is unchanged.

Diff is 23 lines on the source file (one-line propagation + comment) plus 195 lines of regression tests.

Tests

10 new tests in tests/test_litellm/responses/litellm_completion_transformation/test_cache_control_propagation_lit_2594.py:

• 6 direct unit tests on the input-item transform (user / system / assistant string content, list content, no-cache-control baseline, None content short-circuit)
• 4 end-to-end tests driving AnthropicCacheControlHook -> Responses-Chat transform -> AnthropicConfig.translate_system_message + anthropic_messages_pt to assert that:
  • the Anthropic system block carries cache_control
  • the last user message block carries cache_control
  • the middle user message does NOT carry cache_control (no leak)
  • without injection points, no cache_control anywhere (regression safety net)

All 10 tests pass. Full surrounding suite (tests/test_litellm/responses/litellm_completion_transformation/ + tests/test_litellm/integrations/test_anthropic_cache_control_hook.py) is 130 passed, 0 failed.

Evidence

Drove the exact bug-report flow (Responses API input with cache_control_injection_points on system + last user, large system prompt) through AnthropicCacheControlHook -> Responses-Chat transform -> AnthropicConfig.translate_system_message + anthropic_messages_pt.

BEFORE (clean litellm_oss_agent_shin_daily_branch HEAD 1fe911d8):

After AnthropicCacheControlHook:
  role=system    cache_control={'type': 'ephemeral'}
  role=user      cache_control=None
  role=assistant cache_control=None
  role=user      cache_control={'type': 'ephemeral'}

After Responses->Chat transform (CLEAN BASE, NO FIX):
  role=system    cache_control=None
  role=user      cache_control=None
  role=assistant cache_control=None
  role=user      cache_control=None

Anthropic system block:
  type=text text='You are a helpful assistant. system cont...' cache_control=None

Last Anthropic user message blocks:
  type=text text='What is its capital?' cache_control=None

RESULT: cache_control reached Anthropic system AND last user? -> False

AFTER (this branch):

After AnthropicCacheControlHook:
  role=system    cache_control={'type': 'ephemeral'}
  role=user      cache_control=None
  role=assistant cache_control=None
  role=user      cache_control={'type': 'ephemeral'}

After Responses->Chat transform (THIS REPO, WITH FIX):
  role=system    cache_control={'type': 'ephemeral'}
  role=user      cache_control=None
  role=assistant cache_control=None
  role=user      cache_control={'type': 'ephemeral'}

Anthropic system block:
  type=text text='You are a helpful assistant. system cont...' cache_control={'type': 'ephemeral'}

Last Anthropic user message blocks:
  type=text text='What is its capital?' cache_control={'type': 'ephemeral'}

RESULT: cache_control reached Anthropic system AND last user? -> True

Repro script: a 60-line Python harness that builds the bug-report input, drives the hook, runs the transform, then calls AnthropicConfig.translate_system_message + anthropic_messages_pt and prints the cache_control state at every stage. Same script run on both branches with only the source file swapped.

Notes

• Files pushed via the GitHub Contents API because the agent's GITHUB_TOKEN lacks repo+workflow scope for direct git push. Merge-base diff is therefore noisier than the actual change -- the real diff is the two files in this PR.
• Targets litellm_oss_agent_shin_daily_branch per the Shin fork-PR policy.

Session: https://litellm-agent-platform.onrender.com/sessions/75246fbc-562e-44c4-8fb8-032ac0cb2bda

---

Verification (ship-pr)

• Base branch is litellm_oss_agent_shin_daily_branch (Shin fork-PR policy)
• Title carries the Linear ticket id (LIT-2594)
• PR body has ### Evidence with BEFORE/AFTER captured from the real AnthropicCacheControlHook -> Responses->Chat transform -> AnthropicConfig.translate_system_message + anthropic_messages_pt chain
• Unit tests added (10 new, all passing)
• No customer name in title/body/comments
• Greptile: 5/5 ("Safe to merge - the change is a minimal, well-scoped propagation of an existing field with no side-effects on unrelated code paths")
• All 43 functional GitHub Actions green at commit 1a2bd369 (unit-test, lint, semgrep, secret-scan, code-quality, build-ui, validate-model-prices-json, test-server-root-path x2, Codecov, every proxy-* and provider test suite). Only outstanding check is Veria AI - PR Review which has been in_progress for >25 min with no annotations or output -- known intermittent flake (see prior LIT-3320 / LIT-2553 session notes). No security-relevant code paths were touched (the change copies an already-validated dict field onto another dict).
• Codecov: 0% patch coverage flagged by bot; in practice the new propagation line is covered by test_string_user_message_with_cache_control_is_preserved and 5 sibling unit tests (Codecov upload races on multi-shard runs and frequently underreports patch coverage on this repo).
• No secrets in diff, comments, or PR body

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes a silent bug where cache_control_injection_points on the Responses API path was dropped before reaching Anthropic/Bedrock provider transforms. The fix is a targeted one-liner in _transform_responses_api_input_item_to_chat_completion_message that copies cache_control from the input item onto the resulting GenericChatCompletionMessage, plus a matching type-safety change that adds cache_control to the GenericChatCompletionMessage TypedDict.

• transformation.py: In the else-branch of _transform_responses_api_input_item_to_chat_completion_message, after constructing the GenericChatCompletionMessage, the new code reads cache_control from the input dict and sets it on the message when present, so anthropic_messages_pt and AnthropicConfig.translate_system_message can apply it downstream.
• openai.py: cache_control: ChatCompletionCachedContent added to GenericChatCompletionMessage, making the assignment type-safe and removing the need for a type: ignore comment.
• New tests: 10 focused unit/integration tests in test_cache_control_propagation_lit_2594.py cover both the transform in isolation and the full Anthropic pipeline (system block, last user block, middle-user no-leak, and no-injection-points baseline).

Confidence Score: 5/5

Safe to merge — the change is a minimal, well-scoped propagation of an existing field with no side-effects on unrelated code paths.

The fix touches only the else-branch of the input-item transform (tool-call and function-call paths are untouched), adds a type-safe field to GenericChatCompletionMessage, and is protected by 10 focused tests including a no-injection-points regression baseline. No existing behaviour is altered when cache_control is absent.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/transformation.py	Propagates cache_control from Responses-API input items onto GenericChatCompletionMessage; logic is correct and the isinstance(input_item, dict) guard is safe since TypedDicts are dict instances in Python.
litellm/types/llms/openai.py	Adds cache_control: ChatCompletionCachedContent to GenericChatCompletionMessage, making it consistent with all other concrete message TypedDicts and removing the need for a type-ignore comment.
tests/test_litellm/responses/litellm_completion_transformation/test_cache_control_propagation_lit_2594.py	10 new pure-unit tests (no real network calls) covering the transform in isolation and the full Anthropic pipeline; good regression coverage including the no-injection-points baseline.

_{Reviews (2): Last reviewed commit: "style: drop type:ignore now that Generic..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[oss-agent-shin]

@greptileai please re-review — addressed P2 from the prior review by adding cache_control: ChatCompletionCachedContent to the GenericChatCompletionMessage TypedDict in litellm/types/llms/openai.py, and dropped the type: ignore[typeddict-unknown-key] on the propagation line. The assignment is now structurally type-safe. Tests still pass (10/10).

[edelauna]

Would like this fix please 🙏

[edelauna]

Checking isinstance(input_item, dict) will silently drop cache_control if messages are passed in as pre-parsed objects or Pydantic models (such as ChatCompletionSystemMessageParam or ResponseInputTextParam) rather than standard Python dictionaries.

Let's add a fallback check to extract the property from object attributes using hasattr/getattr to make this completely bulletproof:

Suggested change

	else None
	else getattr(input_item, "cache_control", None)

[mateo-berri]

Closing as superseded: main already propagates Responses cache_control through #27727, and the Gemini pieces landed through #41938