fix(guardrails): preserve cache_control breakpoints in compresr write-back

[tin-berri]

TLDR

Problem this solves:

• Compresr silently dropped Anthropic cache_control breakpoints declared on any text block after the first, so a caller's configured prompt cache never formed
• For a row holding an image between two text blocks, compresr moved the post-image text to the slot before the image, so the model saw a different content order than the caller sent

How it solves it:

• Breakpoints are positional, so a compressed string is only written back over a contiguous run of text parts; the merged part carries the last declared breakpoint and its TTL
• Rows holding a non-text part are no longer selected for compression

Relevant issues

• Compresr keeps only the first text part's cache_control when it rewrites a multi-part row, dropping every later breakpoint
• Compresr flattens text from both sides of an image into one context and writes it back before the image, relocating content and dropping its breakpoint
• Both hazards ship today; this is a follow-up to the same positional-breakpoint invariant fixed for the headroom guardrail in fix(guardrails): compress content-parts messages in headroom guardrail (Anthropic traffic) #34586

Linear ticket

Resolves LIT-4804

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Live proxy on localhost:4804 against real claude-sonnet-5 through /v1/messages, so the cache counters below are Anthropic's own accounting. The compresr compression service is a local stub that records every context it is handed and returns a deterministic ~4000 token summary; that is the component under test's collaborator, and stubbing it is what makes "which rows did the gateway select, and what text did it flatten" observable.

Two request shapes, byte-identical across the before and after runs:

# A: all-text row, cache_control {"type":"ephemeral","ttl":"1h"} on the LAST of two text blocks
# B: mixed row [text, image, text(cache_control ephemeral)]
curl -s -X POST http://127.0.0.1:4804/v1/messages \
  -H 'x-api-key: sk-1234' -H 'anthropic-version: 2023-06-01' -H 'content-type: application/json' \
  --data-binary @req_alltext.json | jq .usage

curl -s http://127.0.0.1:8934/stats   # what the gateway asked the compression service to compress

Before, on unfixed code:

A all-text : stub called once with 27322 chars (both blocks flattened)
             input=4977 cache_creation=0 cache_read=0 ephemeral_1h=0
B mixed    : stub called once with 27322 chars (text from BOTH sides of the image)
             input=4981 cache_creation=0 cache_read=0 ephemeral_5m=0

The 27322 chars on B is the hazard in one number; it is block A plus block B concatenated, with the image dropped from the middle, and that single string is written back ahead of the image.

After, same requests:

A all-text : stub called once with 27322 chars (still compressed)
             input=2 cache_creation=4975 cache_read=0 ephemeral_1h=4975
A repeated : input=2 cache_creation=0 cache_read=4975
B mixed    : stub called 0 times (row no longer selected)
             input=2 cache_creation=8130 cache_read=0 ephemeral_5m=8130

A still compresses and now writes a 1h cache entry, and the repeat run reads 4975 tokens back out of it, which is the caching the caller configured and previously never got. B is left alone, so its blocks reach the model in the order they were sent and its breakpoint is honored.

Type

🐛 Bug Fix

Changes

• Add guardrail_hooks/content_text.py holding content_to_text (moved out of compresr) plus is_all_text_parts and merge_rewritten_text_parts, so the positional-breakpoint rule has one owner
• _replace_text_in_content collapses an all-text row into a single part carrying the last declared cache_control and its TTL, and returns any other list unchanged
• _select_targets stops selecting rows that hold a non-text part
• Update test_compresr.py for the new contract and pin both hazards; string content is unaffected throughout

QA runbook

Configure the compresr guardrail against a compression service, send an Anthropic request whose user content is two text blocks with cache_control on the second, and confirm the response reports non-zero cache_creation_input_tokens; repeat the request and confirm cache_read_input_tokens matches. Then send a row shaped [text, image, text] and confirm the content reaches the model unmodified.

Things a reviewer will ask about

Mixed rows stop being compressed, and that is the deliberate trade. A breakpoint caches the prefix ending at its own part, so no single-string write-back can preserve one across an image; compressing each text run separately would mean one service call per run and a per-part rather than per-message batch contract, which is a larger change than the defect warrants. This matches the contract #34586 landed for headroom.

The merged part carries the last declared breakpoint rather than the first. After the merge the row is a single part, so the last one is the only one that still describes it; keeping the first would cache a prefix shorter than any the caller asked for.

content_text.py is also created by #34586, which is open. Whichever lands first, the other resolves to the union; this PR's copy is a superset and content_to_text is byte-identical in both.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes Compresr guardrail message mutation and eligibility rules for multimodal content; behavior is narrower (fewer rows compressed) but affects prompt caching and tool-message shapes on the proxy hot path.

Overview
Fixes Compresr multimodal write-back so Anthropic cache_control breakpoints and part order stay correct.

Shared helpers in content_text.py (content_to_text, is_all_text_parts, merge_rewritten_text_parts) replace inline Compresr logic and document the positional-breakpoint rule (aligned with headroom).

Write-back: all-text part lists collapse to one text part with the last declared cache_control (and TTL). Lists that include images or other non-text parts are returned unchanged—no more flattening text across an image.

Target selection: messages whose content is not an all-text part list are skipped for compression, so mixed rows are never sent to the service or rewritten.

Unit tests cover last-breakpoint preservation, skipping mixed rows, and refusing to relocate text around non-text parts.

^{Reviewed by Cursor Bugbot for commit c63e24b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR prevents Compresr from rewriting mixed-content rows and preserves the final cache-control breakpoint when compressed all-text parts are merged.

• Moves shared content extraction and text-part merging helpers into content_text.py
• Restricts compression targets to strings and all-text part lists
• Adds regressions for cache-control preservation and mixed-content ordering

Confidence Score: 5/5

The PR appears safe to merge; no actionable correctness or security defects were identified.

Mixed-content rows are now excluded before compression, while all-text rows retain their final cache-control breakpoint during write-back, with focused tests covering both corrected behaviors.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/compresr/compresr.py	Uses shared content helpers, skips mixed-content rows, and safely writes compressed text back only to strings or all-text part lists.
litellm/proxy/guardrails/guardrail_hooks/content_text.py	Adds focused helpers that flatten textual content, identify safe all-text rows, and retain the last positional cache-control breakpoint during merging.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_compresr.py	Adds meaningful regression coverage for preserving the final cache-control value and leaving mixed-content ordering untouched.

_{Reviews (1): Last reviewed commit: "fix(guardrails): preserve cache_control ..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 93.93939% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...xy/guardrails/guardrail_hooks/compresr/compresr.py	90.90%	1 Missing ⚠️
...m/proxy/guardrails/guardrail_hooks/content_text.py	95.45%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit4804_compresr_cache_control (c63e24b) with litellm_internal_staging (1a0acaa)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (d816478) during the generation of this report, so 1a0acaa was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c63e24b. Configure here.}