fix(guardrails): compress content-parts messages in headroom guardrail (Anthropic traffic)

[tin-berri]

TLDR

Problem this solves:

• Headroom compression never fired for Anthropic clients (claude-cli, Anthropic SDKs)
• Compression silently skipped any message whose content is a list of parts

How it solves it:

• Flatten text-bearing content-part lists to strings before calling /v1/compress
• Restore original shapes (images, cache_control) from the service response

Relevant issues

• Headroom compression appeared to run only on OpenAI models and never on Anthropic models; the real discriminator is the message content shape, not the model or route
• Anthropic-format requests always translate to list-of-parts content, which the compression service's transforms skip via isinstance(content, str) gates, so the request was forwarded uncompressed while the guardrail still reported itself as applied
• Fix flattens text-bearing parts on the wire and restores the original content shapes afterwards, sharing the content/text helpers compresr already owned

Linear ticket

Resolves LIT-4795

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Changes

• headroom.py: flatten ALL-TEXT list-of-parts content to plain strings in the /v1/compress request, then write returned text back; rows the service returned unchanged keep their exact original parts (all cache_control breakpoints intact), a rewritten all-text row collapses to one part carrying the LAST declared breakpoint (a breakpoint caches the prefix ending at its part, so after the merge the last one and its TTL still describe the row); rows with any non-text part are never flattened, because merging text across a non-text part would move a later breakpoint to the other side of it; if the service restructures the conversation (row count or roles change) its output is adopted as-is, which is the previous behavior
• Flattening and write-back use the shared content_text.py helpers (content_to_text, is_all_text_parts, merge_rewritten_text_parts) that the compresr breakpoint fix (LIT-4804, fix(guardrails): preserve cache_control breakpoints in compresr write-back #34660) already landed on staging
• What does NOT change: OpenAI plain-string traffic sends the identical wire payload as before; this PR no longer touches compresr or content_text.py at all (both are owned by the merged fix(guardrails): preserve cache_control breakpoints in compresr write-back #34660); compression_interception compresses in-process without the service and is unaffected

Root cause

The headroom service's transforms only rewrite string content (smart_crusher.py, code_compressor.py, kompress_compressor.py all gate on isinstance(content, str)) and pass list-of-parts content through untouched. Every Anthropic /v1/messages request translates to parts-list content in structured_messages, so compression was a silent no-op for all Anthropic client traffic. OpenAI clients mostly send plain strings, which made the bug look model-scoped. A /v1/chat/completions request with content-parts skipped compression the same way, and a /v1/messages request with plain-string content compressed fine, confirming content shape as the discriminator (matrix below, run against a live proxy).

content shape	/v1/chat/completions	/v1/messages
plain string	compressed (29k -> 908 tokens)	compressed (906)
content-parts list	not compressed (28,937)	not compressed (28,944)

Proof of fix (live proxy + string-only compress mock + mock upstream)

Before, claude-cli-shaped /v1/messages request (parts content, cache_control, tools):

[compress] model=claude-fable-5 messages=2 chars 47603 -> 47603
[anthropic-upstream] RECEIVED messages_chars=47390 tools=['Bash'] cache_control_count=3

After, same request on fixed code:

[compress] model=claude-fable-5 messages=2 chars 47401 -> 359
[anthropic-upstream] RECEIVED messages_chars=348 tools=['Bash', 'headroom_retrieve'] cache_control_count=3

Negative control on fixed code, plain-string /v1/chat/completions still compresses:

[compress] model=gpt-test messages=1 chars 47320 -> 279
[openai-upstream] RECEIVED messages_chars=279 tools=['headroom_retrieve']

Mutation check: reverting the flatten/restore wiring fails the three new shape tests; full suite is 145 passed.

Things a reviewer will ask about

• Why fix in the guardrail rather than the anthropic translation handler: the same skip hits /v1/chat/completions when a caller sends content-parts, so the invariant ("the compress service only rewrites strings") belongs at the litellm-to-service boundary where it covers every route
• Positional row matching in _restore_content_shapes is only trusted when the row count and every role line up; on any mismatch the service output is adopted wholesale, which is exactly the pre-flattening behavior, so alignment can never make a request worse than today
• Rows with ANY non-text part (e.g. text plus an image) are sent unflattened and pass through the service untouched: cache_control breakpoints are positional, so merging text across a non-text part would silently cache a shorter prefix than the caller configured; image bytes therefore never round-trip through the compression service

---

Note

Medium Risk
Changes request shaping for a guardrail on the proxy path; mistakes could alter cache_control or multimodal rows, though scope is limited to all-text flattening with positional restore guards and broad tests.

Overview
Headroom compression now works for Anthropic-style list-of-parts message content, which previously skipped /v1/compress because the service only rewrites string content.

Before calling compress, the guardrail flattens rows whose parts are all text into plain strings (via shared content_text helpers), then restores the original shapes after the response. Unchanged compressed text keeps the exact original parts (including cache_control); rewritten all-text rows merge back into a single part with the last breakpoint. Rows with any non-text part (e.g. images) stay unflattened so positional cache breakpoints are not shifted.

If the service changes message count or roles, its output is used as-is (unchanged fallback). New unit tests cover wire flattening, restore, row drops, and fail-open behavior.

^{Reviewed by Cursor Bugbot for commit 33fadd7. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

Fixes Headroom compression for all-text content-part messages.

• Flattens eligible content-part lists before sending them to the compression service.
• Restores original message shapes and cache-control metadata after compression.
• Leaves mixed text and non-text rows unflattened.
• Adds regression coverage for rewriting, pass-through, restructuring, and fail-open behavior.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain within the scope of the previous review threads.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/headroom/headroom.py	Adds content-part flattening and shape restoration around the existing Headroom compression call without leaving an eligible follow-up defect.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_headroom.py	Adds focused tests covering all-text flattening, mixed-content preservation, rewritten cache metadata, service restructuring, and fail-open restoration.

_{Reviews (4): Last reviewed commit: "fix(guardrails): compress content-parts ..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 75.55556% with 11 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...xy/guardrails/guardrail_hooks/headroom/headroom.py	75.55%	11 Missing ⚠️

📢 Thoughts on this report? Let us know!

[tin-berri]

@greptileai

[tin-berri]

@greptileai

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 5af69dd. Configure here.}

[tin-berri]

Note on codecov/patch: the failure is a stale-base artifact, not this diff. Codecov's patch report counts missed lines in files this PR does not touch (ui_sso.py 20, saml_sso.py 17, mcp_server/server.py 12, batches_endpoints/endpoints.py 6, proxy/utils.py 5 — all from upstream litellm_internal_staging commits swept into the comparison; none appear in this PR's file list). The lines actually changed by this PR are 51/53 covered (96%), with the 2 misses being defensive branches in headroom.py. Sibling PRs opened against a fresher base show codecov/patch green.

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit4795_headroom_anthropic (33fadd7) with litellm_internal_staging (079003b)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (c37fb75) during the generation of this report, so 079003b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[tin-berri]

Follow-up for the same positional-breakpoint hardening in the compresr guardrail (pre-existing there, deliberately not folded into this PR): LIT-4804

[tin-berri]

@greptileai

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 33fadd7. Configure here.}