fix: sanitize empty text content blocks in /v1/messages endpoint - #22979
fix: sanitize empty text content blocks in /v1/messages endpoint#22979atian8179 wants to merge 1 commit into
Conversation
Claude's API returns assistant messages with empty text blocks alongside tool_use blocks, but rejects them when sent back in subsequent requests. The /v1/messages endpoint passes messages through without sanitizing these blocks, causing 400 errors in multi-turn tool-use conversations. Add _sanitize_anthropic_empty_text_blocks() to strip empty text content blocks from Anthropic-format messages before forwarding. Fixes BerriAI#22930
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile SummaryThis PR adds empty-text-block sanitization to the
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/experimental_pass_through/messages/handler.py | Adds _sanitize_anthropic_empty_text_blocks() and calls it at the top of anthropic_messages_handler to strip empty text content blocks before forwarding to upstream APIs. The logic is mostly correct for the common case (mixed tool_use + empty text), but the all-empty-blocks fallback injects a fabricated "." placeholder rather than removing the content key (inconsistent with Databricks and Anthropic chat sanitizers). No unit tests are included. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Incoming /v1/messages request] --> B[anthropic_messages async wrapper]
B --> C[_execute_pre_request_hooks\nreceives original messages]
C --> D[anthropic_messages_handler]
D --> E[validate_anthropic_api_metadata]
E --> F[_sanitize_anthropic_empty_text_blocks\nNEW]
F --> G{For each message\nwith list content}
G -->|Has empty text blocks| H[Filter out empty text blocks]
H --> I{Any blocks remaining?}
I -->|Yes| J[deepcopy msg\nset content = filtered]
I -->|No - edge case| K[deepcopy msg\nset content = dot placeholder ⚠️]
G -->|No empty blocks| L[Pass through unchanged]
J --> M[Continue to provider routing]
K --> M
L --> M
M --> N{Provider config found?}
N -->|No - OpenAI| O[Responses API]
N -->|No - other| P[Chat Completions]
N -->|Yes e.g. Anthropic/Bedrock| Q[base_llm_http_handler\nanthropic_messages_handler]
Last reviewed commit: fd12506
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if len(filtered) != len(content): | ||
| msg = deepcopy(msg) | ||
| # If all blocks were empty text, keep one with placeholder | ||
| msg["content"] = filtered if filtered else [{"type": "text", "text": "."}] |
There was a problem hiding this comment.
Injecting fabricated placeholder content
When a message's content list consists entirely of empty text blocks (all filtered out), the code replaces the content with [{"type": "text", "text": "."}]. This silently injects a fabricated "." string into the conversation history, which could alter model behaviour or confuse downstream consumers inspecting the request/response.
The existing Databricks sanitizer (_sanitize_empty_content in litellm/llms/databricks/chat/transformation.py, lines 89–92) handles the same edge case by removing the content key entirely rather than substituting a placeholder:
if not filtered:
message_dict.pop("content")A message with no content key is structurally cleaner than one that contains a fabricated sentence fragment. The same approach should be applied here to stay consistent and avoid modifying conversation semantics:
| if len(filtered) != len(content): | |
| msg = deepcopy(msg) | |
| # If all blocks were empty text, keep one with placeholder | |
| msg["content"] = filtered if filtered else [{"type": "text", "text": "."}] | |
| if len(filtered) != len(content): | |
| msg = deepcopy(msg) | |
| if filtered: | |
| msg["content"] = filtered | |
| else: | |
| msg.pop("content", None) |
| def _sanitize_anthropic_empty_text_blocks(messages: List[Dict]) -> List[Dict]: | ||
| """Remove empty text content blocks from Anthropic-format messages. | ||
|
|
||
| Claude's API can return assistant messages with empty text blocks alongside | ||
| tool_use blocks (e.g., {"type": "text", "text": ""}). These are valid in | ||
| responses but rejected when sent back in subsequent requests. This function | ||
| strips empty text blocks from content arrays, preserving all other blocks. | ||
|
|
||
| See: https://github.com/BerriAI/litellm/issues/22930 | ||
| """ | ||
| sanitized = [] | ||
| for msg in messages: | ||
| content = msg.get("content") | ||
| if isinstance(content, list): | ||
| filtered = [ | ||
| block for block in content | ||
| if not ( | ||
| isinstance(block, dict) | ||
| and block.get("type") == "text" | ||
| and not block.get("text", "").strip() | ||
| ) | ||
| ] | ||
| if len(filtered) != len(content): | ||
| msg = deepcopy(msg) | ||
| # If all blocks were empty text, keep one with placeholder | ||
| msg["content"] = filtered if filtered else [{"type": "text", "text": "."}] | ||
| sanitized.append(msg) | ||
| else: | ||
| sanitized.append(msg) | ||
| return sanitized |
There was a problem hiding this comment.
No unit tests added for new sanitization function
_sanitize_anthropic_empty_text_blocks is a pure, side-effect-free function that is straightforward to unit-test, yet no tests are included in this PR. The custom review policy requires evidence that the claimed fix actually resolves the issue (e.g. passing tests or a before/after summary). The existing test file at tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_experimental_pass_through_messages_handler.py is the natural home for these.
Suggested test cases to cover:
- A message with a mix of an empty text block and a
tool_useblock → empty block is stripped,tool_useblock remains. - A message where text is only whitespace (
"\n"," ") → treated as empty and stripped. - A message whose content is a plain string → left unchanged.
- A message with no empty blocks → returned unmodified (no unnecessary deepcopy).
- The all-empty edge case → content handled consistently (e.g. key removed instead of injected placeholder).
Context Used: Rule from dashboard - What: Ensure that any PR claiming to fix an issue includes evidence that the issue is resolved, such... (source)
|
Fix for #22930 was mergd. closing this |
3 participants
Problem
Multi-turn tool-use conversations through
/v1/messagesfail with:Claude's API returns assistant messages with empty text blocks alongside tool_use blocks (
{"type": "text", "text": ""}). These are valid in responses but rejected when sent back. The/v1/messagesendpoint passes messages through without sanitizing.Empty text sanitization already exists for
/v1/chat/completions(Bedrock #7177, Anthropic #20370, Databricks #20384) but was missing from the/v1/messagescode path.Fix
Add
_sanitize_anthropic_empty_text_blocks()in the/v1/messageshandler to strip empty text content blocks before forwarding to the upstream API.Fixes #22930