fix: strip empty text content blocks in /v1/messages endpoint - #24030
fix: strip empty text content blocks in /v1/messages endpoint#24030saurabhghere wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
da44c4d to
d86bf4b
Compare
Greptile SummaryThis PR re-implements the empty-text-block sanitization (fix for #22930) in the correct architectural layer. Claude's API returns assistant messages containing Key points:
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/experimental_pass_through/messages/transformation.py | Adds _sanitize_empty_text_content_blocks() method to AnthropicMessagesConfig and calls it at the top of transform_anthropic_messages_request(), correctly fixing the empty-text-block rejection issue for Anthropic, Bedrock (via direct call), Vertex AI, and Azure AI (via super() calls). Minor concerns already surfaced in prior review threads (user-message scope and silent placeholder). |
| tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_v1_messages_empty_text_sanitization.py | New test file with 12 unit tests covering the sanitization method: the primary bug scenario, whitespace-only text, all-empty replacement, string/no-content pass-through, user messages, tool_result safety, None values, immutability, multi-turn end-to-end, and empty list. Tests are mock-only and correctly importfrom the new location. |
| poetry.lock | Only the content-hash line changed — no new dependencies introduced. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[transform_anthropic_messages_request called\nAnthropic / Bedrock / Vertex / Azure] --> B[_sanitize_empty_text_content_blocks]
B --> C{For each message}
C --> D{content is a list?}
D -- No --> E[Append message unchanged]
E --> C
D -- Yes --> F[Filter: remove blocks where\ntype==text AND text is empty/whitespace]
F --> G{Any blocks filtered?}
G -- No --> H[Append original message]
H --> C
G -- Yes --> I{filtered list empty?}
I -- No --> J[Append message with filtered content list]
I -- Yes --> K[Replace with placeholder\ntype=text, text='...']
K --> J
J --> C
C -- Done --> L[Continue with sanitized messages\ne.g. max_tokens check, AnthropicMessagesRequest construction]
Last reviewed commit: "Update poetry lock"
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if filtered_content != content: | ||
| if not filtered_content: | ||
| filtered_content = [{"type": "text", "text": "..."}] | ||
| result.append({**message, "content": filtered_content}) | ||
| else: | ||
| result.append(message) |
There was a problem hiding this comment.
When every block in a content list is an empty text block, the code replaces them with the hardcoded string "...":
if not filtered_content:
filtered_content = [{"type": "text", "text": "..."}]This silently inserts a "..." into the conversation without any logging or warning, and it applies equally to user messages and assistant messages. A caller debugging a multi-turn conversation would see unexpected content appear in the request. Consider at least emitting a verbose_logger.warning(...) here to make the substitution observable. Additionally, the placeholder "..." is somewhat arbitrary — a comment explaining why this specific string was chosen would help future maintainers.
| filtered_content = [ | ||
| block | ||
| for block in content | ||
| if not ( | ||
| isinstance(block, dict) | ||
| and block.get("type") == "text" | ||
| and (not block.get("text") or not str(block["text"]).strip()) | ||
| ) | ||
| ] |
There was a problem hiding this comment.
The method iterates over every message regardless of role, so user-role messages with list content are also sanitized. The bug being fixed (issue #22930) is specifically about assistant messages that come back from Claude with empty text blocks alongside tool_use blocks. Silently stripping empty text blocks from user messages is a broader change than the fix requires, and user messages normally would never contain Anthropic-generated empty blocks. Scoping the filter to role == "assistant" would make the intent more explicit and avoid unintended side-effects on user-constructed messages:
| filtered_content = [ | |
| block | |
| for block in content | |
| if not ( | |
| isinstance(block, dict) | |
| and block.get("type") == "text" | |
| and (not block.get("text") or not str(block["text"]).strip()) | |
| ) | |
| ] | |
| if not isinstance(content, list) or message.get("role") != "assistant": |
(The test test_user_message_content_also_sanitized covers this path and would need updating if the scope is narrowed, which might be desirable.)
e35e728 to
71e5dda
Compare
|
FIx for #22930 was merged. Closing this |
2 participants
Relevant issues
Fixes #22930
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🐛 Bug Fix
Changes
Moves empty text content block sanitization from the generic HTTP handler (llm_http_handler.py) into the Anthropic-specific provider config
(AnthropicMessagesConfig.transform_anthropic_messages_request()).
Problem: Claude's API returns assistant messages with empty text blocks ({"type": "text", "text": ""}) alongside tool_use blocks in multi-turn conversations. While
Anthropic returns these, it rejects them when sent back, causing 400: text content blocks must be non-empty. The previous fix (PR #23097) was reverted (PR #23232)
because it crashed on bare string messages. The current v1 fix works but places Anthropic-specific logic in the generic HTTP handler.
What changed:
Why this is better: