fix: WebSearch interception returns native Anthropic format for Claude Code compatibility - #25242
fix: WebSearch interception returns native Anthropic format for Claude Code compatibility#252420xxmemo wants to merge 8 commits into
Conversation
…l parameters - Updated the WebSearchInterceptionLogger to extract and utilize the api_base parameter from the search tool's litellm_params. - Modified the search execution logic to pass the api_base along with the search_provider to the asearch function, improving flexibility in API interactions. - Ensured backward compatibility by maintaining existing functionality while adding the new parameter handling.
…se format - Updated the WebSearchInterceptionLogger to execute web searches through the configured provider and return responses in a native Anthropic format. - Implemented structured parsing of search results into hits for the web_search_tool_result, ensuring compatibility with Claude Code's WebSearchTool parser. - Removed legacy checks for providers with native Anthropic Messages support, streamlining the short-circuiting logic for web search requests.
- Changed the standard web search tool name from "litellm_web_search" to "WebSearch" for consistency. - Updated the documentation and descriptions in the web search tool functions to reflect the new naming and improve clarity. - Enhanced the logic for identifying web search tools to accommodate both the new and legacy names, ensuring compatibility across different formats.
…hropicMessagesStreamIterator - Implemented handling for "server_tool_use" and "web_search_tool_result" block types in the FakeAnthropicMessagesStreamIterator. - Added logic to emit content_block_start and content_block_stop events for both block types, aligning with Anthropic's native streaming format. - Enhanced the iterator's functionality to support new message types, improving integration with external tools.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
…ormat - Update test_websearch_short_circuit.py to validate native Anthropic response format (server_tool_use + web_search_tool_result) instead of plain text - Add tests for structured search hits, tool_use_id linking, usage tracking, and uniform provider handling - Fix server_tool_use/web_search_tool_result block handling in _create_content_block_chunks method (was orphaned from cherry-pick) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Providers with native Anthropic Messages support (anthropic, bedrock, vertex_ai) skip the short-circuit and let their API handle web_search_20250305 natively. Only non-native providers (github_copilot, etc.) get the synthetic native-format response. This preserves upstream behavior: Anthropic's API handles web search server-side for providers that support it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Greptile SummaryThis PR fixes web search interception for Claude Code compatibility by making three targeted changes: (1) renaming
Confidence Score: 4/5Safe for new Claude Code deployments; existing users relying on 'litellm_web_search' as the injected tool name in model responses face a silent wire-format break. The core logic is sound and well-tested with mocked unit tests. Score is 4 rather than 5 because of the backwards-incompatible tool name rename (rule b48b7341) — existing proxy deployments inspecting or matching 'litellm_web_search' in responses will silently receive 'WebSearch' instead. All other findings are P2 minor style/observation issues that don't block merge. litellm/constants.py (wire-format rename) and litellm/integrations/websearch_interception/handler.py (trailing text block in short-circuit response)
|
| Filename | Overview |
|---|---|
| litellm/integrations/websearch_interception/handler.py | Core short-circuit rewritten to return native Anthropic format (server_tool_use + web_search_tool_result); a trailing text block in all responses may interfere with Claude Code's synthesis step |
| litellm/llms/anthropic/experimental_pass_through/messages/fake_stream_iterator.py | Adds server_tool_use and web_search_tool_result streaming via content_block_start events, matching Anthropic's SSE protocol for server-side tool blocks |
| litellm/integrations/websearch_interception/tools.py | Detection set updated to recognize both WebSearch and litellm_web_search for backwards compat; minor redundancy in _WEB_SEARCH_NAMES with LITELLM_WEB_SEARCH_TOOL_NAME == 'WebSearch' |
| litellm/integrations/websearch_interception/transformation.py | No functional changes; detection already accepts both old and new tool names via hardcoded list |
| litellm/constants.py | LITELLM_WEB_SEARCH_TOOL_NAME renamed from 'litellm_web_search' to 'WebSearch' — potentially breaks existing deployments inspecting tool names in responses |
| tests/test_litellm/integrations/websearch_interception/test_websearch_short_circuit.py | Tests correctly updated for new 3-block content structure; all use AsyncMock with no real network calls; good coverage of native format, streaming, and error paths |
Sequence Diagram
sequenceDiagram
participant CC as Claude Code
participant LLP as LiteLLM Proxy
participant SC as try_short_circuit_search
participant SE as _execute_search
participant FS as FakeStreamIterator
CC->>LLP: POST /v1/messages {tools: [web_search_20250305], stream: true}
LLP->>SC: try_short_circuit_search(model, messages, tools)
Note over SC: Skips native providers (anthropic, bedrock, vertex_ai)
SC->>SE: _execute_search(query)
SE-->>SC: "Title: ...\nURL: ...\nSnippet: ..."
SC->>SC: Parse into search_hits[]
SC-->>LLP: {server_tool_use + web_search_tool_result + text}
LLP->>FS: FakeAnthropicMessagesStreamIterator(response)
FS-->>CC: SSE: message_start
FS-->>CC: SSE: content_block_start (server_tool_use)
FS-->>CC: SSE: content_block_stop
FS-->>CC: SSE: content_block_start (web_search_tool_result)
FS-->>CC: SSE: content_block_stop
FS-->>CC: SSE: content_block_start (text)
FS-->>CC: SSE: content_block_delta
FS-->>CC: SSE: content_block_stop
FS-->>CC: SSE: message_delta + message_stop
CC->>CC: WebSearchTool.makeOutputFromSearchResponse()
Note over CC: Parses server_tool_use + web_search_tool_result correctly
Greploops — Automatically fix all review issues by running /greploops in Claude Code. It iterates: fix, push, re-review, repeat until 5/5 confidence.
Use the Greptile plugin for Claude Code to query reviews, search comments, and manage custom context directly from your terminal.
Reviews (2): Last reviewed commit: "Fix review feedback: encrypted_content a..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| # LiteLLM standard web search tool name | ||
| # Used for web search interception across providers | ||
| LITELLM_WEB_SEARCH_TOOL_NAME = "litellm_web_search" | ||
| LITELLM_WEB_SEARCH_TOOL_NAME = "WebSearch" |
There was a problem hiding this comment.
Renaming LITELLM_WEB_SEARCH_TOOL_NAME from "litellm_web_search" to "WebSearch" is a breaking change for existing users. This constant controls the name of the tool injected into model requests via get_litellm_web_search_tool() and get_litellm_web_search_tool_openai(). Any user who:
- Has code checking tool names in model responses for
"litellm_web_search" - Has hardcoded the tool name in downstream tooling
- Is testing that the model uses the
"litellm_web_search"tool
...will silently break because model responses will now contain "WebSearch" instead.
While is_web_search_tool() and the detection logic in transformation.py cover both names for detection, the change in the injected tool name changes the wire format for all existing deployments. Per the project style guide, backwards-incompatible changes should use a user-controlled flag.
Suggested approach: keep LITELLM_WEB_SEARCH_TOOL_NAME = "litellm_web_search" as the default and add a separate constant or config flag for Claude Code compatibility. If the rename is intentional, a release note warning users is strongly recommended.
Rule Used: What: avoid backwards-incompatible changes without... (source)
| hit: Dict[str, Any] = { | ||
| "type": "web_search_result", | ||
| "url": url, | ||
| "title": title or url, | ||
| "encrypted_content": "", | ||
| "page_age": None, | ||
| } | ||
| if snippet: | ||
| hit["snippet"] = snippet |
There was a problem hiding this comment.
encrypted_content always empty — model may not see search content
The web_search_result objects are built with "encrypted_content": "". In Anthropic's native format, encrypted_content is an opaque, server-encrypted payload containing the page content that the model reads when processing search results. Setting it to an empty string means the language model won't have access to the actual page content through this field — it relies only on the separate text block at the top level.
Claude Code's WebSearchTool parser reads encrypted_content to surface page content to the LLM in the follow-up turn. With an empty value, search results will show titles and URLs but the model won't process page text, potentially producing low-quality answers even when searches succeed.
Since LiteLLM can't replicate Anthropic's encryption, consider putting the snippet text directly in the encrypted_content field (unencrypted) or verifying that the downstream Claude Code version handles an empty value gracefully.
| ] | ||
|
|
||
| response: Dict[str, Any] = { | ||
| "id": f"msg_{str(uuid.uuid4())}", |
There was a problem hiding this comment.
f"msg_{str(uuid.uuid4())}" generates IDs with hyphens (e.g., msg_<hex>-<hex>-<hex>-<hex>-<hex>). Anthropic's real message IDs use a compact alphanumeric format without hyphens (e.g., msg_01XFDUDYJgAACTJCRiKey6u9). Clients that strictly validate the message ID format may reject this synthetic response.
The tool_use_id on line 184 correctly strips hyphens: str(uuid.uuid4()).replace('-', '')[:24]. Apply the same treatment here:
| "id": f"msg_{str(uuid.uuid4())}", | |
| "id": f"msg_{str(uuid.uuid4()).replace('-', '')[:20]}", |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
- Put snippet text in encrypted_content so the model can read page content from search results (empty string meant the model only saw titles/URLs) - Strip hyphens from synthetic message ID to match Anthropic's compact alphanumeric format (msg_<hex> not msg_<hex>-<hex>-...) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Addressed the review feedback: P1 (tool name rename): This is intentional — P2 (encrypted_content): Fixed in 573b02a — snippet text is now placed in P2 (message ID format): Fixed in 573b02a — hyphens stripped to match Anthropic's compact |
…t blocks) with bedrock short-circuit enabled Combines PR BerriAI#25242 (native server_tool_use/web_search_tool_result format for the websearch_interception short-circuit) with a fix to not skip short-circuit for bedrock — bedrock speaks Anthropic Messages but does not support web_search_20250305 natively. Reference build for testing; not for upstream as-is (PR BerriAI#25242 work by its original author).
2 participants
Relevant issues
Fixes web search interception not working with Claude Code CLI (#24143 follow-up)
Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
🐛 Bug Fix
Changes
WebSearch interception does not work with Claude Code because the response format, tool naming, and streaming do not match what Claude Code expects. Three bugs, one root cause: the interception pipeline was built for programmatic
litellm.messages.acreate()callers, not for Claude Code's streamingWebSearchTool.Problem
When Claude Code sends a
web_search_20250305server tool request through a LiteLLM proxy:Wrong tool name — Interception registers the tool as
litellm_web_search. Claude Code checks forWebSearch. The tool shows up with the wrong name, so Claude Code never recognizes it as its native web search.Wrong response format —
try_short_circuit_searchreturns{type: "text"}. Claude Code'smakeOutputFromSearchResponseexpects native Anthropic format:server_tool_use+web_search_tool_resultcontent blocks with structured search hits.Missing streaming support —
FakeAnthropicMessagesStreamIteratoronly handlestext,thinking,redacted_thinking, andtool_useblocks. It silently dropsserver_tool_useandweb_search_tool_result. Since Claude Code usesqueryModelWithStreaming, the search results never arrive — producing "Did 0 searches in Xs".Fix
constants.py— RenameLITELLM_WEB_SEARCH_TOOL_NAMEfrom"litellm_web_search"to"WebSearch"to match Claude Code's expected tool name.handler.py—try_short_circuit_searchreturns native Anthropic format (server_tool_use+web_search_tool_resultwith structured hits). Also passesapi_basefrom search tool config tolitellm.asearch().tools.py/transformation.py— Detection functions recognize both"WebSearch"and"litellm_web_search"for backwards compatibility.fake_stream_iterator.py— Handleserver_tool_useandweb_search_tool_resultblock types so they stream as proper SSE events.How it works now
web_search_20250305to LiteLLM proxytry_short_circuit_searchdetects search-only request, executes via configured provider (SearXNG/Tavily/Perplexity)FakeAnthropicMessagesStreamIteratorstreams it as SSEWebSearchToolparses correctlyAll providers (Anthropic, Bedrock, Vertex, third-party) go through the same single funnel.
Tested with
claude -pwith web search queriescurl /v1/messageswithweb_search_20250305toollitellm_web_searchname still detected