fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315)

[tin-berri]

TLDR

Problem this solves:

• Replayed intercepted web search turns 400 on Bedrock
• Snippet text never reached the client, forcing a fetch per URL
• A resultless or failed search still 400s the next turn
• No test anywhere replayed an intercepted search turn

How it solves it:

• Carry the snippet on each web_search_result
• Pair each result block with its own srvtoolu_ server_tool_use
• Rewrite replayed unencrypted blocks to text before dispatch
• Flatten an empty result list too, not only a populated one
• Assert the outbound Bedrock body over both cases

User Flow

Before: someone using a native Anthropic client against a Bedrock-backed gateway with web search interception on loses the conversation on the turn after any search

1. They ask a question needing current information, and the gateway runs the search and answers correctly
2. Their client stores the assistant turn it was handed, which lists each source as a url and a title with no page text
3. They send a follow-up message, and the client replays that stored assistant turn as the protocol requires
4. The gateway returns 400 with Input tag 'server_tool_use' found using 'type' does not match any of the expected tags: 'document', 'image', 'redacted_thinking', 'search_result', 'text', 'thinking', 'tool_result', 'tool_use'
5. Every later message in that conversation fails the same way, because the client keeps replaying the same turn
6. Deleting the search turn from the history by hand makes the request succeed, at the cost of the sources

After: the same follow-up succeeds, and the model can still answer from the search evidence

1. They ask the same question and get the same answer
2. Their client stores an assistant turn whose sources now carry the snippet text alongside the url and title
3. They send a follow-up message, and the client replays that stored assistant turn unchanged
4. The gateway returns 200 and the model answers from the replayed evidence rather than fetching each url again
5. This holds when the search found nothing and when the search failed, where the turn previously still returned 400

Relevant issues

Linear ticket

Resolves LIT-5315
Resolves LIT-5320

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Three live proxies against real Bedrock, bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0 in us-east-1, billed to a real account. Config is the one this feature ships for:

model_list:
  - model_name: claude-sonnet-bedrock
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
      aws_region_name: us-east-1

litellm_settings:
  callbacks: ["websearch_interception"]
  websearch_interception_params:
    enabled_providers: ["bedrock"]

The replayed assistant turn is generated by WebSearchTransformation.build_web_search_tool_result_block itself rather than written by hand, so it is byte-for-byte what a native client would be handed:

$ curl -s -X POST http://127.0.0.1:$PORT/v1/messages \
    -H "x-api-key: $KEY" -H 'anthropic-version: 2023-06-01' \
    -H 'content-type: application/json' -d @messages_replay_with_results.json

Each request replays one assistant turn holding server_tool_use + web_search_tool_result and then asks Reply with only the year you just gave me.

Before, at 1e7b39d15c (litellm_internal_staging), port 4071

REPLAY, search returned results              HTTP 400
{"message":"messages.1.content.0: Input tag 'server_tool_use' found using 'type' does not match
 any of the expected tags: 'document', 'image', 'redacted_thinking', 'search_result', 'text',
 'thinking', 'tool_result', 'tool_use'"}

REPLAY, search returned nothing              HTTP 400
{"message":"messages.1.content.0: Input tag 'server_tool_use' found using 'type' does not match ..."}

CONTROL, same turn minus the search blocks   HTTP 200
{"content":[{"type":"text","text":"753 BC"}],"usage":{"input_tokens":620,"output_tokens":7}}

The control answers correctly on the unfixed build, so the 400 is attributable to the replayed block and to nothing else in the request.

Intermediate, at 878c1278e2, port 4072

REPLAY, search returned results              HTTP 200   {"text":"753 BC"}
REPLAY, search returned nothing              HTTP 400   Input tag 'server_tool_use' ...
CONTROL                                      HTTP 200   {"text":"753 BC"}

The resultless case was still dead here, which is what b9c0eee88b fixes.

After, at b9c0eee88b, port 4073

REPLAY, search returned results              HTTP 200
{"id":"msg_bdrk_01MYUypwKX64svUGJFX6h7JS","content":[{"type":"text","text":"753 BC"}],
 "usage":{"input_tokens":651,"output_tokens":7}}

REPLAY, search returned nothing              HTTP 200
{"id":"msg_bdrk_01NxXE35hjenKFMw3fxnEk6w","content":[{"type":"text","text":"753 BC"}],
 "usage":{"input_tokens":637,"output_tokens":7}}

CONTROL, same turn minus the search blocks   HTTP 200
{"id":"msg_bdrk_018g4Ym49QcnAT5hFnF9UmpS","content":[{"type":"text","text":"753 BC"}],
 "usage":{"input_tokens":620,"output_tokens":7}}

The 651 vs 620 input-token gap on the results-present run is the flattened search evidence still being carried into the prompt, so the model is answering from the sources rather than from a stripped history.

Type

🐛 Bug Fix

Changes

build_web_search_tool_result_block carries snippet on each result, and emits its own srvtoolu_ server_tool_use paired with each result block, since a bare result block is rejected on replay.

flatten_unencrypted_web_search_results_in_anthropic_messages rewrites a replayed block that carries no encrypted_content into a text block holding the same title, url and snippet, and drops the paired server_tool_use with it. Blocks Anthropic itself issued keep a real encrypted_content and are left untouched, so native Anthropic behaviour is unchanged.

An empty content list flattens on the same path. That is what the interceptor emits both when a search legitimately returns nothing and when a search raises, and it holds neither evidence to preserve nor an encrypted_content to respect, so leaving it in place only bought back the 400 the flatten exists to avoid. The rendered text says No results were returned. rather than emitting a bare header.

Tests: the outbound Bedrock invoke body is asserted free of both block types, parametrized over the results-present and resultless cases, built from the interceptor's own builder so the fixture cannot drift from what it emits. Mutation checked per site: restoring the empty-content bail fails 3, dropping the resultless rendering fails 3, restored 7 pass. Affected suites 916 passed, 2 skipped.

Scope note: the /chat/completions surfaces mishandle a replayed search turn too, and are deliberately left alone. Interception never mints srvtoolu_ ids or web_search_tool_result blocks there, so reaching them needs a cross-provider replay where a client runs an Anthropic-native search and later replays that history against Bedrock. Confirmed live for completeness: /chat/completions returns 400 "tool_use ids were found without tool_result blocks immediately after", and Bedrock Converse silently drops the search turn instead. Both want their own tickets

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes conversation shaping on every /v1/messages request with replayed search history; incorrect flattening could drop citations or alter multi-turn behavior, though real Anthropic encrypted blocks are explicitly preserved.

Overview
Fixes multi-turn failures when native Anthropic clients replay intercepted web-search assistant turns against Bedrock-backed /v1/messages.

Websearch interception now emits a spec-correct server_tool_use + web_search_tool_result pair per search (shared srvtoolu_ id) instead of a lone result block tied to the model’s toolu_ id. Each web_search_result also carries an additive snippet field so clients and follow-ups have page text without relying on encrypted_content.

Before dispatch, flatten_unencrypted_web_search_results_in_anthropic_messages rewrites LiteLLM-synthesized blocks (no real encrypted_content) into plain text with title/URL/snippet, drops the paired server_tool_use, and handles empty result lists the same way—so Bedrock no longer 400s on unsupported server_tool_use / web_search_tool_result tags while preserving evidence. Genuine Anthropic blocks with encrypted_content are left unchanged.

Wired into the Anthropic messages handler (async + sync paths) alongside existing message sanitizers; tests cover snippets, native block pairs, flattening edge cases, and Bedrock outbound bodies.

^{Reviewed by Cursor Bugbot for commit f3958ab. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

The PR preserves intercepted web-search snippets and converts replayed unencrypted native search blocks into provider-compatible text. It also pairs emitted results with server tool-use blocks, handles empty searches, and verifies repeated flattening remains idempotent

• Carries snippet evidence in synthesized web-search results
• Flattens replayed result pairs before Anthropic Messages dispatch
• Covers populated, empty, failed, and repeated-flattening cases

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains

Important Files Changed

Filename	Overview
litellm/integrations/websearch_interception/handler.py	Emits paired server-tool-use and web-search-result blocks with generated matching identifiers
litellm/integrations/websearch_interception/transformation.py	Preserves each search result’s snippet in the synthesized native result block
litellm/llms/anthropic/common_utils.py	Rewrites unencrypted replayed search results into text while preserving encrypted native blocks
litellm/llms/anthropic/experimental_pass_through/messages/handler.py	Applies replay flattening consistently across asynchronous and synchronous dispatch paths
tests/test_litellm/llms/anthropic/test_anthropic_common_utils.py	Covers evidence preservation, resultless searches, native encrypted blocks, and repeated idempotent flattening
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py	Confirms Bedrock receives text evidence without unsupported native search block types

_{Reviews (2): Last reviewed commit: "test(websearch): pin flatten idempotency..." | Re-trigger Greptile}

[tin-berri]

bugbot run

[codecov]

Codecov Report

❌ Patch coverage is 97.89474% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...llm/integrations/websearch_interception/handler.py	92.85%	1 Missing ⚠️
litellm/llms/anthropic/common_utils.py	98.57%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_websearch_snippet (f3958ab) with litellm_internal_staging (cb211b5)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (8db2fba) during the generation of this report, so cb211b5 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[devin-ai-integration]

Pushed 878c127: snippet alone still 400s on replay, so blocks now pair with a srvtoolu_ server_tool_use and unencrypted results flatten to text. Verified live turn 1 and turn 2

[yassin-berriai]

Gap: _flattenable_web_search_tool_result bails on empty content, so a zero-result or failed search still ships web_search_tool_result to Bedrock and 400s on replay

[yassin-berriai]

@greptileai pushed b9c0eee88b closing that gap: empty content now flattens, plus the multi-turn replay test. Verified live on real Bedrock, 400 to 200

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ tin-berri
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

The fix looks correct. The three-part solution is coherent:

1. srvtoolu_-prefixed IDs — fresh UUID per pair, satisfying Anthropic's pattern validation on replay.
2. Flatten on ingress — _flattenable_web_search_tool_result correctly identifies synthesized blocks by the absence of encrypted_content, rewrites them to text, and drops the orphaned server_tool_use. Real Anthropic blocks (non-empty encrypted_content) pass through untouched.
3. Empty content handled — the docstring on _flattenable_web_search_tool_result explains the reasoning, and test_flatten_unencrypted_web_search_results_flattens_a_resultless_search covers both the None-response and zero-result cases.

The Bedrock integration test is the right regression anchor — it exercises the actual transform_anthropic_messages_request path rather than mocking it, so drift between the flattener output and what Bedrock accepts will break the test immediately.

One minor observation: _rewrite_replayed_web_search_block uses flattenable both as the map of results and as the gate for dropping the server_tool_use. That's fine because the server_tool_use id is in flattenable only when its paired result was flattenable, so the drop condition is precise. No issue there.

LGTM.

[veria-ai]

Low: Search snippets bypass output guardrails

This exposes raw page text in a web_search_tool_result, but the Anthropic output-guardrail translator only extracts text and tool_use blocks. An authenticated caller can choose a query whose result contains prohibited data and receive the snippet unchanged even when the synthesized answer is blocked or masked. Extend the guardrail translation to scan these snippets and apply transformed values back before returning them.

[yassin-berriai]

Real and confirmed: extraction is limited to text/input_text/output_text, so nested snippets are never scanned. Tracked as LIT-5321, separate from the replay fix

[veria-ai]

PR overview

This pull request restores search-result snippet text in native web_search_tool_result blocks within the web search interception transformation.

One security issue remains open: snippet text is returned without passing through the configured Anthropic output guardrails. An authenticated caller could therefore retrieve prohibited content from chosen search results even when the synthesized response is blocked or masked, and no issues have yet been addressed.

Open issues (1)

• Low: Search snippets bypass output guardrails — litellm/integrations/websearch_interception/transformation.py:450

Fixed/addressed: 0 · PR risk: 5/10

[yassin-berriai]

@greptileai please review the current head f3958abe8d. Adds an idempotency test: the agentic loop re-flattens the same history each iteration

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f3958ab. Configure here.}