fix(anthropic): stop repeating streamed thinking text in the signature chunk

[devin-ai-integration]

TLDR

Problem this solves:

• Streamed Claude thinking text was saved twice under one signature
• Every additive client (Agents SDK, the Responses bridge) stored the doubled block
• The next turn replayed the doubled thinking to Claude on every later request

How it solves it:

• The signature chunk now carries a signature-only thinking block
• Thinking text streams once, in the per-delta blocks, like Bedrock already does
• Accumulators rebuild one block with the text once plus the signature

User Flow

Before: a developer whose Agents SDK app runs Claude with extended thinking saves a tool-call turn and finds the thinking text doubled under one signature

1. Their agent (LitellmModel("anthropic/claude-sonnet-4-5"), Reasoning(effort="medium")) streams a Claude turn that thinks, then calls get_weather and get_local_time for Paris
2. They save result.to_input_list() to JSON and open the reasoning item: thinking_blocks[0].thinking is twice as long as the thinking they watched stream, the same text repeated once, under one signature
3. On the next turn the app replays that block to POST https://api.anthropic.com/v1/messages; Claude accepts it, so every later turn pays input tokens for the thinking twice and any UI of the saved history shows it twice
4. The same turn through the proxy, POST https://litellm-domain/v1/responses with "stream": true, returns a reasoning item whose encrypted_content holds the doubled text

After: the same turn saves the thinking text exactly once

1. Their agent (LitellmModel("anthropic/claude-sonnet-4-5"), Reasoning(effort="medium")) streams a Claude turn that thinks, then calls get_weather and get_local_time for Paris
2. They save result.to_input_list() to JSON and open the reasoning item: thinking_blocks[0].thinking equals the thinking they watched stream, once, with the same signature
3. On the next turn the app replays that block to POST https://api.anthropic.com/v1/messages; Claude accepts it and the history carries each thought once
4. The same turn through the proxy, POST https://litellm-domain/v1/responses with "stream": true, returns a reasoning item whose encrypted_content holds the text once

Linear ticket

Resolves LIT-9150

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Last updated: 775fe9f. QA and /live-pr-risk ran on 1955444, the last commit with a product diff; the commits since are the audit's tests-only commits 6e40e47 and 775fe9f

QA ran at 1955444, the last commit with a product diff. Before = merge base 3f39fef, After = 1955444. Each leg is its own worktree and venv, its own proxy booted with litellm --config proxy_config.yaml --host 127.0.0.1 --port <random port> --num_workers 2 and no database, real Anthropic calls to claude-sonnet-4-5, OpenAI Agents SDK 0.22.3 on openai 3.20.0. The scripts and curls below are identical for both legs; only the proxy's commit differs

proxy_config.yaml

model_list:
  - model_name: claude-sonnet-4-5
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Case 1: Agents SDK LitellmModel("anthropic/claude-sonnet-4-5"), in process

The app streams a Paris tool-call turn, saves result.to_input_list() to JSON, reloads it, and runs a Rome turn on top. thinking chars is the saved provider_data.thinking_blocks[0].thinking the SDK will replay to Claude; streamed is the reasoning the user watched arrive

case1_sdk_litellm_model.py

import asyncio, json, pathlib, sys
from collections import defaultdict

from agents import Agent, ModelSettings, Runner, function_tool, set_tracing_disabled
from agents.extensions.models.litellm_model import LitellmModel
from openai.types.shared import Reasoning

set_tracing_disabled(True)
OUT = pathlib.Path(sys.argv[1]); OUT.mkdir(parents=True, exist_ok=True)
MODEL = "anthropic/claude-sonnet-4-5"


@function_tool
def get_weather(city: str) -> str:
    return f"The weather in {city} is 22C and sunny."


@function_tool
def get_local_time(city: str) -> str:
    return f"The local time in {city} is 14:05."


def agent() -> Agent:
    return Agent(
        name="Assistant",
        instructions="Use get_weather and get_local_time whenever a city is mentioned. Answer briefly.",
        model=LitellmModel(model=MODEL),
        model_settings=ModelSettings(reasoning=Reasoning(effort="medium")),
        tools=[get_weather, get_local_time],
    )


def same_text_twice(text: str) -> bool:
    half = len(text) // 2
    return half > 0 and len(text) % 2 == 0 and text[:half] == text[half:]


def describe_saved_reasoning(items: list, streamed_chars: int) -> None:
    for index, item in enumerate(items):
        if item.get("type") != "reasoning":
            continue
        summary_chars = sum(len(s.get("text", "")) for s in item.get("summary") or [])
        print(f"  saved[{index}] reasoning item: summary chars={summary_chars}")
        for block in (item.get("provider_data") or {}).get("thinking_blocks") or []:
            text = block.get("thinking", "")
            ratio = f"{len(text) / streamed_chars:.2f}x" if streamed_chars else "n/a"
            print(f"    provider_data.thinking_blocks[]: type={block.get('type')} thinking chars={len(text)} "
                  f"({ratio} the streamed reasoning) same_text_twice={same_text_twice(text)} signed={bool(block.get('signature'))}")


async def turn(label: str, history: list, user_text: str) -> list:
    print(f"\n=== {label}: {MODEL} <- {user_text!r}")
    result = Runner.run_streamed(agent(), input=history + [{"role": "user", "content": user_text}])
    streamed = defaultdict(str)
    async for ev in result.stream_events():
        if ev.type == "raw_response_event":
            kind = getattr(ev.data, "type", "")
            delta = getattr(ev.data, "delta", None)
            if "reasoning" in kind and isinstance(delta, str):
                streamed[kind] += delta
        elif ev.type == "run_item_stream_event":
            item = ev.item
            detail = ""
            if item.type == "tool_call_item":
                detail = f" {item.raw_item.name}({item.raw_item.arguments})"
            elif item.type == "tool_call_output_item":
                detail = f" -> {item.output!r}"
            print(f"  item: {item.type}{detail}")
    print(f"  final_output: {result.final_output!r}")
    for kind, text in streamed.items():
        print(f"  streamed {kind}: {len(text)} chars")
    streamed_chars = max((len(text) for text in streamed.values()), default=0)
    saved = OUT / f"{label}.json"
    saved.write_text(json.dumps(result.to_input_list(), indent=1))
    reloaded = json.loads(saved.read_text())
    print(f"  saved+reloaded {len(reloaded)} items -> {saved.name}")
    describe_saved_reasoning(reloaded, streamed_chars)
    return reloaded


async def main() -> None:
    paris = await turn("t1_paris", [], "What's the weather and local time in Paris?")
    await turn("t2_rome_on_reloaded_history", paris, "Now the same two things for Rome.")


asyncio.run(main())

$ .venv/bin/python case1_sdk_litellm_model.py out/case1

Before (3f39fef): the saved block is the streamed thinking twice, and the Rome turn replays the doubled block

=== t1_paris: anthropic/claude-sonnet-4-5 <- "What's the weather and local time in Paris?"
  item: reasoning_item
  item: tool_call_item get_weather({"city": "Paris"})
  item: tool_call_item get_local_time({"city": "Paris"})
  item: tool_call_output_item -> 'The weather in Paris is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Paris is 14:05.'
  item: message_output_item
  final_output: "In Paris, it's currently **14:05** (2:05 PM) and the weather is **22°C and sunny**."
  streamed response.reasoning_summary_text.delta: 429 chars
  saved+reloaded 7 items -> t1_paris.json
  saved[1] reasoning item: summary chars=429
    provider_data.thinking_blocks[]: type=thinking thinking chars=858 (2.00x the streamed reasoning) same_text_twice=True signed=True

=== t2_rome_on_reloaded_history: anthropic/claude-sonnet-4-5 <- 'Now the same two things for Rome.'
  item: reasoning_item
  item: tool_call_item get_weather({"city": "Rome"})
  item: tool_call_item get_local_time({"city": "Rome"})
  item: tool_call_output_item -> 'The weather in Rome is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Rome is 14:05.'
  item: message_output_item
  final_output: "In Rome, it's currently **14:05** (2:05 PM) and the weather is **22°C and sunny**."
  streamed response.reasoning_summary_text.delta: 119 chars
  saved+reloaded 14 items -> t2_rome_on_reloaded_history.json
  saved[1] reasoning item: summary chars=429
    provider_data.thinking_blocks[]: type=thinking thinking chars=858 (7.21x the streamed reasoning) same_text_twice=True signed=True
  saved[8] reasoning item: summary chars=119
    provider_data.thinking_blocks[]: type=thinking thinking chars=238 (2.00x the streamed reasoning) same_text_twice=True signed=True

After (1955444): the saved block equals the streamed thinking, once, still signed, and the Rome turn succeeds on it

=== t1_paris: anthropic/claude-sonnet-4-5 <- "What's the weather and local time in Paris?"
  item: reasoning_item
  item: tool_call_item get_weather({"city": "Paris"})
  item: tool_call_item get_local_time({"city": "Paris"})
  item: tool_call_output_item -> 'The weather in Paris is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Paris is 14:05.'
  item: message_output_item
  final_output: "In Paris, it's currently 14:05 (2:05 PM) and the weather is sunny with a temperature of 22°C."
  streamed response.reasoning_summary_text.delta: 294 chars
  saved+reloaded 7 items -> t1_paris.json
  saved[1] reasoning item: summary chars=294
    provider_data.thinking_blocks[]: type=thinking thinking chars=294 (1.00x the streamed reasoning) same_text_twice=False signed=True

=== t2_rome_on_reloaded_history: anthropic/claude-sonnet-4-5 <- 'Now the same two things for Rome.'
  item: reasoning_item
  item: tool_call_item get_weather({"city": "Rome"})
  item: tool_call_item get_local_time({"city": "Rome"})
  item: tool_call_output_item -> 'The weather in Rome is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Rome is 14:05.'
  item: message_output_item
  final_output: "In Rome, it's currently 14:05 (2:05 PM) and the weather is sunny with a temperature of 22°C."
  streamed response.reasoning_summary_text.delta: 97 chars
  saved+reloaded 14 items -> t2_rome_on_reloaded_history.json
  saved[1] reasoning item: summary chars=294
    provider_data.thinking_blocks[]: type=thinking thinking chars=294 (3.03x the streamed reasoning) same_text_twice=False signed=True
  saved[8] reasoning item: summary chars=97
    provider_data.thinking_blocks[]: type=thinking thinking chars=97 (1.00x the streamed reasoning) same_text_twice=False signed=True

Case 2: Agents SDK OpenAIResponsesModel through the proxy, POST http://127.0.0.1:<port>/v1/responses with "stream": true

Same script, with set_default_openai_client(AsyncOpenAI(base_url="http://127.0.0.1:<port>/v1", api_key=os.environ["LITELLM_MASTER_KEY"])), model="claude-sonnet-4-5", and the saved reasoning item's encrypted_content decoded with json.loads instead of reading provider_data

$ .venv/bin/python case2_sdk_responses_via_proxy.py out/case2 <port>

Before (3f39fef)

=== t1_paris: claude-sonnet-4-5 via http://127.0.0.1:52224/v1/responses <- "What's the weather and local time in Paris?"
  item: reasoning_item
  item: message_output_item
  item: tool_call_item get_weather({"city": "Paris"})
  item: tool_call_item get_local_time({"city": "Paris"})
  item: tool_call_output_item -> 'The weather in Paris is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Paris is 14:05.'
  item: message_output_item
  final_output: "In Paris, it's currently **14:05** (2:05 PM) and the weather is **22°C and sunny**."
  streamed response.reasoning_summary_text.delta: 404 chars
  saved+reloaded 8 items -> t1_paris.json
  saved[1] reasoning item: summary chars=0 content chars=404
    encrypted_content block: type=thinking thinking chars=808 (2.00x the streamed reasoning) same_text_twice=True signed=True

=== t2_rome_on_reloaded_history: claude-sonnet-4-5 via http://127.0.0.1:52224/v1/responses <- 'Now the same two things for Rome.'
  item: reasoning_item
  item: message_output_item
  item: tool_call_item get_weather({"city": "Rome"})
  item: tool_call_item get_local_time({"city": "Rome"})
  item: tool_call_output_item -> 'The weather in Rome is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Rome is 14:05.'
  item: message_output_item
  final_output: "In Rome, it's currently **14:05** (2:05 PM) and the weather is **22°C and sunny**."
  streamed response.reasoning_summary_text.delta: 97 chars
  saved+reloaded 16 items -> t2_rome_on_reloaded_history.json
  saved[1] reasoning item: summary chars=0 content chars=404
    encrypted_content block: type=thinking thinking chars=808 (8.33x the streamed reasoning) same_text_twice=True signed=True
  saved[9] reasoning item: summary chars=0 content chars=97
    encrypted_content block: type=thinking thinking chars=194 (2.00x the streamed reasoning) same_text_twice=True signed=True

After (1955444)

=== t1_paris: claude-sonnet-4-5 via http://127.0.0.1:21604/v1/responses <- "What's the weather and local time in Paris?"
  item: reasoning_item
  item: message_output_item
  item: tool_call_item get_weather({"city": "Paris"})
  item: tool_call_item get_local_time({"city": "Paris"})
  item: tool_call_output_item -> 'The weather in Paris is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Paris is 14:05.'
  item: message_output_item
  final_output: "In Paris, it's currently **14:05** (2:05 PM) and the weather is **22°C and sunny**."
  streamed response.reasoning_summary_text.delta: 366 chars
  saved+reloaded 8 items -> t1_paris.json
  saved[1] reasoning item: summary chars=0 content chars=366
    encrypted_content block: type=thinking thinking chars=366 (1.00x the streamed reasoning) same_text_twice=False signed=True

=== t2_rome_on_reloaded_history: claude-sonnet-4-5 via http://127.0.0.1:21604/v1/responses <- 'Now the same two things for Rome.'
  item: reasoning_item
  item: message_output_item
  item: tool_call_item get_weather({"city": "Rome"})
  item: tool_call_item get_local_time({"city": "Rome"})
  item: tool_call_output_item -> 'The weather in Rome is 22C and sunny.'
  item: tool_call_output_item -> 'The local time in Rome is 14:05.'
  item: message_output_item
  final_output: "In Rome, it's currently **14:05** (2:05 PM) and the weather is **22°C and sunny**."
  streamed response.reasoning_summary_text.delta: 111 chars
  saved+reloaded 16 items -> t2_rome_on_reloaded_history.json
  saved[1] reasoning item: summary chars=0 content chars=366
    encrypted_content block: type=thinking thinking chars=366 (3.30x the streamed reasoning) same_text_twice=False signed=True
  saved[9] reasoning item: summary chars=0 content chars=111
    encrypted_content block: type=thinking thinking chars=111 (1.00x the streamed reasoning) same_text_twice=False signed=True

Case 3: POST /v1/chat/completions stream with thinking enabled

$ curl -sN http://127.0.0.1:$PORT/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
    -d '{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"In one sentence, why is the sky blue?"}],"stream":true,"thinking":{"type":"enabled","budget_tokens":1024}}' > chat.sse
$ echo "reasoning_content streamed: $(grep '^data: {' chat.sse | sed 's/^data: //' | jq -j '.choices[0].delta.reasoning_content // empty' | wc -c | tr -d ' ') chars"
$ grep '^data: {' chat.sse | sed 's/^data: //' | jq -c 'select(.choices[0].delta.thinking_blocks != null) | .choices[0].delta.thinking_blocks[] | {type, thinking_chars: (.thinking|length), signed: (.signature != null)}' > chat.thinking_blocks.jsonl
$ echo "thinking_blocks deltas: $(wc -l < chat.thinking_blocks.jsonl | tr -d ' '); the unsigned ones carry $(jq -s 'map(select(.signed|not) | .thinking_chars) | add' chat.thinking_blocks.jsonl) chars in total"
$ echo "signed thinking_blocks delta(s):"; jq -c 'select(.signed)' chat.thinking_blocks.jsonl

Before (3f39fef): the signed delta repeats the whole thinking text

reasoning_content streamed: 424 chars
thinking_blocks deltas: 32; the unsigned ones carry 424 chars in total
signed thinking_blocks delta(s):
{"type":"thinking","thinking_chars":424,"signed":true}

After (1955444): the signed delta carries the signature only

reasoning_content streamed: 473 chars
thinking_blocks deltas: 37; the unsigned ones carry 473 chars in total
signed thinking_blocks delta(s):
{"type":"thinking","thinking_chars":0,"signed":true}

Case 4: POST /v1/messages stream with thinking enabled (unchanged surface)

$ curl -sN http://127.0.0.1:$PORT/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
    -d '{"model":"claude-sonnet-4-5","max_tokens":2048,"stream":true,"thinking":{"type":"enabled","budget_tokens":1024},"messages":[{"role":"user","content":"In one sentence, why is the sky blue?"}]}' > messages.sse
$ grep '^data: ' messages.sse | sed 's/^data: //' | jq -c 'select(.type=="content_block_delta" and (.delta.type=="thinking_delta" or .delta.type=="signature_delta")) | {t: .delta.type, chars: ((.delta.thinking // "")|length), signed: (.delta.signature != null)}' > messages.deltas.jsonl
$ echo "thinking_delta events: $(jq -c 'select(.t=="thinking_delta")' messages.deltas.jsonl | wc -l | tr -d ' '), $(jq -s 'map(select(.t=="thinking_delta") | .chars) | add' messages.deltas.jsonl) chars in total"
$ echo "signature_delta event(s):"; jq -c 'select(.t=="signature_delta")' messages.deltas.jsonl

Before (3f39fef)

thinking_delta events: 44, 610 chars in total
signature_delta event(s):
{"t":"signature_delta","chars":0,"signed":true}

After (1955444)

thinking_delta events: 26, 390 chars in total
signature_delta event(s):
{"t":"signature_delta","chars":0,"signed":true}

Observations from the legs

• Proxy /v1/responses puts reasoning text in content, not summary; pre-existing, left alone
• Proxy /v1/responses emits an empty message item before tool calls; pre-existing, left alone
• Claude accepts the doubled block, so the symptom is silent; this PR removes it

/live-pr-risk at 1955444

Two proxies A/B'd from this PR's merge base 3f39fef and from 1955444, each with its own native Postgres (schema migrated by the proxy's own boot), --num_workers 2, store_prompts_in_spend_logs: true, a recording reverse proxy in front of api.anthropic.com for the Responses bridge scenario, and real calls to Anthropic, Bedrock, and Vertex AI. Nine scenarios ran on both sides in the same order with the same inputs; spend rows were read back from GET /spend/logs after each one. Then the top of the list ran a third time on 1955444 merged with origin/main (d729f97, five commits of drift, uv.lock included): the merge applied clean, and all nine scenarios on the merged tree matched the head side (signed deltas with no text, the spend log's thinking_blocks holding the text once, the Responses bridge replaying a 354-char block equal to the streamed text, non-stream unchanged, the deleted key answering 401)

Breaking

• None observed. Every scenario answered 200 with the same item shapes on both sides, and the Responses bridge replayed a signed block Claude accepted on both sides. One baseline differed for a reason outside the diff: the key-lifecycle scenario's call after POST /key/delete got 401 on the base side and 200 on the head side. The two workers each keep an in-memory copy of the key for user_api_key_cache_ttl (60 s default), the delete clears only the worker that served it, and the follow-up call landed on the other worker on the head side. The diff touches litellm/llms/anthropic/chat/handler.py and its test only, so this is the rig's two-worker setup, not this PR

Backward incompatible

• The signed thinking_blocks delta on streamed chat chunks now carries thinking: "" plus the signature, where it repeated the whole text. Observed on direct Anthropic (408 chars to 0), Bedrock invoke (445 to 0), Vertex AI Claude (429 to 0), and the router fallback replay (429 to 0), each with the per-delta blocks unchanged and the spend log's rebuilt thinking_blocks holding the text once (816 to 439, 890 to 469, 858 to 435, 858 to 437). A client that reads only the signed delta and ignores the deltas before it now gets no text; every client that accumulates deltas (the Agents SDK, the Responses bridge, stream_chunk_builder) gets the text once instead of twice. This is the change the PR ships (1955444); the alternative, text in the signed delta too, is the bug itself
• The rebuilt streamed message's provider_specific_fields.thinking_blocks (spend log response, and what stream_chunk_builder hands a caller) now mirrors the signature-only chunk instead of the full block: direct chat 408 chars to 0, /v1/messages 519 to 0, Bedrock invoke 445 to 0, Vertex 429 to 0, pass-through 343 to 0, fallback 429 to 0. The message's own thinking_blocks, the field LiteLLM documents for replay, carries the text once in every one of those rows. Left as is: merging the text back into the signature chunk's copy would put the text on the wire twice for every client that accumulates deltas, which is what this PR removes; a consumer of provider_specific_fields.thinking_blocks on a rebuilt stream was not found in the repo. Open for the reviewer's call

Regression risk

• Snowflake Claude, Bedrock mantle, claude_platform, and Azure AI Anthropic deployments reuse this streaming iterator and were traced, not run: the rig has no deployment for them (no Snowflake or claude_platform credentials on this box; Azure AI and mantle credentials exist but no deployment was added in this run). Exercising them is one model_list entry each in the rig config and a rerun of the direct-stream scenario
• The /v1/messages adapter for non-Anthropic models does not run this iterator (traced, not on the changed path)

Dependency graph

• ModelResponseIterator signature chunk (the changed code): unit tests in tests/unit/llms/anthropic/chat/test_anthropic_chat_handler.py (per-chunk shape and the next-turn replay); verified live, direct Anthropic stream
• CustomStreamWrapper to client chunks: verified live on direct, Bedrock invoke, Vertex, and fallback streams
• stream_chunk_builder to the spend log response and to callers: verified live, spend rows of every stream scenario
• Bedrock invoke handler (reuses the iterator): verified live
• Vertex AI partner Claude (runs the Anthropic chat handler): verified live
• Native /v1/messages route and its spend log: verified live (signature_delta carried no text before or after; the spend row's block halves)
• /anthropic/v1/messages pass-through logging handler (rebuilds the response with the iterator): verified live, spend row halves
• Responses bridge accumulate and replay (LiteLLMCompletionResponsesConfig, streaming iterator): verified live with the recorder; the Rome turn replays a 264-char signed block that equals the streamed text, where the base replayed 588 chars (the text twice) on three later requests
• Router fallback replay: verified live
• Non-stream transformation: verified live, unchanged (510 chars once on both sides)
• Agents SDK LitellmModel in process: verified live in the QA legs above
• Snowflake, mantle, claude_platform, Azure AI Anthropic: traced, untested live
• Legacy live suites on 1955444 (tests/llm_translation/test_anthropic_completion.py, tests/local_testing/test_streaming.py, tests/llm_translation/test_bedrock_completion.py, -k "thinking or reasoning or signature"): 15 passed, 1 skipped (test_bedrock_thinking_in_assistant_message[False], the suite's own skip)

Not verified

• Snowflake Claude streaming (no credentials on this box)
• claude_platform streaming (no credentials on this box)
• Bedrock mantle streaming (credentials exist, no deployment added in this run)
• Azure AI Anthropic streaming (credentials exist, no deployment added in this run)

Audit round 1 of 3

/audit of tip 19554443c80fe1dba329996aa03fa3f5cb65c613 against merge base 3f39fef52f85b9cc161abba6bb47ca6740801200. Rig: owned Postgres, Redis, and a scripted Anthropic-wire upstream, one 2-worker proxy per leg, built from tests/integration/providers/test_anthropic_thinking_signature_stream_wire.py and test_anthropic_thinking_signature_logging_wire.py (--seed 44127 --order-seed 44127, head run twice with identical selections and results). T is the scripted thinking text, S its signature. "red" on the merge base is the fix-sensitive thinking assertion (doubled T under one S), "green" on both legs is a control

id	cell	head 1	head 2	base	verdict
H1	test_openai_sdk_sync_stream_accumulates_the_thinking_once	green	green	red	VERIFIED
H2	test_openai_sdk_async_stream_accumulates_the_thinking_once	green	green	red	VERIFIED
H3	test_signature_chunk_carries_no_thinking_text_on_every_anthropic_wire_provider[anthropic]	green	green	red	VERIFIED
H4	test_non_streaming_completion_keeps_the_signed_thinking_block_intact	green	green	green	VERIFIED
H5	..._on_every_anthropic_wire_provider[bedrock_invoke]	green	green	red	VERIFIED
H7	..._on_every_anthropic_wire_provider[claude_platform]	green	green	red	VERIFIED
H8	..._on_every_anthropic_wire_provider[vertex_ai]	green	green	red	VERIFIED
H9	..._on_every_anthropic_wire_provider[snowflake]	green	green	red	VERIFIED
H10	..._on_every_anthropic_wire_provider[azure_ai]	green	green	red	VERIFIED
H11	test_responses_stream_encrypts_the_thinking_once	green	green	red	VERIFIED
H12	test_responses_non_stream_encrypts_the_signed_block_as_received	green	green	green	VERIFIED
H13	test_chat_stream_spend_row_stores_the_thinking_once	green	green	red	VERIFIED
H14	test_cache_hit_replays_the_answer_from_one_upstream_call_and_never_doubles_the_thinking	green	green	red	VERIFIED
H15	test_native_messages_stream_through_the_anthropic_sdk_logs_the_thinking_once	green	green	red	VERIFIED
H16	test_native_messages_stream_on_bedrock_mantle_logs_the_thinking_once	green	green	red	VERIFIED
H17	test_adapter_messages_stream_on_snowflake_logs_the_thinking_once	green	green	red	VERIFIED
P1	test_anthropic_passthrough_stream_relays_the_frames_and_logs_the_thinking_once	green	green	red	VERIFIED
P2	test_bedrock_invoke_passthrough_stream_relays_the_frames_and_logs_the_thinking_once	green	green	red	VERIFIED
P3	live Vertex :streamRawPredict pass-through, Claude Sonnet 5.5	154 chars logged once	n/a	362 chars logged (client saw 181)	VERIFIED
E1	test_two_signed_blocks_each_keep_their_own_text_through_a_replay	green	green	red	VERIFIED
E2	test_redacted_block_before_a_signed_block_replays_each_once	green	green	red	VERIFIED
E3	test_signature_only_block_without_thinking_deltas_is_relayed_as_is	green	green	green	VERIFIED
E4	test_replaying_the_accumulated_turn_sends_the_thinking_once_with_its_signature	green	green	red	VERIFIED
E5	test_two_identical_requests_with_no_cache_each_land_their_own_spend_row	green	green	red	VERIFIED
S1	test_unusable_signature_values_yield_no_signed_block_and_keep_the_stream_intact[integer]	green	green	green	VERIFIED
S2	..._keep_the_stream_intact[list]	green	green	green	VERIFIED
S3	..._keep_the_stream_intact[empty]	green	green	green	VERIFIED
S4	test_five_kilobyte_signature_is_relayed_verbatim_without_thinking_text	green	green	red	VERIFIED
S5	test_duplicate_signature_deltas_never_repeat_the_thinking_text	green	green	red	VERIFIED
S6	test_non_string_thinking_delta_is_ignored_and_the_signed_block_still_lands_once	green	green	red	VERIFIED
S7	test_upstream_authentication_error_reaches_the_caller_and_leaves_the_proxy_healthy	green	green	red	VERIFIED
S8	test_unauthenticated_stream_is_refused_before_the_upstream_is_called	green	green	green	VERIFIED
C1	test_mid_thinking_upstream_aborts_in_a_mixed_burst_leave_every_completed_call_logged_once	green	green	red	VERIFIED
C2	test_worker_sigkill_mid_burst_leaves_the_sibling_streaming_signed_thinking_once	green	green	red	VERIFIED
L1	live Agents SDK LitellmModel replay, Claude Sonnet 5.5	one block, 160 chars, replayed once	n/a	block text doubled, replayed doubled	VERIFIED

Live rows ran once per leg against the real Anthropic API and Vertex AI with adaptive thinking (reasoning_effort: high, display: summarized). On the merge base the Agents SDK saved the thinking text twice under one signature and sent it back doubled on turn 2 (the API still answered 200); on the tip it saved and replayed the text once. H6 (Bedrock Mantle on /v1/chat/completions) was dropped from the inventory because that surface never touches the Anthropic iterator

Three pre-existing behaviors outside the diff were ticketed internally: a cached stream replay drops thinking_blocks and reasoning_content on both legs, the Anthropic pass-through logging handler only runs when the upstream hostname is api.anthropic.com, and the router pass-through keeps the invoke/ route marker of a bedrock/invoke/... deployment in the upstream model path

Rerun at 775fe9f (tests only, tightens the cache cell's replay assertion to exactly an empty tuple or one signed block): H14 ran again on both legs, green on the tip and red on the merge base at the first-stream thinking assertion as before (the stream module: 27 passed on the tip, 20 failed and 7 passed on the base, the same red set as the first base run); every other row keeps its result from the run at 6e40e47

• 775fe9f passes /audit

Type

🐛 Bug Fix

Caveats (if any)

Low

• The /v1/messages adapter already turned a signed block into a signature_delta and dropped that chunk's text, so its output is unchanged
• The provider_specific_fields.thinking_blocks change on rebuilt streams is recorded under Backward incompatible in the /live-pr-risk section above, with the decision to leave it

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 1955444 passes /live-pr-risk

---

Note

Medium Risk
Changes streamed chat thinking_blocks shape on the signature chunk (intentional contract shift); clients that read only that delta lose inline text, though downstream accumulation and logging paths are heavily tested.

Overview
Fixes doubled extended-thinking text when Claude streams a signature_delta: the Anthropic chat stream iterator no longer re-emits the full accumulated thinking in that chunk and instead attaches a signature-only thinking_blocks entry (thinking: "" plus the signature). Per-delta thinking streams unchanged via reasoning_content and unsigned thinking blocks, so accumulators (stream_chunk_builder, Agents SDK, Responses bridge) rebuild one signed block with the text once.

Tests add shared wire fixtures and broad integration coverage (chat/messages/responses, multiple providers, spend logging, replay, cache, chaos) plus updated unit expectations and a round-trip replay test.

^{Reviewed by Cursor Bugbot for commit 775fe9f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Fixes how thinking text appears in streamed Anthropic responses.

The PR appears safe to merge; no new actionable issue was established.

Findings

1.  Cache test can pass without thinking ▶

Summary

The PR changes Anthropic streaming so the signature chunk carries no repeated thinking text, and adds unit and integration coverage for accumulation and replay.

• The latest commit tightens the cache-hit assertion to accept either no thinking block or exactly the expected signed block.

_{Reviews (3) · Last reviewed commit: "test(anthropic): pin the cached replay's..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_anthropic_stream_signature_only_block (775fe9f) with main (c5c0a48)}

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

Stale Bugbot comment from a previous run.

[greptile-apps]

Cache test can pass without thinking

If the cached stream has no thinking_blocks, accumulate(second_deltas) returns an empty tuple and all(...) still passes. The test would report success even though the cache replay lost the signed thinking it is meant to check. Compare the accumulated blocks with the expected block directly.

Suggested change

	assert all(block == thinking_block(THINKING, SIGNATURE) for block in accumulate(second_deltas)), second.text
	assert accumulate(second_deltas) == (thinking_block(THINKING, SIGNATURE),), second.text

[mateo-berri]

Direct equality fails because the cached replay drops thinking blocks before this PR, tracked separately. The check now admits only empty or one signed block

[greptile-apps]

That context makes sense. Since cached stream replays dropping thinking_blocks is a separately tracked pre-existing behavior, requiring the expected block here would make this PR’s test fail for an unrelated issue. The current assertion still rejects the regression this PR targets (a duplicated or otherwise unexpected non-empty block) while allowing the known empty replay, so I’m withdrawing the finding.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 775fe9f. Configure here.}

[mateo-berri]

LGTM