Fix Anthropic streaming reasoning token usage

[ishaan-berri]

Summary

Anthropic streaming responses emitted thinking deltas but discarded the thinking text before final usage calculation. This stores streamed thinking text in the iterator and passes it to the existing Anthropic usage calculator so completion_tokens_details.reasoning_tokens is populated and text_tokens excludes reasoning tokens. The usage split now also caps locally-estimated reasoning tokens to numeric provider-reported output totals so text_tokens cannot go negative and mocked token values remain safe.

Repro

Send an Anthropic chat completion stream with extended thinking enabled and stream_options.include_usage=true. Reasoning deltas arrive during the stream, but the final usage chunk previously reported reasoning_tokens: 0 and counted all output tokens as text.

Evidence

End-to-end local transcript using litellm.completion() against an Anthropic-compatible HTTP server:

// Non-streaming request shape sent through LiteLLM
{
  "model": "anthropic/claude-sonnet-4-6",
  "api_base": "http://127.0.0.1:<port>",
  "messages": [{"role": "user", "content": "Solve a probability problem and show thinking."}],
  "thinking": {"type": "adaptive"},
  "max_tokens": 128
}

// Non-streaming response usage from LiteLLM
{
  "completion_tokens": 50,
  "prompt_tokens": 10,
  "total_tokens": 60,
  "completion_tokens_details": {
    "reasoning_tokens": 19,
    "text_tokens": 31
  }
}

// Streaming request shape sent through LiteLLM
{
  "model": "anthropic/claude-sonnet-4-6",
  "api_base": "http://127.0.0.1:<port>",
  "messages": [{"role": "user", "content": "Solve a probability problem and show thinking."}],
  "thinking": {"type": "adaptive"},
  "max_tokens": 128,
  "stream": true,
  "stream_options": {"include_usage": true}
}

// Streaming deltas observed from LiteLLM
{
  "reasoning_content": [
    "First I need to count the favorable outcomes. ",
    "Then I compare that count with all possible outcomes."
  ],
  "content": ["The probability is 3/8."]
}

// Final streaming response usage from LiteLLM
{
  "completion_tokens": 50,
  "prompt_tokens": 10,
  "total_tokens": 60,
  "completion_tokens_details": {
    "reasoning_tokens": 19,
    "text_tokens": 31
  }
}

// Anthropic-compatible server requests received from LiteLLM
[
  {"path": "/v1/messages", "model": "claude-sonnet-4-6", "stream": false, "thinking": {"type": "adaptive"}},
  {"path": "/v1/messages", "model": "claude-sonnet-4-6", "stream": true, "thinking": {"type": "adaptive"}}
]

The same flow is now pinned in test_anthropic_completion_streaming_usage_matches_non_streaming_with_thinking.

Tests

• Added test_streaming_thinking_deltas_count_reasoning_tokens_in_usage in tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_handler.py.
• Added test_anthropic_completion_streaming_usage_matches_non_streaming_with_thinking, which runs litellm.completion() against a local Anthropic-compatible HTTP server and compares non-streaming vs streaming usage.
• Added test_calculate_usage_clamps_text_tokens_when_reasoning_estimate_exceeds_output for Greptile’s token split concern.
• Added test_calculate_usage_handles_mocked_output_tokens_with_reasoning_content for the mocked usage object edge case surfaced by CI.
• Confirmed the initial regression test fails before the fix with reasoning_tokens=0.
• Ran uv run pytest tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_handler.py tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py -q (248 passed).
• Ran uv run black ., then kept only scoped formatting changes.
• Ran CI lint/typecheck equivalents: uv lock --check, uv run --no-sync black --check --exclude '/enterprise/' . in litellm/, uv run --no-sync ruff check . in litellm/, uv run --no-sync mypy . in litellm/, circular import check, and import safety check.

CI

• GitHub Actions: all checks green on the latest push, including lint, integrations, proxy-infra, and test.
• CircleCI: all contexts green on the latest push, including proxy_e2e_anthropic_messages_tests, build_and_test, and llm_translation_testing.
• Veria AI review: green.

Review

• Greptile score: 4/5.
• Addressed the actionable concern by capping estimated reasoning tokens to numeric provider-reported output totals, which prevents negative text_tokens and keeps the split well-formed.
• No inline review threads were present.

Relevant issues

Fixes the Anthropic streaming reasoning token accounting bug.

Linear ticket

N/A

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review and received a confidence score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

See ## Evidence for request/response transcript.

Type

🐛 Bug Fix
✅ Test

Changes

• Accumulate streamed Anthropic thinking deltas in ModelResponseIterator.
• Use the accumulated reasoning text when calculating final streaming usage.
• Cap estimated reasoning tokens to numeric provider-reported output token totals to avoid negative text token counts and mock comparison errors.
• Add iterator-level, completion-level, token-split, and mocked-usage regression tests.

  

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This change corrects the Anthropic streaming path where emitted thinking-delta chunks were never stored, causing the final usage chunk to always report zero reasoning tokens. Thinking text is now accumulated in the iterator and forwarded to the existing usage calculator so the token split is populated.

• Adds a reasoning_content_chunks list to ModelResponseIterator that collects each thinking-delta string; the joined text is passed to calculate_usage when the terminal usage event arrives.
• Adds a regression test that drives synthetic Anthropic stream events through chunk_parser and verifies that completion_tokens_details.reasoning_tokens is positive and text_tokens is consistent.

Confidence Score: 4/5

The change is safe to merge; it is narrowly scoped to the streaming iterator and adds no new code paths that could break existing non-reasoning-mode callers.

The accumulation logic is correct and consistent with the non-streaming path. The one item worth watching is that text_tokens is derived by subtracting a locally-estimated token count from Anthropic's reported output token count — if the estimate overshoots, the field goes negative. That arithmetic lives in transformation.py and predates this PR, but this change now activates it for streaming callers too.

litellm/llms/anthropic/chat/transformation.py — the text_tokens subtraction in calculate_usage has no floor guard, which is now reachable from the streaming path.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/chat/handler.py	Adds reasoning_content_chunks accumulator to ModelResponseIterator and feeds the joined text into _handle_usage so the existing calculate_usage path can estimate reasoning_tokens for streaming responses; the token split uses a local token_counter approximation (pre-existing design) that can yield slightly inaccurate text_tokens.
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_handler.py	Adds test_streaming_thinking_deltas_count_reasoning_tokens_in_usage — a new unit test (no real network calls) that feeds synthetic Anthropic stream events through chunk_parser and asserts that reasoning deltas are forwarded and final usage is split into reasoning_tokens > 0 and matching text_tokens.

Comments Outside Diff (1)

1. litellm/llms/anthropic/chat/transformation.py, line 2234-2241 (link)

   The reasoning_tokens value is derived from a local token_counter estimate on the accumulated thinking text, and text_tokens is then completion_tokens - reasoning_tokens. If token_counter overestimates for a long reasoning chain, text_tokens goes negative. A max(0, ...) clamp keeps the value well-formed. The same pre-existing pattern on the non-streaming call site would benefit from the same guard.

_{Reviews (1): Last reviewed commit: "fix anthropic streaming reasoning token ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!