fix(streaming): preserve OpenAI SDK usage details in stream chunk builder

[harryzhou2000]

TLDR

Problem this solves:

• Streamed usage chunks built from raw OpenAI SDK CompletionUsage objects silently lose prompt_tokens_details and completion_tokens_details
• Cache-read tokens then fall back to fresh-input pricing and reasoning tokens disappear from usage accounting

How it solves it:

• Convert raw pydantic usage models to litellm Usage at the stream chunk extraction boundary
• Add a regression test using the raw openai.CompletionUsage shape

User Flow

Before: a developer streams chat completions through an OpenAI-compatible endpoint and their cost dashboard never sees cached-token or reasoning-token detail.

1. They send POST /v1/chat/completions with "stream": true and "stream_options": {"include_usage": true}
2. The final SSE chunk arrives with prompt_tokens/completion_tokens, but prompt_tokens_details is missing
3. Their cost dashboard bills the full input at the uncached input rate and shows no reasoning-token breakdown

After: the same request keeps the provider-reported usage details.

1. They send the same POST /v1/chat/completions with "stream": true and "stream_options": {"include_usage": true}
2. The final SSE chunk now carries prompt_tokens_details.cached_tokens and completion_tokens_details.reasoning_tokens
3. Their cost dashboard bills cached tokens at the cache-read rate and shows the reasoning-token breakdown

Relevant issues

Related: #34801 (upstream #34812 fixed the generic stream-reassembly path; this covers raw OpenAI SDK pydantic usage shapes that still lost the details).

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

E2E proof against the real DeepSeek API through a litellm proxy, streamed chat completions with stream_options.include_usage.

Before (v1.97.0-rc.1 without this patch, second call with a shared prefix):

call 1: usage=CompletionUsage(completion_tokens=1, prompt_tokens=1462, total_tokens=1463, completion_tokens_details=..., prompt_tokens_details=None)

After (same proxy image plus this patch; original capture at commit 96af89842c, rebased onto litellm_internal_staging as c3db2800e3 with no production code changes):

call 0: usage=CompletionUsage(..., completion_tokens_details=..., reasoning_tokens=12, prompt_tokens_details=PromptTokensDetails(..., cached_tokens=1664))
call 1: usage=CompletionUsage(..., completion_tokens_details=..., reasoning_tokens=8, prompt_tokens_details=PromptTokensDetails(..., cached_tokens=1664))

The regression test also fails on unpatched litellm_internal_staging:

>       assert usage.prompt_tokens == 1714
E       assert 0 == 1714
E        +  where 0 = Usage(..., prompt_tokens_details=None, ...).prompt_tokens

Type

🐛 Bug Fix

Changes

• litellm/litellm_core_utils/streaming_chunk_builder_utils.py: in _extract_usage_chunk(), convert raw pydantic usage objects (e.g. OpenAI SDK CompletionUsage) to litellm Usage via model_dump() before the chunk builder reads them.
• tests/test_litellm/litellm_core_utils/test_streaming_chunk_builder_utils.py: add a regression test using a raw openai.CompletionUsage on a stream chunk, asserting cached_tokens and reasoning_tokens survive calculate_usage().

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

All committers have signed the CLA.

[greptile-apps]

Greptile Summary

The PR normalizes raw Pydantic usage objects before streamed usage calculation so provider-reported token details remain available.

• Converts objects exposing model_dump() into LiteLLM Usage instances.
• Adds coverage for cached prompt tokens and reasoning tokens from OpenAI SDK usage models.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/streaming_chunk_builder_utils.py	Normalizes raw Pydantic usage models into LiteLLM’s usage representation while preserving nested token details.
tests/test_litellm/litellm_core_utils/test_streaming_chunk_builder_utils.py	Adds a representative OpenAI SDK usage regression test using a newly constructed, non-mutated chunk dictionary.

_{Reviews (3): Last reviewed commit: "test(streaming): construct usage fixture..." | Re-trigger Greptile}

[greptile-apps]

Avoid mutating the test fixture

The test adds usage by mutating the dumped chunk after construction, contrary to the repository's immutability convention and making the complete fixture shape less explicit.

Suggested change

	chunk["usage"] = CompletionUsage(
	prompt_tokens=1714,
	completion_tokens=14,
	total_tokens=1728,
	prompt_tokens_details={"cached_tokens": 1664},
	completion_tokens_details={"reasoning_tokens": 12},
	)
	chunk = {
	**chunk,
	"usage": CompletionUsage(
	prompt_tokens=1714,
	completion_tokens=14,
	total_tokens=1728,
	prompt_tokens_details={"cached_tokens": 1664},
	completion_tokens_details={"reasoning_tokens": 12},
	),
	}

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing harryzhou2000:fix/stream-usage-cache-details-pr (c3db280) with litellm_internal_staging (8941f2a)}

[harryzhou2000]

@greptileai

[harryzhou2000]

@greptileai

[harryzhou2000]

Note: the misc / Run tests failure is pre-existing on litellm_internal_staging (test_handle_completed_vertex_batch_computes_cost_usage_and_models expects non-batch pricing). It is unrelated to this PR; upstream draft #37443 fixes the base assertion.