fix(cost_tracking): map OpenAI cache_write_tokens for prompt cache creation billing

[devin-ai-integration]

TLDR

Problem this solves:

• OpenAI gpt-5.6 cache-write tokens were dropped by LiteLLM
• They were billed at plain input rate, not cache-write rate
• Cache creation was missing from spend logs and daily metrics

How it solves it:

• Add cache_write_tokens as canonical OpenAI field on the usage wrapper
• Auto-mirror it with legacy cache_creation_tokens via __setattr__
• Map the field on Chat Completions and Responses API usage paths
• Derive spend-log cache_creation_input_tokens from it (no schema change)

Relevant issues

Linear ticket

Resolves LIT-4633

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proxy, real OpenAI gpt-5.6-luna, real spend, Postgres-backed spend logs + daily metrics. Each request sends a fresh ~18k-token unique prefix to force a cache write.

Before (base 3810130105)

$ curl -s -D - .../v1/chat/completions -d '{"model":"gpt-5.6-luna", ...}'
x-litellm-response-cost: 0.017574
prompt_tokens_details: {"audio_tokens": 0, "cached_tokens": 0, "cache_write_tokens": 17547}

# LiteLLM_SpendLogs (customer symptom: cache creation omitted from logs)
prompt_tokens      | 17550
spend              | 0.017574
log_cache_creation |            <- NULL, cache creation tokens dropped
log_ptd            | {"audio_tokens": 0, "cached_tokens": 0, "cache_write_tokens": 17547}

After (this branch, 790141ad78)

x-litellm-response-cost: 0.02321325
prompt_tokens_details: {"audio_tokens": 0, "cached_tokens": 0, "cache_write_tokens": 18549, "cache_creation_tokens": 18549}

# LiteLLM_SpendLogs
prompt_tokens      | 18552
spend              | 0.02321325
log_cache_creation | 18549       <- now logged
log_cache_write    | 18549

# LiteLLM_DailyUserSpend (daily metrics), no schema change
cache_creation_input_tokens | 36097
cache_read_input_tokens     | 35096

Per-token rate confirms the billing fix: before 0.017574 / 17547 ~ 1.00e-6 (plain input rate); after 0.02321325 / 18549 ~ 1.25e-6 (the 1.25x cache-write rate the cost map defines for the gpt-5.6 family). Cache reads were already correct and stay correct.

The LIT-4633 repro comes through the Responses API (/v1/responses), where usage.input_tokens_details.cache_write_tokens was dropped by the usage transform. This is now mapped in _transform_response_api_usage_to_chat_usage and covered by test_transform_response_api_usage_maps_cache_write_tokens, so the same normalization (and the litellm_input_cache_creation_tokens_metric) applies to that route.

QA on tip (2026-07-23)

Re-verified with the branch merged onto the current litellm_internal_staging tip a507394841 (283 commits ahead of the PR base; merge is conflict-free). On tip this branch also composes with the newer kimi-k2 cache_write_tokens handling in cost_calculator.py and db_spend_update_writer.py: the fallback this branch deletes from get_token_type_cost_breakdown is covered because _parse_prompt_tokens_details now reads cache_write_tokens first, so no path loses the count

Unit QA on the merge: the suites for llm_cost_calc, spend_tracking, responses and tests/test_litellm/test_utils.py pass (1081 tests), plus a sweep over every other tests/test_litellm file touching prompt_tokens_details / cache_creation_tokens (streaming aggregation, Anthropic adapters, prometheus token detail metrics, cost calculator: 1550 passed). The one failure in that sweep, test_main.py::test_openai_env_base[OPENAI_API_BASE], fails identically on tip without this branch, so it is pre-existing and unrelated

Live proxy on the merge, real OpenAI gpt-5.6-luna, Postgres spend DB, fresh ~42k-token unique prefixes:

# 1st send (cache write)
x-litellm-response-cost: 0.053267
prompt_tokens_details: {"audio_tokens": 0, "cached_tokens": 0, "cache_write_tokens": 42592, "cache_creation_tokens": 42592}
rate check: 3 x 1.00e-6 + 42592 x 1.25e-6 + 4 x 6.00e-6 = 0.053267 exactly

# 2nd send, same body (cache read)
x-litellm-response-cost: 0.0042862
prompt_tokens_details: {"audio_tokens": 0, "cached_tokens": 42592, "cache_write_tokens": 0, "cache_creation_tokens": 0}
rate check: 3 x 1.00e-6 + 42592 x 1.00e-7 + 4 x 6.00e-6 = 0.0042862 exactly

# /v1/responses, fresh prefix (cache write, the LIT-4633 route)
x-litellm-response-cost: 0.053588
input_tokens_details: {"audio_tokens": null, "cached_tokens": 0, "text_tokens": null, "cache_write_tokens": 42844}
rate check: 3 x 1.00e-6 + 42844 x 1.25e-6 + 5 x 6.00e-6 = 0.053588 exactly

# LiteLLM_SpendLogs (metadata.additional_usage_values)
chat write      | cache_creation_input_tokens 42592 | prompt_tokens_details.cache_write_tokens 42592
chat read       | cache_read_input_tokens 42592, cache_creation absent (correct for a pure read)
responses write | cache_creation_input_tokens 42844 (filled from the usage_object fallback)

# LiteLLM_DailyUserSpend, same day, no schema change
chat row      | cache_creation_input_tokens 85347 | cache_read_input_tokens 85347
responses row | cache_creation_input_tokens 85604 | cache_read_input_tokens 0
(exact sums of the QA requests; daily spend total matches the per-request costs to the cent)

Type

Bug Fix

Changes

OpenAI's gpt-5.6 family returns cache-write tokens in usage.prompt_tokens_details.cache_write_tokens (Chat Completions) and usage.input_tokens_details.cache_write_tokens (Responses API), a field absent from the OpenAI SDK's typed token-details models. LiteLLM only ever mapped the Anthropic/Bedrock top-level cache_creation_input_tokens onto its internal cache_creation_tokens, so for OpenAI the cache-write count was never surfaced: it was dropped from spend logs and the tokens were billed at the plain input rate instead of the cache-write rate defined in the cost map. This was reported by customers for the GPT-5.6 series

This makes cache_write_tokens the canonical OpenAI-native name on PromptTokensDetailsWrapper and normalizes both provider dialects onto it. The wrapper keeps cache_write_tokens (OpenAI) and cache_creation_tokens (Anthropic/Bedrock, kept for backwards compatibility) in sync automatically: assigning either mirrors to the other via __setattr__, so downstream cost calc, streaming aggregation, and logging can read or write either name and always agree

class PromptTokensDetailsWrapper:
    cache_write_tokens: Optional[int]      # canonical, OpenAI naming
    cache_creation_tokens: Optional[int]   # legacy, mirrored from cache_write_tokens

    def __setattr__(self, name, value):    # assigning either name mirrors to the other
        super().__setattr__(name, value)
        if name == "cache_write_tokens":    super().__setattr__("cache_creation_tokens", value)
        elif name == "cache_creation_tokens": super().__setattr__("cache_write_tokens", value)

Usage.__init__ maps the Anthropic/Bedrock cache_creation_input_tokens param onto cache_write_tokens. The Responses API usage transform (_transform_response_api_usage_to_chat_usage) now carries input_tokens_details.cache_write_tokens through instead of dropping it while rebuilding the token details. The cost parser reads cache_write_tokens first, then falls back to cache_creation_tokens

For spend logs, get_logging_payload derives the existing cache_creation_input_tokens field from prompt_tokens_details.cache_write_tokens, mirroring how it already derives cache_read_input_tokens from cached_tokens; daily metrics reuse that same field, so there are no new DB columns. On the Responses API route the response usage is not chat-Usage-shaped, so additional_usage_values has no prompt_tokens_details to read; in that case it falls back to the normalized standard_logging usage_object, which is what makes the Admin UI Logs "Cache Creation Tokens" row render for /v1/responses

Review found the __setattr__ mirror introduced a double-count in BaseTokenUsageProcessor.combine_usage_objects, which sums every field in prompt_tokens_details.model_fields: the mirrored pair got added twice (a single 50-token usage combined to 100), which would have overbilled Anthropic batch cost calc, mid-stream fallback usage merges, and realtime usage aggregation. _summable_prompt_token_fields now collapses the mirrored pair to one representative before summing

QA runbook

Live flows (proxy from this branch with a gpt-5.6 family model and a Postgres DATABASE_URL; every cache write needs a fresh, never-seen prefix above the caching minimum, ~18k tokens is comfortable):

• POST /v1/chat/completions with a fresh unique system prefix; expect prompt_tokens_details to carry equal cache_write_tokens and cache_creation_tokens, and x-litellm-response-cost to price those tokens at the model's cache_creation_input_token_cost (1.25x input for gpt-5.6)
• Send the identical body again; expect cached_tokens equal to the previous write count, cache_write_tokens 0, and the cheaper cache-read cost
• POST /v1/responses with a fresh prefix; expect usage.input_tokens_details.cache_write_tokens in the response and the same cache-write pricing (this is the LIT-4633 route)
• In LiteLLM_SpendLogs, expect metadata.additional_usage_values.cache_creation_input_tokens populated for both write requests (chat and responses) and absent for the pure read
• In LiteLLM_DailyUserSpend and the Admin UI Logs "Cache Creation Tokens" row, expect cache creation to accumulate with no schema change
• Mid-stream fallback: stream against a deployment that fails mid-generation with a real overloaded_error SSE event and falls back to a real Anthropic deployment using a fresh cache_control prefix; expect the final usage chunk's cache_write_tokens/cache_creation_tokens to equal the provider's actual write count instead of double it (live proof in the PR comments, before 69640 vs after 34878 on ~35k-token prompts)

Added tests and what they pin down:

• tests/test_litellm/test_utils.py (wrapper normalization): OpenAI prompt_tokens_details.cache_write_tokens and Anthropic top-level cache_creation_input_tokens both populate the field pair, the pair stays mirrored on post-construction assignment, and neither field materializes on a read-only cache hit
  • Sanity check: this test makes sense to add and is not hand-wavey (asserts exact token values on both names, not just presence) or potentially flaky
• tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py (billing): gpt-5.6 cache-write tokens are billed at cache_creation_input_token_cost, including the Responses API breakdown itemization
  • Sanity check: this test makes sense to add and is not hand-wavey (asserts the exact expected cost, not cost > 0) or potentially flaky
• tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py (spend logs): cache_creation_input_tokens is derived from cache_write_tokens, Anthropic's explicit value is preserved and not clobbered, zero or missing counts stay absent, and the Responses API route fills it from the usage_object fallback
  • Sanity check: this test makes sense to add and is not hand-wavey (covers the absent/zero edge cases, not just the happy path) or potentially flaky
• tests/test_litellm/responses/test_responses_utils.py (LIT-4633 regression): _transform_response_api_usage_to_chat_usage carries input_tokens_details.cache_write_tokens through instead of dropping it
  • Sanity check: this test makes sense to add and is not hand-wavey (asserts both mirrored names and the cached_tokens passthrough) or potentially flaky
• tests/test_litellm/test_cost_calculator.py (aggregation regression): combine_usage_objects sums the mirrored cache_write_tokens/cache_creation_tokens pair once; a single 50-token usage stays 50 and two combine to 100, not double
  • Sanity check: this test makes sense to add and is not hand-wavey (fails on the pre-fix double-count and asserts exact totals on both names) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/dc0e2d7e95374c7eb7fcbaf0d21ff1f6
Requested by: @mateo-berri

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ krrish-berri
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes missing cache-write token billing for OpenAI's gpt-5.6 family, which returns cache-write counts in prompt_tokens_details.cache_write_tokens (Chat Completions) and input_tokens_details.cache_write_tokens (Responses API) — fields LiteLLM previously dropped, causing those tokens to be billed at the plain input rate instead of the 1.25× cache-creation rate.

• Adds cache_write_tokens as the canonical field on PromptTokensDetailsWrapper with a __setattr__ mirror keeping cache_creation_tokens in sync; Usage.__init__ maps the Anthropic cache_creation_input_tokens param onto cache_write_tokens so both provider dialects converge on one representation.
• Carries cache_write_tokens through the Responses API usage transform (the LIT-4633 route) and derives cache_creation_input_tokens in spend logs from it, including a fallback to the normalized usage_object for the /v1/responses path where no chat-shaped prompt_tokens_details is available.
• Fixes a double-count regression in combine_usage_objects via _summable_prompt_token_fields, which excludes the mirrored cache_creation_tokens from the per-field accumulation loop and lets __setattr__ keep it in sync; all new tests use the local cost map and mock objects with no network calls.

Confidence Score: 5/5

Safe to merge — changes are scoped to token-detail normalization and spend-log derivation, with no schema changes or backwards-incompatible API surface modifications.

The fix is well-contained: the setattr mirror is correctly guarded so that Pydantic's internal field initialization is handled by the explicit assignment in init; the double-count fix via _summable_prompt_token_fields is logically sound and covered by an exact-value regression test; the Anthropic path is guarded so existing values are never clobbered. Tests are all mock-based and cover the edge cases.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/types/utils.py	Adds cache_write_tokens as canonical field on PromptTokensDetailsWrapper with __setattr__ mirror to cache_creation_tokens; __init__ normalization ensures both fields stay in sync and are deleted together when absent.
litellm/cost_calculator.py	Adds _summable_prompt_token_fields to prevent double-counting the mirrored cache_write_tokens/cache_creation_tokens pair in combine_usage_objects; correctly skips cache_creation_tokens and lets the __setattr__ mirror handle it.
litellm/litellm_core_utils/llm_cost_calc/utils.py	Reads cache_write_tokens first (OpenAI name) then falls back to cache_creation_tokens (Anthropic name) in _parse_prompt_tokens_details; removes now-redundant kimi-k2 fallback in get_token_type_cost_breakdown since _parse_prompt_tokens_details covers it.
litellm/proxy/spend_tracking/spend_tracking_utils.py	Refactors prompt_tokens_details lookup to check the Responses API usage_object fallback and derives cache_creation_input_tokens from cache_write_tokens in spend logs; Anthropic-populated cache_creation_input_tokens is guarded against overwrite.
litellm/responses/utils.py	Passes cache_write_tokens through in _transform_response_api_usage_to_chat_usage for the typed-object path; the dict path already works via PromptTokensDetailsWrapper(**input_tokens_details).
tests/test_litellm/test_cost_calculator.py	Adds regression test verifying combine_usage_objects sums the mirrored field pair exactly once (single 50-token object stays 50; two 50-token objects combine to 100).
tests/test_litellm/test_utils.py	Adds four targeted wrapper tests: OpenAI cache_write_tokens populates both names; Anthropic cache_creation_input_tokens normalizes onto both names; no cache-write fields when absent; bidirectional sync on post-construction assignment.
tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py	Adds billing regression tests for gpt-5.6 cache-write tokens via Chat Completions and Responses API; asserts exact expected cost at cache_creation_input_token_cost rate using local cost map.
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py	Adds four spend-log tests: OpenAI cache_write_tokens maps to cache_creation_input_tokens; Anthropic value is preserved; zero/absent counts stay absent; Responses API fallback path reads from usage_object.
tests/test_litellm/responses/test_responses_utils.py	Adds LIT-4633 regression test asserting _transform_response_api_usage_to_chat_usage preserves input_tokens_details.cache_write_tokens and mirrors it to cache_creation_tokens on the resulting Usage object.

_{Reviews (4): Last reviewed commit: "fix(cost_calculator): sum mirrored cache..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_openai_cache_write_tokens (d3f5c6d) with litellm_internal_staging (c255f53)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (d3f5c6d) during the generation of this report, so 9239351 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[devin-ai-integration]

Proof of fix: OpenAI cache-write tokens now logged and billed at the 1.25x rate

Ran the proxy locally against the real OpenAI API (openai/gpt-5.6-luna) with a Postgres spend DB, forcing a fresh prompt-cache write (~20k-token unique prompt) on the base commit vs this branch. Same request, viewed on the Admin UI Logs page

BEFORE (3810130105): cost $0.021115 (plain input rate); the log detail has no "Cache Creation Tokens" row and metadata additional_usage_values holds only cache_write_tokens: 21040, so the cache-creation tokens are dropped from spend

AFTER (this branch): same call now bills $0.025750; the log shows "Cache Creation Tokens: 20,540" and the Cost Breakdown charges them as Cache Write Cost $0.025675 (= 20540 x 1.25e-6), with cache_creation_input_tokens populated in the spend log and daily metrics

Before vs after QA recording

Admin UI Logs - BEFORE (no cache creation row) vs AFTER (row present, billed 1.25x)

Spend log rows

AFTER  chatcmpl-E3uIh9...  spend 0.02575   cache_creation_input_tokens 20540
BEFORE chatcmpl-E3uGb...   spend 0.021115  cache_creation_input_tokens (null)

Tested via Devin: https://app.devin.ai/sessions/54e6ef97949f4a85942817e18e84e587

[mateo-berri]

The bidirectional sync between cache_write_tokens and cache_creation_tokens runs only in init. Any code that assigns to one field directly on an existing instance (e.g. wrapper.cache_write_tokens = X without also updating cache_creation_tokens) will leave the pair silently out of sync. The PR handles all current mutation sites correctly (the Anthropic else-branch in Usage.init sets both manually), but this pattern is easy to miss for future contributors. A @field_validator or a setattr override that keeps the two fields in sync would make the invariant self-enforcing.

Is this a legit concern? Or no

[devin-ai-integration]

@mateo-berri re the sync-only-in-__init__ concern: legit, fixed in d76eb5d. PromptTokensDetailsWrapper now overrides __setattr__ so assigning either cache_write_tokens or cache_creation_tokens mirrors to the other, making the invariant self-enforcing instead of relying on every call site to set both. The Anthropic else-branch in Usage.__init__ is simplified to set only cache_write_tokens, and test_prompt_tokens_details_cache_write_creation_stay_in_sync_on_assignment covers post-construction assignment in both directions.

Same commit also lands the exact LIT-4633 repro. That Nvidia case comes through the Responses API (/v1/responses), whose usage transform _transform_response_api_usage_to_chat_usage rebuilt the token details and dropped cache_write_tokens. Now mapped there too, with test_transform_response_api_usage_maps_cache_write_tokens.

[devin-ai-integration]

Responses API QA (LIT-4633): cache-write tokens now logged + billed on /v1/responses

Ran the proxy against the real OpenAI API on /v1/responses (gpt-5.6-luna, fresh ~20k-token unique prompt, cached_tokens: 0), Postgres spend DB, base 3810130105 vs this branch at 04d603eb67.

The gpt-5.6 Responses route dropped input_tokens_details.cache_write_tokens in the usage transform, so cache creation was neither logged nor billed. After the fix the same call bills at the 1.25x cache-write rate ($0.025036 vs plain-rate before), the spend log's metadata.additional_usage_values now carries cache_creation_input_tokens: 20002 (was empty {}), and the Admin UI Logs detail renders the "Cache Creation Tokens: 20,002" row plus the "(20,002 tokens)" suffix on the Cache Write Cost line. Daily metrics and Prometheus litellm_input_cache_creation_tokens_metric_total also populate.

Before vs after QA recording

After fix: Cache Creation Tokens row + token suffix render on the responses path

Before this commit: row + suffix missing

Tested via Devin: https://app.devin.ai/sessions/54e6ef97949f4a85942817e18e84e587

[shivamrawat1]

@gerptile review again with new commits that resolve p2 issue

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[devin-ai-integration]

QA on tip: prompt-cache-creation (cache_write) billing verified live

Ran a live before/after against real openai/gpt-5.6-luna through two proxies sharing one local Postgres, sending fresh ~44k-token unique prefixes each time so the provider reports cached_tokens: 0 and a large cache_write_tokens, forcing a real prompt-cache write. Verified billing rate, response usage, Postgres spend-logs, and the Admin UI Logs detail for both the Chat Completions and Responses API routes

• TIP (on tip): 6fd568c9c82a0e02f75c5ad0ca8223da691dd428
• BASE (merge-base with litellm_internal_staging): 34561482ed092d78c296cab7999486022af5a938

Route	Build	x-litellm-response-cost	prompt tok	rate / tok	spend-log cache_creation_input_tokens
Chat	BASE	$0.044601	44499	1.0023e-6	NULL
Chat	TIP	$0.055717	44555	1.2505e-6	44552
Responses	BASE	$0.044511	44481	1.0007e-6	NULL
Responses	TIP	$0.055816	44557	1.2527e-6	44554

The per-token rate is the billing proof: BASE bills cache-creation tokens at the plain input rate (~1.0e-6) and drops them from spend logs and the UI; TIP bills at the 1.25x cache-write rate and surfaces cache_creation_input_tokens in the spend log and the Admin UI

Response usage on TIP also mirrors the two names (chat prompt_tokens_details carries both cache_write_tokens and cache_creation_tokens: 44552) and the Responses path now maps cache_write_tokens through instead of dropping it

Before vs after QA recording

Admin UI Logs detail: TIP (after)

Chat: "Cache Creation Tokens: 44,552" row, and the Cost Breakdown is now consistent (Input $0.00000300 + Cache Write $0.05569000 (44,552 tokens) + Output $0.00002400 = $0.05571700)

Responses API (LIT-4633): "Cache Creation Tokens: 44,554" row, and "Cache Write Cost: $0.05569250 (44,554 tokens)"

BASE (before): bug visible

Chat on BASE has no logged cache creation and the breakdown is internally inconsistent: a negative Input Cost (-$0.01112100) and an itemized Cache Write Cost ($0.05562000) that does not match the billed total ($0.04460100, plain rate). TIP makes itemization and total agree and bills at 1.25x

Responses on BASE shows no cache-write handling at all (Input + Output only, plain rate), the clean LIT-4633 repro

[mateo-berri]

@greptileai

[mateo-berri]

Live QA for 44c7d62: mid-stream fallback no longer doubles cache tokens

Ran the proxy from this branch on a random port with a Postgres spend DB. Model group haiku-broken is anthropic/claude-haiku-4-5 pointed at a local mock that streams a couple of real-format SSE events and then a genuine overloaded_error event mid-generation, with a router fallback to haiku-real on the real Anthropic API. Each run sends a streaming chat completion with a fresh ~35k-token cache_control system prefix (forcing a real cache write) and stream_options.include_usage, which exercises _combine_fallback_usage -> combine_usage_objects on the fallback's real usage

curl -s http://127.0.0.1:52717/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d @body.json   # streaming, fresh random cached prefix per run

Before (6fd568c9c8, this PR without the combine fix), the final usage chunk returned to the client doubles the mirrored pair; 69640 = 2 x 34820 actual cache-write tokens, more than the entire 34895-token prompt

"usage":{"completion_tokens":7,"prompt_tokens":34895,"total_tokens":34902,
  "prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0,"text_tokens":75,
  "cache_write_tokens":69640,"cache_creation_tokens":69640}}

After (44c7d622d4), the same scenario returns the exact pair once, and the arithmetic closes: 34878 + 76 = 34954

"usage":{"completion_tokens":5,"prompt_tokens":34954,"total_tokens":34959,
  "prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0,"text_tokens":76,
  "cache_write_tokens":34878,"cache_creation_tokens":34878}}

# LiteLLM_SpendLogs
prompt_tokens      | 34954
spend              | 0.0436985      <- 76 x 1e-6 + 34878 x 1.25e-6 + 5 x 5e-6
log_cache_creation | 34878

One precision on blast radius: on this fallback surface the spend log was correct even before the fix because get_logging_payload re-derives usage from the raw chunks; what the double-count corrupted here is the usage returned to the client, which is what callers meter on. The other combine_usage_objects callers (Anthropic batches transform, realtime), where the combined object feeds cost directly, are pinned by the unit regression test added in the same commit

[mateo-berri]

LGTM; thanks!

[devin-ai-integration]

QA on tip (follow-up d3f5c6d): streaming cache-creation tokens logged once (1x), not doubled

Re-ran the live before/after against real openai/gpt-5.6-luna through proxies sharing one local Postgres, fresh ~44k-token unique prefixes each time so the provider reports cached_tokens: 0 and a large cache_write_tokens, forcing a real prompt-cache write

• TIP (on tip): d3f5c6dbf6e5dc13821af5cce90ed212e35fc835
• BASE (pre-feature): c255f53bfb85010c8f895aa29154ca675c200721
• BUGGY (prior tip, mirror in place but no combine fix): 6fd568c9c82a0e02f75c5ad0ca8223da691dd428

This follow-up lives in combine_usage_objects. Because the __setattr__ mirror keeps cache_write_tokens and cache_creation_tokens equal, the old field-by-field sum counted the same value twice. Direct call proves the fix: BUGGY sums the mirrored pair twice (1000 -> 2000), TIP collapses to one representative (1000 -> 1000)

Route	Build	x-litellm-response-cost	prompt tok	rate / tok	spend-log cache_creation_input_tokens
Stream	BASE	$0.044626	44602	1.00e-6	NULL
Stream	TIP	$0.055821	44638	1.25e-6	44635 (1x)
Chat	TIP	$0.055867	44675	1.25e-6	44672
Responses	TIP	$0.055863	44667	1.25e-6	44664

Key streaming assertion: on TIP the logged and billed cache-creation count is 44,635, which equals the provider's cache_write_tokens (1x, roughly the prompt token count), not ~88k. One caveat worth stating plainly: a single plain streaming request does not itself exercise the doubling, because streaming aggregates via ChunkProcessor.calculate_usage (last-wins) rather than combine_usage_objects. The double-count surfaces on merge paths that call combine_usage_objects (router/mid-stream fallback usage merges, Anthropic batch, realtime), which is what the direct 2000-vs-1000 proof above covers; the live stream confirms the normal path still logs 1x

Before vs after QA recording

Admin UI Logs: TIP streaming (after)

Cache Creation Tokens: 44,635; breakdown Input $0.00000300 + Cache Write $0.05579375 (44,635 tokens) + Output $0.00002400 = $0.05582075

TIP non-streaming chat and Responses API rows show the same 1.25x billing and populated cache-creation count

BASE streaming (before)

No "Cache Creation Tokens" row and billed at the plain rate $0.04462600, spend-log cache_creation_input_tokens NULL (the UI recomputes a broken negative Input Cost from the raw cache_write_tokens)

[devin-ai-integration]

QA, live mid-stream router fallback: the combine_usage_objects doubling is real at the unit level but does not reach live billing through this path

Followed up on the request for a live end-to-end router-fallback repro. Built a fault-injected primary (a tiny local OpenAI-compatible SSE server that emits one content chunk then drops the connection mid-stream, which LiteLLM maps to APIConnectionError -> MidStreamFallbackError) with a real openai/gpt-5.6-luna fallback, so the fallback's real cache-write usage flows through Router._combine_fallback_usage -> combine_usage_objects. Two proxies sharing one Postgres, fresh ~44k-token unique prefixes to force real cache writes

• BUGGY (mirror in place, no combine fix): 6fd568c9c82a0e02f75c5ad0ca8223da691dd428
• TIP (this fix): d3f5c6dbf6e5dc13821af5cce90ed212e35fc835

Unit anchor, direct combine_usage_objects call against each build's real code: BUGGY doubles a single mirrored object (44487 -> 88974), TIP sums once (44487 -> 44487). The bug and the fix are exactly as described

Honest, unbiased live result: through the mid-stream fallback path the doubling does NOT reach any customer-visible number, on either build. Fallback fired on both proxies (MidStreamFallbackError -> Falling back to model_group = gpt-5.6-luna -> Successful fallback b/w models), and the final numbers were effectively identical

Metric	BUGGY	TIP
Fallback model	openai/gpt-5.6-luna	openai/gpt-5.6-luna
Streamed cache_creation_tokens	44,545 (1x)	44,629 (1x)
Any ~88k doubled value live?	No	No
Spend-log spend	$0.05579225	$0.05588525
Spend-log cache_creation_input_tokens	44,545 (1x)	44,629 (1x)
Effective rate (spend / prompt tok)	1.2524e-6	1.2521e-6

Dollar delta attributable to the doubling: $0.00. The reason: the final streamed usage and the spend log for a recovered fallback come from the real fallback deployment's own accounting (1x, via ChunkProcessor, which overwrites on the same request id), so the doubled combine_usage_objects output is effectively inert in this path. The fix is still correct and worth keeping (it removes a latent double-count and makes the unit behavior provably 1x), but a customer would not have seen doubled billing through mid-stream fallback

One unrelated note observed on this path: in mid-stream fallback the response cost headers reflect the primary deployment (x-litellm-response-cost-original: 0.0, model-id flaky-primary-1) and there is no positive x-litellm-response-cost header on the streamed response; the billed cost lands in the spend log and UI. Identical on both builds, so it's a pre-existing fallback quirk, not related to this fix

If it's worth confirming whether the doubling ever bites a real billed path, the other combine_usage_objects callers are Anthropic batch cost calc and realtime usage merge; those are a separate investigation from mid-stream fallback

Before vs after (BUGGY vs TIP) recording

Admin UI Logs, TIP fallback (1x, billed 1.25x)

Admin UI Logs, BUGGY fallback (also 1x, no doubling live)

[devin-ai-integration]

QA, Anthropic batch + realtime: the combine_usage_objects doubling does not reach real billing here either

Probed the other two combine_usage_objects callers live, same unbiased mandate as the fallback run

• BUGGY (mirror in place, no combine fix): 6fd568c9c82a0e02f75c5ad0ca8223da691dd428
• TIP (this fix): d3f5c6dbf6e5dc13821af5cce90ed212e35fc835

Anthropic Message Batches: real claude-sonnet-5 batches with an ephemeral cache_control block forcing a real cache write, created and retrieved through LiteLLM (passthrough create -> poll to ended -> unified /v1/batches/{id} retrieve, which writes the spend log)

Measurement	BUGGY	TIP
AnthropicBatchesConfig.transform_response combine (the caller)	29,606 (2x)	14,803 (1x)
Real cost/spend path calculate_batch_cost_and_usage (per-item)	14,803 (1x)	14,803 (1x)
Live spend-log cache_creation_input_tokens	14,835 (1x, = provider)	14,781 (1x, = provider)
Live spend-log spend	$0.01856175	$0.01849425

The combine bug does fire on real Anthropic tokens: BUGGY's transform_response doubles cache-creation to 2x, TIP stays 1x, so the fix works on real data. But the batch cost/spend that customers actually see is computed per-item in _handle_completed_batch -> calculate_batch_cost_and_usage, which never consumes the combined object, so the retrieved usage, computed cost, and Postgres spend are 1x on both builds. Dollar delta to customer billing: $0.00

Realtime usage merge: does not apply. OpenAI realtime reports only cached_tokens (cache READ), which is not part of the mirrored cache_write/cache_creation pair, and realtime models have no cache_creation_input_token_cost and no prompt-caching write. So this caller cannot exhibit the doubling regardless of the build

Net across all three combine_usage_objects callers (mid-stream fallback, Anthropic batch, realtime): d3f5c6d is a correct defensive fix that makes the combine output provably 1x, but none of the live paths produced customer-visible doubled billing. The doubling was latent, not customer-facing, on the paths that exist today

QA recording (BUGGY vs TIP batch spend detail)

Admin UI Logs, batch spend rows (both 1x)

TIP: Cache Creation Tokens 14,781 (1x), $0.01849425

BUGGY: Cache Creation Tokens 14,835 (1x), $0.01856175