fix(stats): include Anthropic cache tokens in usage rankings token_used

[broven]

Problem

The usage rankings / quota_data export records token_used as prompt_tokens + completion_tokens (model/log.go, in RecordConsumeLog).

• OpenAI-style usage: prompt_tokens already includes cache-read tokens, so prompt_tokens + completion_tokens is the full token volume. ✅
• Anthropic-style usage: cache-read (cache_tokens) and cache-creation (cache_creation_tokens) are reported separately and are not part of prompt_tokens. They were therefore dropped from token_used entirely. ❌

For cache-heavy Claude traffic, cache tokens dominate, so the model is massively undercounted in the leaderboard even though its quota (billing) is computed correctly from the cache tokens + ratios.

Observed (production, 24h, one Claude Opus model)

metric	value
rankings token_used (prompt+completion)	~1.96M
real cache-read tokens	~11.7M
real cache-creation tokens	~51.0M
real total volume	~145M (98%+ cache)

The leaderboard showed ~1.4% of the real token volume. OpenAI models on the same instance matched their downstream stats exactly, which isolated the issue to the Anthropic-only separate cache accounting.

Fix

Compute token_used in a small provider-aware helper statsTokenUsage:

• add cache-read + cache-creation tokens back only for anthropic usage semantics (where they are excluded from prompt_tokens);
• OpenAI-style usage is unchanged (no double counting).

The discriminator is other["usage_semantic"] == "anthropic", which is set in exactly the same place the cache tokens are excluded from prompt_tokens, so the two stay in sync. The cache counts are read from the existing other payload, so no call-site or struct-signature changes are needed.

This only affects the statistics token count; quota / billing is untouched. Historical quota_data rows are not rewritten by this change.

Tests

Added model/log_stats_test.go covering OpenAI (no double-count), Anthropic with/without cache tokens (including the real opus-4-8 shape), float64 payloads, and nil other.

go test ./model/ -run TestStatsTokenUsage -v

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• Bug Fixes

  • Improved token usage accounting to correctly calculate and track tokens across different AI providers, with proper support for cache tokens in Anthropic-based integrations.

• Tests

  • Added test coverage for token usage calculation scenarios across multiple provider types and edge cases.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ff1f9a89-508b-4d08-b9b7-eb726c020223

📥 Commits

Reviewing files that changed from the base of the PR and between d2576dd and f8c0b3a.

📒 Files selected for processing (2)

• model/log.go
• model/log_stats_test.go

---

Walkthrough

The PR refactors token accounting for quota logging to separately compute token totals via a new statsTokenUsage helper. This helper defaults to prompt + completion tokens, but when usage_semantic indicates Anthropic-style usage, it additionally includes cache and cache creation tokens read from params.Other. A companion otherTokenCount helper safely extracts numeric values with type tolerance. The computed total is passed into LogQuotaData asynchronously, replacing inline arithmetic.

Changes

Token Usage Accounting for Cache-Aware Logging

Layer / File(s)	Summary
Token counting helpers model/log.go (lines 276–308)	statsTokenUsage sums prompt and completion tokens, and for Anthropic-style usage adds cache and cache creation token counts from params.Other. otherTokenCount tolerates int, int64, and float64 types in the Other map, casting to int and returning 0 for unsupported/missing keys.
LogQuotaData integration and test coverage model/log.go (lines 269–274), model/log_stats_test.go	RecordConsumeLog calls statsTokenUsage(params) and passes the computed tokenUsed into the LogQuotaData goroutine. Test cases validate token totals across OpenAI defaults, Anthropic cache scenarios, nil/missing Other fields, and numeric type conversions.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• QuantumNous/new-api#2811: Both PRs change how Anthropic/Claude cached and cache creation token counts are accounted for—main PR adds these cache tokens into the logged token totals via statsTokenUsage, while the retrieved PR populates Claude usage fields (PromptTokensDetails.CachedTokens / CachedCreationTokens) so that cached token usage is preserved for billing/logging.
• QuantumNous/new-api#2477: Both PRs adjust Anthropic token/cache accounting—main PR changes statsTokenUsage to include cache_tokens/cache_creation_tokens in the recorded token total, while retrieved PR fixes postConsumeQuota to avoid deducting those cache tokens again for ChannelTypeAnthropic.

Suggested reviewers

• seefs001

Poem

🐰 Tokens cache and tokens flow,
Anthropic's bounty now we know,
Type-safe counts in maps of gold,
Quota logs complete and bold!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately identifies the main change: adding Anthropic cache token support to usage stats. It's specific and clear about both the problem domain (stats) and the fix (include cache tokens).
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[noir017]

Independent confirmation of this from another deployment, plus one correction worth folding in before merge.

Same root cause, same magnitude. On new-api v1.0.0-rc.24 (revision 5c3abff), a Claude-native channel over 7 days:

metric	value
quota_data.token_used (prompt+completion)	13,874,185
real cache-read tokens	2,584,841,713
real cache-creation tokens	59,209,621
real total volume	2,657,251,579

The dashboard and rankings showed 0.52% of real volume — close to the 1.4% reported here. OpenAI-semantic models on the same instance (deepseek et al.) matched exactly, isolating it to the Anthropic separate-cache accounting. Billing/quota was correct throughout.

Correction — cache_write_tokens is an alias, not an extra component. Over 10,914 production rows on one model, three keys carried byte-identical sums:

cache_creation_tokens     59,209,621
cache_write_tokens        59,209,621
cache_creation_tokens_5m  59,209,621

text_quota.go writes cache_write_tokens as the normalized cache-creation total (its own comment says "normalized cache creation total for UI display"; it's the 5m+1h sum, falling back to cache_creation_tokens). So summing only cache_tokens + cache_creation_tokens as this PR does is correct — but it's worth an explicit test, since cache_write_tokens looks like an independent figure and adding it would silently double the cache-creation half. Suggested case:

{
    name: "anthropic cache_write_tokens alias must not double count",
    params: RecordConsumeLogParams{
        PromptTokens: 2, CompletionTokens: 535,
        Other: map[string]interface{}{
            "usage_semantic":           "anthropic",
            "cache_tokens":             28097,
            "cache_creation_tokens":    281100,
            "cache_write_tokens":       281100,  // alias of cache_creation_tokens
            "cache_creation_tokens_5m": 281100,  // ditto
        },
    },
    want: 2 + 535 + 28097 + 281100,
},

Also worth noting input_tokens_total is not a shortcut here: text_quota.go only writes it when the final request format is not RelayFormatClaude, so it is absent (0) for exactly the Claude-native traffic that needs fixing.

Rebase note. This no longer applies cleanly: LogQuotaData has since moved to a QuotaDataLogParams struct, so the changed line is now TokenUsed: params.PromptTokens + params.CompletionTokens inside a struct literal. The statsTokenUsage/otherTokenCount helpers are unaffected.

Adapted this onto rc.24 and it has been running in production since 2026-08-28 — new quota_data rows now track real volume, and quota is byte-identical to before. Would be glad to see this land, or a token stats include cache toggle if the current accounting is deliberate.