fix: correct token usage conversion between Claude and OpenAI formats

[zer0kara]

问题描述

Claude API 的 input_tokens 不包含缓存命中的 tokens，而 OpenAI API 的 prompt_tokens 包含缓存 tokens。
当前代码直接将两者等同，导致格式转换时 usage 数据错误。

修复内容

1. Claude→OpenAI（relay-claude.go）：
   PromptTokens = InputTokens + CacheReadInputTokens
2. OpenAI→Claude（convert.go）：
   InputTokens = PromptTokens - CachedTokens

相关PR

#2477

Summary by CodeRabbit

• Bug Fixes
  • Corrected token usage accounting to accurately reflect cached input tokens in API response metrics and usage reports.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Updates token usage calculations in Claude relay and service conversion code to properly account for cached tokens. Specifically, PromptTokens now includes CacheReadInputTokens, and InputTokens are reduced by CachedTokens in OpenAI-to-Claude conversions to ensure accurate token accounting.

Changes

Cohort / File(s)	Summary
Claude relay token calculations relay/channel/claude/relay-claude.go, go.mod	Updated PromptTokens to combine InputTokens and CacheReadInputTokens across message_start, message_delta, and finalization paths; recomputed TotalTokens accordingly.
Service conversion token adjustments service/convert.go	Modified OpenAI-to-Claude response conversion functions to subtract CachedTokens from InputTokens in streaming and non-streaming paths, ensuring cached tokens are properly deducted from reported input token counts.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• QuantumNous/new-api#2670: Adjusts Claude-related token accounting for cached-token subtraction and usage tagging in Claude paths.
• QuantumNous/new-api#1120: Modifies token-usage accounting in service/convert.go with cached token field adjustments affecting prompt/input totals.
• QuantumNous/new-api#1531: Modifies Claude/OpenAI conversion functions in service/convert.go (e.g., StreamResponseOpenAI2Claude).

Suggested reviewers

• seefs001

Poem

🐰 Cached tokens dance in the relay,
No longer double-counted today,
PromptTokens and Inputs align,
The accounting now sparks and will shine! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: correcting token usage conversion between Claude and OpenAI API formats. It is concise, clear, and directly reflects the primary objective of the changeset.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

service/convert.go (1)

322-326: Prevent negative InputTokens when cached tokens exceed prompt tokens.

These subtractions can go negative if upstream usage is inconsistent (e.g., cached tokens reported but prompt tokens missing/underreported), which can break downstream billing or assertions. Clamp to zero before assigning.

🐛 Proposed fix (apply to all four occurrences)

-						InputTokens:              oaiUsage.PromptTokens - oaiUsage.PromptTokensDetails.CachedTokens,
+						InputTokens:              promptTokensExcludingCache(oaiUsage.PromptTokens, oaiUsage.PromptTokensDetails.CachedTokens),

-						InputTokens:              oaiUsage.PromptTokens - oaiUsage.PromptTokensDetails.CachedTokens,
+						InputTokens:              promptTokensExcludingCache(oaiUsage.PromptTokens, oaiUsage.PromptTokensDetails.CachedTokens),

-						InputTokens:              oaiUsage.PromptTokens - oaiUsage.PromptTokensDetails.CachedTokens,
+						InputTokens:              promptTokensExcludingCache(oaiUsage.PromptTokens, oaiUsage.PromptTokensDetails.CachedTokens),

-		InputTokens:          openAIResponse.PromptTokens - openAIResponse.PromptTokensDetails.CachedTokens,
+		InputTokens:          promptTokensExcludingCache(openAIResponse.PromptTokens, openAIResponse.PromptTokensDetails.CachedTokens),

+func promptTokensExcludingCache(promptTokens, cachedTokens int) int {
+	if cachedTokens < 0 {
+		cachedTokens = 0
+	}
+	if promptTokens < cachedTokens {
+		return 0
+	}
+	return promptTokens - cachedTokens
+}

Also applies to: 350-354, 479-483, 535-538

[PMExtra]

我疑似遇到了这个场景，看起来这个PR似乎能解决问题，求尽快合入。

场景：
通过 claude-code 客户端访问 new-api 提供的 GLM 模型，上游类型是 Siliconflow 。
然后产生的费用日志中，计费的【提示】token数包含了【缓存】token数，也就是缓存token被重复计费了。

举例：