fix: 修复智普、Moonshot渠道在stream=true时无法拿到cacheTokens的统计数据。

[shikaiwei1]

根本原因：

1. 在OaiStreamHandler流式处理函数中，调用applyUsagePostProcessing(info, usage, nil)时传入的responseBody为nil，导致无法从响应体中提取缓存tokens。
2. 两个渠道的cached_tokens位置不同：

• 智普：标准位置 usage.prompt_tokens_details.cached_tokens
• Moonshot：非标准位置 choices[].usage.cached_tokens

处理方案：

1. 传递body信息到applyUsagePostProcessing中
2. 拆分智普和Moonshot的解析，并为Moonshot单独写一个解析方法。

Summary by CodeRabbit

Bug Fixes

• Improved accuracy of cached token tracking for Moonshot integration with enhanced fallback mechanisms.
• Refined token usage post-processing to better handle non-standard response formats.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Modified OaiStreamHandler to pass last streamed data to applyUsagePostProcessing; refactored ChannelType handling to give Moonshot a dedicated case with custom cached-tokens extraction from non-standard response body structure; added extractMoonshotCachedTokensFromBody helper function with fallback chain.

Changes

Cohort / File(s)

Summary

Usage Post-Processing Refactoring relay/channel/openai/relay-openai.go

Separated Moonshot from Zhipu_v4 combined case into dedicated handler; added custom extractMoonshotCachedTokensFromBody helper to parse non-standard choices[].usage.cached_tokens location; implemented fallback chain (InputTokensDetails → Moonshot-specific extractor → standard extractor → PromptCacheHitTokens); OaiStreamHandler now passes last streamed data as bytes to applyUsagePostProcessing instead of nil.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

• fix(openai): account cached tokens for zhipu_v4 usage #1982: Extends shared work on usage post-processing and cached-token extraction logic with similar modifications to relay-openai.go.
• 为Moonshot添加缓存tokens读取逻辑 #2493: Modifies applyUsagePostProcessing to add Moonshot cached_tokens handling with an alternative grouping approach.

Suggested reviewers

• seefs001

Poem

🐰 Hops through streams with bytes so bright,
Moonshot's tokens found just right,
Non-standard paths the rabbit maps,
Fallback chains close every gap! ✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: fixing cache token statistics retrieval for Zhipu and Moonshot channels in streaming mode.

✨ Finishing touches

• 📝 Generate docstrings

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 48d358f and ab81d6e.

📒 Files selected for processing (1)

• relay/channel/openai/relay-openai.go

🧰 Additional context used

🧠 Learnings (2)

📚 Learning: 2025-08-21T06:31:11.073Z

Learnt from: jiajunly
Repo: QuantumNous/new-api PR: 1629
File: relay/channel/openai/relay-openai.go:170-174
Timestamp: 2025-08-21T06:31:11.073Z
Learning: In relay/channel/openai/relay-openai.go, the streaming logic for the AddThinkFirst feature is designed so that only the first chunk of a stream gets the "<think>\n" prefix. The final flush in the streaming handler intentionally uses addThink=false because the last chunk should never receive the prefix, even in single-chunk streams where the prefix would have been applied during normal processing.

Applied to files:

• relay/channel/openai/relay-openai.go

📚 Learning: 2025-06-21T03:37:41.726Z

Learnt from: 9Ninety
Repo: QuantumNous/new-api PR: 1273
File: relay/channel/gemini/relay-gemini.go:97-116
Timestamp: 2025-06-21T03:37:41.726Z
Learning: In relay/channel/gemini/relay-gemini.go, the thinking budget calculation logic (including the MaxOutputTokens multiplication) was introduced in PR #1247. PR #1273 focused specifically on decoupling the thoughts summary feature from thinking budget settings and did not modify the existing thinking budget behavior.

Applied to files:

• relay/channel/openai/relay-openai.go

🔇 Additional comments (4)

relay/channel/openai/relay-openai.go (4)

189-189: LGTM! Core fix for streaming cached_tokens extraction.

This change correctly passes the last stream data to applyUsagePostProcessing, enabling extraction of cached tokens that was previously impossible when nil was passed. The downstream functions handle empty/nil body gracefully.

---

599-609: LGTM! Clean separation of Zhipu_v4 handling.

The fallback chain for extracting cached tokens is well-structured, prioritizing the most reliable sources first.

---

610-622: LGTM! Comprehensive fallback chain for Moonshot.

The extraction order correctly prioritizes the Moonshot-specific location (choices[].usage.cached_tokens) before falling back to standard locations. This ensures proper extraction regardless of which format Moonshot returns.

---

657-684: LGTM! Well-implemented extraction for Moonshot's non-standard format.

The function correctly handles edge cases (empty body, unmarshal errors) and uses pointer type for CachedTokens to distinguish between zero and missing values. The iteration through choices handles the case where usage might appear in different choice indices.

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}