fix: Claude 流式断流时不再整份覆盖 usage，保留 cache 计费字段 - #4128
Conversation
HandleStreamFinalResponse 在 !Done 时调用 ResponseText2Usage 整份覆盖 claudeInfo.Usage，导致 message_start 已获取的 CacheReadInputTokens、 CacheCreationInputTokens 等字段丢失，prompt 退化为占位值 1。 修复： - 只补缺失的 CompletionTokens/PromptTokens，保留已有 cache 数据 - PromptTokens 兜底改用 info.GetEstimatePromptTokens()（与其他渠道对齐） Fixes QuantumNous#4127
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughHandleStreamFinalResponse in the Claude relay now preserves existing usage fields from message_start and selectively backfills only missing PromptTokens and CompletionTokens (using an estimated prompt token when needed), then recomputes TotalTokens instead of replacing the entire Usage object. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@relay/channel/claude/relay-claude.go`:
- Around line 814-816: The fallback only backfills
claudeInfo.Usage.CompletionTokens when it's exactly 0, but interrupted streams
can leave the placeholder value 1 (from message_start) and still be
!claudeInfo.Done; update the check in the block that touches claudeInfo (the
clause using claudeInfo.Usage.CompletionTokens and fallback.CompletionTokens) to
treat the placeholder as unset (e.g., CompletionTokens <= 1) so you copy
fallback.CompletionTokens when CompletionTokens is 0 or the placeholder 1;
ensure this logic executes in the same place where claudeInfo.Done is considered
so interrupted-stream counts get corrected.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 74d324a0-02f0-464d-82ff-fd0eafea2a48
📒 Files selected for processing (1)
relay/channel/claude/relay-claude.go
| if claudeInfo.Usage.CompletionTokens == 0 { | ||
| claudeInfo.Usage.CompletionTokens = fallback.CompletionTokens | ||
| } |
There was a problem hiding this comment.
Fallback still undercounts interrupted streams when CompletionTokens is placeholder 1.
When !claudeInfo.Done, CompletionTokens can still be the message_start placeholder value (e.g., 1), but Line 814 only backfills when it is 0. This leaves severe undercounting in exactly the abnormal-stream path this block is handling (see relay/channel/claude/relay_claude_test.go:12-48, where message_start uses OutputTokens: 1).
💡 Suggested fix
- if claudeInfo.Usage.CompletionTokens == 0 {
+ if claudeInfo.Usage.CompletionTokens == 0 ||
+ (!claudeInfo.Done && fallback.CompletionTokens > claudeInfo.Usage.CompletionTokens) {
claudeInfo.Usage.CompletionTokens = fallback.CompletionTokens
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if claudeInfo.Usage.CompletionTokens == 0 { | |
| claudeInfo.Usage.CompletionTokens = fallback.CompletionTokens | |
| } | |
| if claudeInfo.Usage.CompletionTokens == 0 || | |
| (!claudeInfo.Done && fallback.CompletionTokens > claudeInfo.Usage.CompletionTokens) { | |
| claudeInfo.Usage.CompletionTokens = fallback.CompletionTokens | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@relay/channel/claude/relay-claude.go` around lines 814 - 816, The fallback
only backfills claudeInfo.Usage.CompletionTokens when it's exactly 0, but
interrupted streams can leave the placeholder value 1 (from message_start) and
still be !claudeInfo.Done; update the check in the block that touches claudeInfo
(the clause using claudeInfo.Usage.CompletionTokens and
fallback.CompletionTokens) to treat the placeholder as unset (e.g.,
CompletionTokens <= 1) so you copy fallback.CompletionTokens when
CompletionTokens is 0 or the placeholder 1; ensure this logic executes in the
same place where claudeInfo.Done is considered so interrupted-stream counts get
corrected.
message_start 阶段可能给 CompletionTokens 非零占位值， 只检查 == 0 不够，加上 !Done && fallback > current 条件。
Calcium-Ion
added
the
ready to merge
seefs001
removed
the
ready to merge
…rite fix: Claude 流式断流时不再整份覆盖 usage，保留 cache 计费字段
3 participants
问题
HandleStreamFinalResponse在流式未正常结束（!claudeInfo.Done或CompletionTokens == 0）时，调用ResponseText2Usage整份覆盖claudeInfo.Usage：ResponseText2Usage返回一个全新的空dto.Usage，只填了 3 个字段。赋值后，message_start阶段已获取的CachedTokens、CacheCreationInputTokens、ClaudeCacheCreation5mTokens等字段全部归零。同时，
PromptTokens使用了claudeInfo.Usage.PromptTokens（message_start 的占位值，常为 1），而其他所有渠道（OpenAI、Gemini、XAI、Cohere、Palm……）在类似 fallback 场景下都使用info.GetEstimatePromptTokens()。影响
prompt_tokens=1修复
不整份覆盖 usage，只补缺失字段：
改动点：
claudeInfo.Usage，只补CompletionTokens和PromptTokens的缺失值CachedTokens、CacheCreationInputTokens等已从message_start获取的字段PromptTokens兜底改用info.GetEstimatePromptTokens()（与其他渠道对齐）Fixes #4129
Summary by CodeRabbit