fix: 补全 streaming message_delta 事件缺失的 input_tokens 和 cache 相关字段 - #2887
Conversation
当上游为 AWS Bedrock 时，message_delta 的 usage 可能缺少 input_tokens、 cache_creation_input_tokens、cache_read_input_tokens 等字段，导致与原生 Anthropic 格式不一致。从 message_start 积累的 claudeInfo 中补全这些字段后 重新序列化，确保客户端收到一致的 usage 格式。
WalkthroughThis pull request adds support for patching incomplete Claude message_delta usage data in relay responses. It introduces utility functions to derive missing usage fields from cached information and conditionally apply them to JSON streams, with comprehensive test coverage for the new patching logic. Changes
Sequence DiagramsequenceDiagram
participant Stream as Upstream Stream
participant Handler as Message Delta Handler
participant Skipper as shouldSkipClaudeMessageDeltaUsagePatch
participant Builder as buildMessageDeltaPatchUsage
participant Patcher as patchClaudeMessageDeltaUsageData
participant Output as Downstream
Stream->>Handler: message_delta event
Handler->>Skipper: Check PassThroughBodyEnabled
Skipper-->>Handler: skip patching?
alt Skip Patching
Handler->>Output: Send original data
else Apply Patching
Handler->>Builder: Derive missing fields from claudeInfo
Builder-->>Handler: Complete usage data
Handler->>Patcher: Patch JSON with derived usage
Patcher->>Patcher: setMessageDeltaUsageInt for each field
Patcher-->>Handler: Mutated JSON
Handler->>Output: Send patched data
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@relay/channel/claude/relay-claude.go`:
- Around line 577-585: The function shouldSkipClaudeMessageDeltaUsagePatch
dereferences info.ChannelSetting without checking the embedded pointer
info.ChannelMeta; update the guard to also verify info.ChannelMeta is non-nil
before accessing ChannelSetting (or read ChannelSetting via a nil-safe
accessor), i.e. add a nil-check for info.ChannelMeta in
shouldSkipClaudeMessageDeltaUsagePatch so it returns false (or the appropriate
default) when ChannelMeta is nil rather than panicking.
🧹 Nitpick comments (2)
relay/channel/claude/relay_claude_test.go (1)
1-175: Good test coverage forFormatClaudeResponseInfoacross key scenarios.The tests are well-structured and cover the important cases (message_start propagation, full vs. partial message_delta usage, nil guard, and text accumulation). A couple of observations:
- Inconsistent assertion style: This file uses raw
t.Errorf/t.Fatalwhile the sibling test file (message_delta_usage_patch_test.go) usestestify/require/assert. Consider aligning for consistency.- Missing coverage for
oaiResponsepath: All tests passnilfor theoaiResponseparameter. The branch at lines 683–687 of the implementation (oaiResponse.Id,.Created,.Modelpropagation) is untested.relay/channel/claude/message_delta_usage_patch_test.go (1)
48-64: Consider adding a test case fornil ChannelMetawhen global pass-through isfalse.Currently,
&relaycommon.RelayInfo{}(with nilChannelMeta) is only tested whenPassThroughRequestEnabledistrue, which hits the early return. If the implementation is updated to guard against nilChannelMeta(as suggested in the relay-claude.go review), a test confirming the safe fallback would be valuable.
| func shouldSkipClaudeMessageDeltaUsagePatch(info *relaycommon.RelayInfo) bool { | ||
| if model_setting.GetGlobalSettings().PassThroughRequestEnabled { | ||
| return true | ||
| } | ||
| if info == nil { | ||
| return false | ||
| } | ||
| return info.ChannelSetting.PassThroughBodyEnabled | ||
| } |
There was a problem hiding this comment.
Nil ChannelMeta will cause a panic.
If info is non-nil but info.ChannelMeta is nil (embedded pointer), accessing info.ChannelSetting on line 584 dereferences a nil pointer. The nil-check on line 581 only guards info itself.
🐛 Proposed fix
func shouldSkipClaudeMessageDeltaUsagePatch(info *relaycommon.RelayInfo) bool {
if model_setting.GetGlobalSettings().PassThroughRequestEnabled {
return true
}
- if info == nil {
+ if info == nil || info.ChannelMeta == nil {
return false
}
return info.ChannelSetting.PassThroughBodyEnabled
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| func shouldSkipClaudeMessageDeltaUsagePatch(info *relaycommon.RelayInfo) bool { | |
| if model_setting.GetGlobalSettings().PassThroughRequestEnabled { | |
| return true | |
| } | |
| if info == nil { | |
| return false | |
| } | |
| return info.ChannelSetting.PassThroughBodyEnabled | |
| } | |
| func shouldSkipClaudeMessageDeltaUsagePatch(info *relaycommon.RelayInfo) bool { | |
| if model_setting.GetGlobalSettings().PassThroughRequestEnabled { | |
| return true | |
| } | |
| if info == nil || info.ChannelMeta == nil { | |
| return false | |
| } | |
| return info.ChannelSetting.PassThroughBodyEnabled | |
| } |
🤖 Prompt for AI Agents
In `@relay/channel/claude/relay-claude.go` around lines 577 - 585, The function
shouldSkipClaudeMessageDeltaUsagePatch dereferences info.ChannelSetting without
checking the embedded pointer info.ChannelMeta; update the guard to also verify
info.ChannelMeta is non-nil before accessing ChannelSetting (or read
ChannelSetting via a nil-safe accessor), i.e. add a nil-check for
info.ChannelMeta in shouldSkipClaudeMessageDeltaUsagePatch so it returns false
(or the appropriate default) when ChannelMeta is nil rather than panicking.
fix: 补全 streaming message_delta 事件缺失的 input_tokens 和 cache 相关字段
3 participants
Summary by CodeRabbit
Release Notes
New Features
Tests