feat: gpt-5.6 - #6049
Conversation
WalkthroughThe change adds prompt-cache request fields, cache-write token details, propagation across Claude and OpenAI response handlers, compaction forwarding, and default cache ratios for three GPT models. ChangesPrompt cache support
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant CompactionRequest
participant ResponsesHelper
participant ResponsesRequest
participant ResponseHandler
participant ChatUsage
CompactionRequest->>ResponsesHelper: Prompt cache fields
ResponsesHelper->>ResponsesRequest: Forward prompt cache fields
ResponseHandler->>ChatUsage: Map CacheWriteTokens to cache creation usage
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@relay/channel/claude/relay-claude.go`:
- Around line 619-621: Update the Claude handler’s token accounting to keep both
cache creation fields consistent: in the logic using
cacheCreationTokensForOpenAIUsage, assign cacheCreationTokens to
clone.PromptTokensDetails.CachedCreationTokens as well as CacheWriteTokens
before calculating totalInputTokens, matching the behavior of the other response
handlers.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d89df1df-0e3b-43c9-97ba-50c3069c0e5f
📒 Files selected for processing (9)
dto/openai_request.godto/openai_response.godto/openai_responses_compaction_request.gorelay/channel/claude/relay-claude.gorelay/channel/openai/relay_responses.gorelay/channel/openai/relay_responses_compact.gorelay/responses_handler.goservice/relayconvert/responses_to_chat.gosetting/ratio_setting/cache_ratio.go
| cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage) | ||
| clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens | ||
| totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
CachedCreationTokens not updated on the clone, inconsistent with other handlers.
cacheCreationTokens is computed as the max of splitCacheCreationTokens and usage.PromptTokensDetails.CachedCreationTokens. Line 620 sets clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens, but clone.PromptTokensDetails.CachedCreationTokens retains the original (possibly smaller) value from the shallow copy at line 613.
When splitCacheCreationTokens > 0 and the original CachedCreationTokens <= splitCacheCreationTokens, cacheCreationTokens equals splitCacheCreationTokens, but clone.CachedCreationTokens stays at the smaller original value. Downstream quota accounting (text_quota.go:209) reads CachedCreationTokens, not CacheWriteTokens, so cache creation tokens would be undercounted.
All other handlers (relay_responses.go:54, relay_responses.go:108, relay_responses_compact.go:40, responses_to_chat.go:151-152) explicitly set CachedCreationTokens from the same source as CacheWriteTokens. The Claude path should follow suit.
🔧 Proposed fix
cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage)
clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens
+ clone.PromptTokensDetails.CachedCreationTokens = cacheCreationTokens
totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage) | |
| clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens | |
| totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens | |
| cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage) | |
| clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens | |
| clone.PromptTokensDetails.CachedCreationTokens = cacheCreationTokens | |
| totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@relay/channel/claude/relay-claude.go` around lines 619 - 621, Update the
Claude handler’s token accounting to keep both cache creation fields consistent:
in the logic using cacheCreationTokensForOpenAIUsage, assign cacheCreationTokens
to clone.PromptTokensDetails.CachedCreationTokens as well as CacheWriteTokens
before calculating totalInputTokens, matching the behavior of the other response
handlers.
|
I reproduced the accounting gap on Before merge, could we add focused tests for:
I also agree with the existing review: the Claude conversion should keep |
2 participants
Important
📝 变更描述 / Description
(简述：做了什么？为什么这样改能生效？请基于你对代码逻辑的理解来写，避免粘贴未经整理的内容)
引入gpt-5.6 缓存创建
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
Bug fix，我已提交或关联对应 Issue，且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
(请在此粘贴截图、关键日志或测试报告，以证明变更生效)
Summary by CodeRabbit
New Features
Bug Fixes