fix: claude affinity cache counter - #2980
Merged
Calcium-Ion merged 3 commits intoFeb 22, 2026
Merged
Conversation
Contributor
WalkthroughAdds relay-format–aware channel-affinity cache observation and token-rate mode propagation in backend, a small nil-check in quota consumption, frontend UI changes to display/interpret cached-token rates, and unit tests for RelayInfo final-format resolution and cache behavior. Changes
Sequence Diagram(s)sequenceDiagram
participant Browser
participant Backend
participant RelayInfo as RelayInfo/Adaptors
participant Cache
Browser->>Backend: request that consumes Claude quota (may include usage)
Backend->>RelayInfo: determine final relay format (GetFinalRequestRelayFormat)
alt usage != nil
Backend->>Cache: ObserveChannelAffinityUsageCacheByRelayFormat(usage, relayFormat)
Cache-->>Backend: ack/update counters (including cached_token_rate_mode)
end
Backend-->>Browser: quota consume response
Browser->>Backend: request channel-affinity stats for modal
Backend-->>Browser: stats (including cached_token_rate_mode, token counts)
Browser->>Browser: formatCachedTokenRate(mode, promptTokens, cachedTokens)
Browser-->>User: render modal with token rows and explanatory text
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
web/src/components/table/usage-logs/modals/ChannelAffinityUsageCacheModal.jsx (1)
47-48:cached > promptheuristic can produce false positives on aggregated non-Claude stats.For a rule that mixes Claude and non-Claude upstream requests over time, cumulative
cached_tokenscould exceedprompt_tokensfor purely numeric reasons unrelated to Claude's API token-reporting structure. This would silently flip the denominator toprompt + cached, changing the displayed percentage without user awareness.The comment on line 46 already acknowledges this as a known hack. Consider at minimum adding a
TODOnoting the intent to replace this with an explicit per-channel model-family tag once that metadata is available, so the heuristic is easier to find and remove.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@web/src/components/table/usage-logs/modals/ChannelAffinityUsageCacheModal.jsx` around lines 47 - 48, Add a TODO comment above the likelyClaudeStyle assignment explaining that the cached > prompt heuristic is a temporary hack for detecting Claude-style token reporting and should be replaced once per-channel model-family metadata exists; also annotate the line referencing normalizedRuleName, cached, and prompt so future maintainers can find and remove this heuristic easily (keep the expression using normalizedRuleName.includes('claude') || cached > prompt but add the TODO and a short explanatory comment referencing likelyClaudeStyle, normalizedRuleName, cached and prompt).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@web/src/components/table/usage-logs/modals/ChannelAffinityUsageCacheModal.jsx`:
- Around line 163-167: The displayed rule name passed into
formatCachedTokenRateAuto may be undefined because s.rule_name comes from the
API; update the call that builds the value (the value using cachedTokens,
promptTokens, and ruleName) to pass params.rule_name as a fallback: use
s.rule_name ?? params.rule_name (or similar) when calling
formatCachedTokenRateAuto so the normalizedRuleName logic in
formatCachedTokenRateAuto sees the expected rule name; adjust the same call that
builds the value string for cachedTokens to use this fallback.
---
Nitpick comments:
In
`@web/src/components/table/usage-logs/modals/ChannelAffinityUsageCacheModal.jsx`:
- Around line 47-48: Add a TODO comment above the likelyClaudeStyle assignment
explaining that the cached > prompt heuristic is a temporary hack for detecting
Claude-style token reporting and should be replaced once per-channel
model-family metadata exists; also annotate the line referencing
normalizedRuleName, cached, and prompt so future maintainers can find and remove
this heuristic easily (keep the expression using
normalizedRuleName.includes('claude') || cached > prompt but add the TODO and a
short explanatory comment referencing likelyClaudeStyle, normalizedRuleName,
cached and prompt).
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
service/channel_affinity.go (1)
724-733: Consider whether other relay formats (e.g., Gemini with cached tokens) should map to a mode.Currently Gemini returns
"", which means token stats are hidden in the UI even thoughusageCacheSignalswill still count cache hits for Gemini. If Gemini's cache semantics follow the OpenAI convention (cached tokens included in prompt tokens), it could map tocached_over_prompt.This is just a consideration for future — the current behavior is safe (defaults to hidden).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@service/channel_affinity.go` around lines 724 - 733, Update the mapping in cachedTokenRateModeByRelayFormat to include Gemini-style relay formats: add a case for the Gemini relay format constant (e.g., types.RelayFormatGemini) and return cacheTokenRateModeCachedOverPrompt so Gemini will surface cached token stats in the UI when its cache semantics match OpenAI's; leave the default branch untouched so unknown formats still return "".
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@service/quota.go`:
- Around line 238-244: PostClaudeConsumeQuota currently checks usage only to
call ObserveChannelAffinityUsageCacheByRelayFormat but then unconditionally
accesses usage.PromptTokens (and other fields) causing a nil-pointer panic;
modify the function so that if usage == nil it returns early (e.g., change the
initial guard to "if usage == nil { return }" before any further use) or
otherwise wrap all subsequent logic (useTimeSeconds, promptTokens, etc.) in a
usage != nil branch; reference PostClaudeConsumeQuota,
ObserveChannelAffinityUsageCacheByRelayFormat, and usage.PromptTokens when
making the change.
---
Nitpick comments:
In `@service/channel_affinity.go`:
- Around line 724-733: Update the mapping in cachedTokenRateModeByRelayFormat to
include Gemini-style relay formats: add a case for the Gemini relay format
constant (e.g., types.RelayFormatGemini) and return
cacheTokenRateModeCachedOverPrompt so Gemini will surface cached token stats in
the UI when its cache semantics match OpenAI's; leave the default branch
untouched so unknown formats still return "".
Comment on lines
238
to
244
| func PostClaudeConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage *dto.Usage) { | ||
| if usage != nil { | ||
| ObserveChannelAffinityUsageCacheByRelayFormat(ctx, usage, relayInfo.GetFinalRequestRelayFormat()) | ||
| } | ||
|
|
||
| useTimeSeconds := time.Now().Unix() - relayInfo.StartTime.Unix() | ||
| promptTokens := usage.PromptTokens |
Contributor
There was a problem hiding this comment.
Nil-check for usage doesn't prevent nil-pointer dereference below.
The guard at line 239 only protects ObserveChannelAffinityUsageCacheByRelayFormat, but execution continues to line 244 where usage.PromptTokens is accessed unconditionally. If usage is nil, this will panic.
Either return early when usage == nil (consistent with how the rest of the function uses usage), or wrap the entire function body after the observe call.
🐛 Proposed fix
func PostClaudeConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage *dto.Usage) {
+ if usage == nil {
+ return
+ }
+ ObserveChannelAffinityUsageCacheByRelayFormat(ctx, usage, relayInfo.GetFinalRequestRelayFormat())
- if usage != nil {
- ObserveChannelAffinityUsageCacheByRelayFormat(ctx, usage, relayInfo.GetFinalRequestRelayFormat())
- }
useTimeSeconds := time.Now().Unix() - relayInfo.StartTime.Unix()
promptTokens := usage.PromptTokens📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| func PostClaudeConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage *dto.Usage) { | |
| if usage != nil { | |
| ObserveChannelAffinityUsageCacheByRelayFormat(ctx, usage, relayInfo.GetFinalRequestRelayFormat()) | |
| } | |
| useTimeSeconds := time.Now().Unix() - relayInfo.StartTime.Unix() | |
| promptTokens := usage.PromptTokens | |
| func PostClaudeConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage *dto.Usage) { | |
| if usage == nil { | |
| return | |
| } | |
| ObserveChannelAffinityUsageCacheByRelayFormat(ctx, usage, relayInfo.GetFinalRequestRelayFormat()) | |
| useTimeSeconds := time.Now().Unix() - relayInfo.StartTime.Unix() | |
| promptTokens := usage.PromptTokens |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@service/quota.go` around lines 238 - 244, PostClaudeConsumeQuota currently
checks usage only to call ObserveChannelAffinityUsageCacheByRelayFormat but then
unconditionally accesses usage.PromptTokens (and other fields) causing a
nil-pointer panic; modify the function so that if usage == nil it returns early
(e.g., change the initial guard to "if usage == nil { return }" before any
further use) or otherwise wrap all subsequent logic (useTimeSeconds,
promptTokens, etc.) in a usage != nil branch; reference PostClaudeConsumeQuota,
ObserveChannelAffinityUsageCacheByRelayFormat, and usage.PromptTokens when
making the change.
dreamlx
pushed a commit
to dreamlx/new-api
that referenced
this pull request
Mar 3, 2026
* fix: claude affinity cache counter * fix: claude affinity cache counter * fix: stabilize cache usage stats format and simplify modal rendering
ennnnny
pushed a commit
to ennnnny/new-api
that referenced
this pull request
Mar 17, 2026
* fix: claude affinity cache counter * fix: claude affinity cache counter * fix: stabilize cache usage stats format and simplify modal rendering
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
New Features
Bug Fixes