Skip to content

fix: openrouter claude cache usage #2811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Calcium-Ion merged 2 commits into from
Feb 2, 2026
Merged

fix: openrouter claude cache usage #2811

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
72139df
fix: openrouter claude cache usage
seefs001 Feb 2, 2026
b22849a
fix(claude): preserve cached usage in stream message_delta
seefs001 Feb 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion relay/channel/claude/relay-claude.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -631,7 +631,21 @@ func FormatClaudeResponseInfo(requestMode int, claudeResponse *dto.ClaudeRespons
// 不叠加，只取最新的
claudeInfo.Usage.PromptTokens = claudeResponse.Usage.InputTokens
}
claudeInfo.Usage.CompletionTokens = claudeResponse.Usage.OutputTokens
if claudeResponse.Usage.CacheReadInputTokens > 0 {
claudeInfo.Usage.PromptTokensDetails.CachedTokens = claudeResponse.Usage.CacheReadInputTokens
}
if claudeResponse.Usage.CacheCreationInputTokens > 0 {
claudeInfo.Usage.PromptTokensDetails.CachedCreationTokens = claudeResponse.Usage.CacheCreationInputTokens
}
if cacheCreation5m := claudeResponse.Usage.GetCacheCreation5mTokens(); cacheCreation5m > 0 {
claudeInfo.Usage.ClaudeCacheCreation5mTokens = cacheCreation5m
}
if cacheCreation1h := claudeResponse.Usage.GetCacheCreation1hTokens(); cacheCreation1h > 0 {
claudeInfo.Usage.ClaudeCacheCreation1hTokens = cacheCreation1h
}
if claudeResponse.Usage.OutputTokens > 0 {
claudeInfo.Usage.CompletionTokens = claudeResponse.Usage.OutputTokens
}
Comment on lines 633 to +648

@coderabbitai coderabbitai Bot Feb 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# Search for Anthropic/Claude API documentation or comments about streaming behavior
rg -i "message_delta|cache.*token|streaming" relay/channel/claude/relay-claude.go -B 3 -A 3 | head -100

Repository: QuantumNous/new-api

Length of output: 3206

🏁 Script executed:

# Check if there's any documentation or constants related to how tokens are handled
rg -i "usage|cache.*token" relay/channel/claude/ --type go | head -50

Repository: QuantumNous/new-api

Length of output: 5591

🏁 Script executed:

# Look at the function context to understand how values are initialized and accumulated
sed -n '600,700p' relay/channel/claude/relay-claude.go

Repository: QuantumNous/new-api

Length of output: 4368

🌐 Web query:

Anthropic Claude API message_delta usage fields streaming: are zero values omitted or explicitly sent?

💡 Result:

In Anthropic streaming, the message_delta.usage object is sparse: it may include only the usage fields that were added/updated in that delta (often just output_tokens), so you cannot rely on “zero” fields being explicitly sent—they may be omitted entirely. This is reflected in Anthropic’s own streaming examples where message_delta.usage contains only {"output_tokens": ...}. [1]

When a field is included, it can legitimately be 0 (e.g., server responses have been observed sending cache_creation_input_tokens: 0). [2]

Also, depending on the client/SDK/proxy, you may see fields present but set to null (an SDK serialization behavior), so robust code should treat missing or null as “not provided” rather than assuming explicit zeros. [3]

Practical takeaway: treat message_delta.usage as a partial patch; maintain your own cumulative usage state and update only the fields that appear. [1]

Sources: [1] Anthropic streaming docs example shows message_delta with usage: {"output_tokens": ...} only. [2] Captured server streaming message_delta shows explicit cache_creation_input_tokens: 0. [3] SDK/proxy issue report shows message_delta.usage fields sometimes emitted as null.

🌐 Web query:

Anthropic Claude streaming API documentation message_delta cache_read_input_tokens cache_creation_input_tokens behavior

💡 Result:

In Claude’s streaming Messages API, you’ll see one or more message_delta SSE events near the end of the stream. These events apply top-level updates to the in-progress Message object (not per-text chunks—that’s content_block_delta). The usage object on message_delta is cumulative across the whole request/stream, so you should treat the latest message_delta.usage you receive as the final totals. [2]

Within that cumulative usage:

  • cache_creation_input_tokens: how many input tokens were written into the prompt cache (i.e., used to create/extend a cache entry) for this request. These are counted as part of total input usage. [1]
  • cache_read_input_tokens: how many input tokens were served from an existing prompt cache hit (i.e., read from cache) for this request. Also counted as part of total input usage. [1]
  • Total input tokens for billing/rate limits are:
    input_tokens + cache_creation_input_tokens + cache_read_input_tokens. [1]

So, behaviorally: if caching is enabled and you get a cache hit, you’ll see non-zero cache_read_input_tokens; if your request creates a new cache entry, you’ll see non-zero cache_creation_input_tokens—and both show up in the cumulative message_delta.usage during streaming. [1][2]

Consider using pointer fields or custom JSON unmarshaling to distinguish omitted from explicit zero values in message_delta.usage.

The > 0 checks assume omitted fields are unmarshaled as zero, which aligns with Anthropic's sparse message_delta.usage API (fields not sent are treated as "not provided"). However, Anthropic can legitimately send fields with value 0 (e.g., cache_read_input_tokens: 0), and the current code will skip those updates, preserving stale values from message_start instead of accepting the authoritative cumulative values. While unlikely to manifest in practice for cache fields (rarely present in message_delta), this is a subtle defect. To be fully correct, consider using *int pointers or custom JSON unmarshaling so explicit zeros overwrite prior values while truly omitted fields don't.

🤖 Prompt for AI Agents 
In `@relay/channel/claude/relay-claude.go` around lines 633 - 648, The code
currently uses >0 checks on claudeResponse.Usage fields (e.g.,
CacheReadInputTokens, CacheCreationInputTokens,
GetCacheCreation5mTokens/GetCacheCreation1hTokens, OutputTokens) which treats
omitted JSON fields and explicit zero values the same and can leave stale values
in claudeInfo.Usage; change the usage struct to use pointer types (e.g., *int)
or implement custom JSON unmarshaling for the Usage type so you can detect nil
(omitted) vs. explicitly zero, then update the assignment logic in the block
referencing claudeResponse.Usage and claudeInfo.Usage to check for nil (or
presence from Unmarshal) and assign the value (including zero) to
claudeInfo.Usage fields so explicit zeros overwrite prior values.

claudeInfo.Usage.TotalTokens = claudeInfo.Usage.PromptTokens + claudeInfo.Usage.CompletionTokens
}

Expand Down