Skip to content

feat: gpt-5.6 #6049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
seefs001 wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat: gpt-5.6 #6049

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions dto/openai_request.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -849,6 +849,7 @@ type OpenAIResponsesRequest struct {
MaxOutputTokens *uint `json:"max_output_tokens,omitempty"`
TopLogProbs *int `json:"top_logprobs,omitempty"`
Metadata json.RawMessage `json:"metadata,omitempty"`
Moderation json.RawMessage `json:"moderation,omitempty"`
ParallelToolCalls json.RawMessage `json:"parallel_tool_calls,omitempty"`
PreviousResponseID string `json:"previous_response_id,omitempty"`
Reasoning *Reasoning `json:"reasoning,omitempty"`
Expand All @@ -859,6 +860,7 @@ type OpenAIResponsesRequest struct {
// This field is allowed by default and can be disabled via channel setting disable_store.
Store json.RawMessage `json:"store,omitempty"`
PromptCacheKey json.RawMessage `json:"prompt_cache_key,omitempty"`
PromptCacheOptions json.RawMessage `json:"prompt_cache_options,omitempty"`
PromptCacheRetention json.RawMessage `json:"prompt_cache_retention,omitempty"`
// SafetyIdentifier carries client identity for policy abuse detection.
// This field is filtered by default and can be enabled via channel setting allow_safety_identifier.
Expand Down
1 change: 1 addition & 0 deletions dto/openai_response.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@ type OpenAIVideoResponse struct {

type InputTokenDetails struct {
CachedTokens int `json:"cached_tokens"`
CacheWriteTokens int `json:"cache_write_tokens,omitempty"`
CachedCreationTokens int `json:"cached_creation_tokens,omitempty"`
TextTokens int `json:"text_tokens"`
AudioTokens int `json:"audio_tokens"`
Expand Down
10 changes: 6 additions & 4 deletions dto/openai_responses_compaction_request.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ import (
)

type OpenAIResponsesCompactionRequest struct {
Model string `json:"model"`
Input json.RawMessage `json:"input,omitempty"`
Instructions json.RawMessage `json:"instructions,omitempty"`
PreviousResponseID string `json:"previous_response_id,omitempty"`
Model string `json:"model"`
Input json.RawMessage `json:"input,omitempty"`
Instructions json.RawMessage `json:"instructions,omitempty"`
PromptCacheOptions json.RawMessage `json:"prompt_cache_options,omitempty"`
PromptCacheRetention json.RawMessage `json:"prompt_cache_retention,omitempty"`
PreviousResponseID string `json:"previous_response_id,omitempty"`
}

func (r *OpenAIResponsesCompactionRequest) GetTokenCountMeta() *types.TokenCountMeta {
Expand Down
1 change: 1 addition & 0 deletions relay/channel/claude/relay-claude.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -617,6 +617,7 @@ func buildOpenAIStyleUsageFromClaudeUsage(usage *dto.Usage) dto.Usage {
usage.ClaudeCacheCreation1hTokens,
)
cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage)
clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens
totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens
Comment on lines 619 to 621

@coderabbitai coderabbitai Bot Jul 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

CachedCreationTokens not updated on the clone, inconsistent with other handlers.

cacheCreationTokens is computed as the max of splitCacheCreationTokens and usage.PromptTokensDetails.CachedCreationTokens. Line 620 sets clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens, but clone.PromptTokensDetails.CachedCreationTokens retains the original (possibly smaller) value from the shallow copy at line 613.

When splitCacheCreationTokens > 0 and the original CachedCreationTokens <= splitCacheCreationTokens, cacheCreationTokens equals splitCacheCreationTokens, but clone.CachedCreationTokens stays at the smaller original value. Downstream quota accounting (text_quota.go:209) reads CachedCreationTokens, not CacheWriteTokens, so cache creation tokens would be undercounted.

All other handlers (relay_responses.go:54, relay_responses.go:108, relay_responses_compact.go:40, responses_to_chat.go:151-152) explicitly set CachedCreationTokens from the same source as CacheWriteTokens. The Claude path should follow suit.

🔧 Proposed fix 
 	cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage)
 	clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens
+	clone.PromptTokensDetails.CachedCreationTokens = cacheCreationTokens
 	totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage)
clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens
totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens
cacheCreationTokens := cacheCreationTokensForOpenAIUsage(usage)
clone.PromptTokensDetails.CacheWriteTokens = cacheCreationTokens
clone.PromptTokensDetails.CachedCreationTokens = cacheCreationTokens
totalInputTokens := usage.PromptTokens + usage.PromptTokensDetails.CachedTokens + cacheCreationTokens
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/channel/claude/relay-claude.go` around lines 619 - 621, Update the
Claude handler’s token accounting to keep both cache creation fields consistent:
in the logic using cacheCreationTokensForOpenAIUsage, assign cacheCreationTokens
to clone.PromptTokensDetails.CachedCreationTokens as well as CacheWriteTokens
before calculating totalInputTokens, matching the behavior of the other response
handlers.

clone.PromptTokens = totalInputTokens
clone.InputTokens = totalInputTokens
Expand Down
2 changes: 2 additions & 0 deletions relay/channel/openai/relay_responses.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ func OaiResponsesHandler(c *gin.Context, info *relaycommon.RelayInfo, resp *http
usage.TotalTokens = responsesResponse.Usage.TotalTokens
if responsesResponse.Usage.InputTokensDetails != nil {
usage.PromptTokensDetails.CachedTokens = responsesResponse.Usage.InputTokensDetails.CachedTokens
usage.PromptTokensDetails.CachedCreationTokens = responsesResponse.Usage.InputTokensDetails.CacheWriteTokens
}
}
if info == nil || info.ResponsesUsageInfo == nil || info.ResponsesUsageInfo.BuiltInTools == nil {
Expand Down Expand Up @@ -104,6 +105,7 @@ func OaiResponsesStreamHandler(c *gin.Context, info *relaycommon.RelayInfo, resp
}
if streamResponse.Response.Usage.InputTokensDetails != nil {
usage.PromptTokensDetails.CachedTokens = streamResponse.Response.Usage.InputTokensDetails.CachedTokens
usage.PromptTokensDetails.CachedCreationTokens = streamResponse.Response.Usage.InputTokensDetails.CacheWriteTokens
}
}
if streamResponse.Response.HasImageGenerationCall() {
Expand Down
1 change: 1 addition & 0 deletions relay/channel/openai/relay_responses_compact.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ func OaiResponsesCompactionHandler(c *gin.Context, resp *http.Response) (*dto.Us
usage.TotalTokens = compactResp.Usage.TotalTokens
if compactResp.Usage.InputTokensDetails != nil {
usage.PromptTokensDetails.CachedTokens = compactResp.Usage.InputTokensDetails.CachedTokens
usage.PromptTokensDetails.CachedCreationTokens = compactResp.Usage.InputTokensDetails.CacheWriteTokens
}
}

Expand Down
10 changes: 6 additions & 4 deletions relay/responses_handler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,12 @@ func ResponsesHelper(c *gin.Context, info *relaycommon.RelayInfo) (newAPIError *
responsesReq = req
case *dto.OpenAIResponsesCompactionRequest:
responsesReq = &dto.OpenAIResponsesRequest{
Model: req.Model,
Input: req.Input,
Instructions: req.Instructions,
PreviousResponseID: req.PreviousResponseID,
Model: req.Model,
Input: req.Input,
Instructions: req.Instructions,
PromptCacheOptions: req.PromptCacheOptions,
PromptCacheRetention: req.PromptCacheRetention,
PreviousResponseID: req.PreviousResponseID,
}
default:
return types.NewErrorWithStatusCode(
Expand Down
2 changes: 2 additions & 0 deletions service/relayconvert/responses_to_chat.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,8 @@ func UsageFromResponsesUsage(src *dto.Usage) *dto.Usage {
}
if src.InputTokensDetails != nil {
usage.PromptTokensDetails.CachedTokens = src.InputTokensDetails.CachedTokens
usage.PromptTokensDetails.CacheWriteTokens = src.InputTokensDetails.CacheWriteTokens
usage.PromptTokensDetails.CachedCreationTokens = src.InputTokensDetails.CacheWriteTokens
usage.PromptTokensDetails.ImageTokens = src.InputTokensDetails.ImageTokens
usage.PromptTokensDetails.AudioTokens = src.InputTokensDetails.AudioTokens
}
Expand Down
3 changes: 3 additions & 0 deletions setting/ratio_setting/cache_ratio.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,9 @@ var defaultCacheRatio = map[string]float64{
}

var defaultCreateCacheRatio = map[string]float64{
"gpt-5.6-sol": 1.25,
"gpt-5.6-terra": 1.25,
"gpt-5.6-luna": 1.25,
"claude-3-sonnet-20240229": 1.25,
"claude-3-opus-20240229": 1.25,
"claude-3-haiku-20240307": 1.25,
Expand Down