fix: openai 音频模型流模式未正确计费 - #2160
Conversation
WalkthroughAdds audio-model handling to OpenAI stream flow by introducing Changes
Sequence Diagram(s)sequenceDiagram
participant Upstream
participant Relay as relay-openai.go
participant Store as secondLastStreamData
participant Response
Note over Relay: isAudioModel = contains("audio")
Upstream->>Relay: Stream data (item N)
alt is Audio Model
Relay->>Store: Save previous data
end
Relay->>Response: Process current data
Upstream->>Relay: Stream complete
alt is Audio Model & secondLastStreamData exists
rect rgb(220, 240, 255)
Note over Relay: Extract usage from secondLastStreamData
Relay->>Relay: Unmarshal usage
Relay->>Response: Set usage & containStreamUsage
end
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes
Possibly related PRs
Poem
Pre-merge checks and finishing touches✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
relay/channel/openai/relay-openai.go (1)
127-128: Consider more robust audio model detection.The current substring matching could match models with "audio" in their name that don't exhibit this streaming behavior. While this works for known models like
gpt-4o-audio-preview, consider maintaining an explicit list of audio models or using a more specific pattern if this causes issues in the future.Example with explicit list:
- // 检查是否为音频模型 - isAudioModel := strings.Contains(strings.ToLower(model), "audio") + // 检查是否为音频模型 + audioModels := []string{"gpt-4o-audio-preview", "gpt-4o-realtime-preview"} + isAudioModel := false + modelLower := strings.ToLower(model) + for _, am := range audioModels { + if strings.Contains(modelLower, am) { + isAudioModel = true + break + } + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
relay/channel/openai/relay-openai.go(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-21T06:31:11.073Z
Learnt from: jiajunly
Repo: QuantumNous/new-api PR: 1629
File: relay/channel/openai/relay-openai.go:170-174
Timestamp: 2025-08-21T06:31:11.073Z
Learning: In relay/channel/openai/relay-openai.go, the streaming logic for the AddThinkFirst feature is designed so that only the first chunk of a stream gets the "<think>\n" prefix. The final flush in the streaming handler intentionally uses addThink=false because the last chunk should never receive the prefix, even in single-chunk streams where the prefix would have been applied during normal processing.
Applied to files:
relay/channel/openai/relay-openai.go
🧬 Code graph analysis (1)
relay/channel/openai/relay-openai.go (4)
dto/openai_response.go (1)
Usage(222-235)service/usage_helpr.go (1)
ValidUsage(28-30)common/constants.go (1)
DebugEnabled(71-71)logger/logger.go (1)
LogDebug(69-74)
🔇 Additional comments (2)
relay/channel/openai/relay-openai.go (2)
125-125: LGTM!The variable declaration is clear and the comment explains its purpose for audio model billing.
138-141: LGTM!The logic correctly captures the penultimate stream data by saving
lastStreamDatabefore it gets updated. This ensures that after streaming completes,secondLastStreamDatacontains the second-to-last event where usage information resides for audio models.
| // 对音频模型，从倒数第二个stream data中提取usage信息 | ||
| if isAudioModel && secondLastStreamData != "" { | ||
| var streamResp struct { | ||
| Usage *dto.Usage `json:"usage"` | ||
| } | ||
| err := json.Unmarshal([]byte(secondLastStreamData), &streamResp) | ||
| if err == nil && streamResp.Usage != nil && service.ValidUsage(streamResp.Usage) { | ||
| usage = streamResp.Usage | ||
| containStreamUsage = true | ||
|
|
||
| if common.DebugEnabled { | ||
| logger.LogDebug(c, fmt.Sprintf("Audio model usage extracted from second last SSE: PromptTokens=%d, CompletionTokens=%d, TotalTokens=%d, InputTokens=%d, OutputTokens=%d", | ||
| usage.PromptTokens, usage.CompletionTokens, usage.TotalTokens, | ||
| usage.InputTokens, usage.OutputTokens)) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
Verify that handleLastResponse doesn't overwrite audio model usage.
The usage extraction logic is correct and properly validates the usage before setting containStreamUsage = true. However, handleLastResponse is called after this block (line 169), which could potentially modify usage or reset containStreamUsage, thereby overwriting the audio model usage you just extracted.
Run the following script to examine handleLastResponse and verify it won't overwrite the audio model usage:
🏁 Script executed:
#!/bin/bash
# Description: Find and examine handleLastResponse function to verify it doesn't unconditionally overwrite usage or containStreamUsage
# Find the handleLastResponse function definition
ast-grep --pattern 'func handleLastResponse($$$) $$$'
# Also search for any assignments to containStreamUsage within handleLastResponse
rg -n -A 10 -B 2 'func handleLastResponse' Length of output: 2840
handleLastResponse will overwrite audio model usage extracted from secondLastStreamData if the last stream also contains valid usage.
The audio model usage extraction at lines 149-165 is undermined because handleLastResponse (called at line 169) unconditionally overwrites usage and containStreamUsage when lastStreamData contains valid usage. This causes the second-to-last SSE extraction to be discarded if the final SSE response has usage data, defeating the purpose of the audio model special handling.
To fix this, handleLastResponse should only update usage for audio models if containStreamUsage is not already set (to preserve the secondLastStreamData extraction), or explicitly document that final response usage takes precedence for all models.
🤖 Prompt for AI Agents
In relay/channel/openai/relay-openai.go around lines 149-165, the code extracts
usage from secondLastStreamData for audio models but handleLastResponse (invoked
at line 169) later unconditionally overwrites usage; update the logic so
handleLastResponse does not replace usage for audio models when
containStreamUsage is already true (i.e., if isAudioModel && containStreamUsage
then skip updating usage), or alternatively add a parameter/flag to
handleLastResponse indicating stream-derived usage should be preserved and honor
that flag to avoid overwriting the extraction from secondLastStreamData.
2 participants
fix #2108, fix #2110
使用 gpt-4o-audio-preview 测试发现 openai 音频模型流模式下包含有效 usage 的响应为倒数第二个
Summary by CodeRabbit