feat: support gpt tts series model quota calculate - #2434
Conversation
… and streaming support
WalkthroughRefactors OpenAI audio handling by extracting TTS/STT handlers, changes AudioRequest token-type selection and stream detection, and switches post-response quota routing to use actual audio token presence; also adjusts a hardcoded model ratio. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Relay as Relay (gin)
participant OpenAI as OpenAI Upstream
participant Quota as Quota Service
participant UsageParser as Usage parsing/token estimation
Client->>Relay: Send audio TTS/STT request
Relay->>OpenAI: Forward request (streaming or non-stream)
alt streaming chunks
OpenAI-->>Relay: Stream chunks (may contain "usage" JSON chunks)
Relay->>UsageParser: Parse streaming usage updates
UsageParser-->>Relay: Updated usage (prompt/completion/audio tokens)
Relay->>Client: Forward audio chunks
else non-streaming response
OpenAI-->>Relay: Full response body
Relay->>Client: Write body to client
Relay->>UsageParser: Extract/estimate usage from body (duration, size, upstream usage)
end
Relay->>Quota: If usage.AudioTokens > 0 -> PostAudioConsumeQuota else -> postConsumeQuota
Quota-->>Relay: Quota response/ack
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used🧠 Learnings (2)📚 Learning: 2025-08-21T06:31:11.073ZApplied to files:
📚 Learning: 2025-06-21T03:37:41.726ZApplied to files:
🔇 Additional comments (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
dto/audio.go (1)
28-30: Consider case-insensitive model name matching.The check
strings.Contains(r.Model, "gpt")is case-sensitive. If a model name ever arrives in a different case (e.g., "GPT-4o-mini-tts"), the tokenizer type won't be applied correctly. Consider usingstrings.Contains(strings.ToLower(r.Model), "gpt")for robustness.- if strings.Contains(r.Model, "gpt") { + if strings.Contains(strings.ToLower(r.Model), "gpt") { meta.TokenType = types.TokenTypeTokenizer }relay/channel/openai/audio.go (1)
105-105: Simplify redundant rounding.
math.Round(math.Ceil(duration))is redundant sincemath.Ceilalready returns a whole number. Consider simplifying.- completionTokens := int(math.Round(math.Ceil(duration) / 60.0 * 1000)) + completionTokens := int(math.Ceil(duration / 60.0 * 1000))
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
dto/audio.go(2 hunks)relay/audio_handler.go(1 hunks)relay/channel/openai/audio.go(1 hunks)relay/channel/openai/relay-openai.go(2 hunks)relay/compatible_handler.go(1 hunks)setting/ratio_setting/model_ratio.go(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-06-21T03:37:41.726Z
Learnt from: 9Ninety
Repo: QuantumNous/new-api PR: 1273
File: relay/channel/gemini/relay-gemini.go:97-116
Timestamp: 2025-06-21T03:37:41.726Z
Learning: In relay/channel/gemini/relay-gemini.go, the thinking budget calculation logic (including the MaxOutputTokens multiplication) was introduced in PR #1247. PR #1273 focused specifically on decoupling the thoughts summary feature from thinking budget settings and did not modify the existing thinking budget behavior.
Applied to files:
relay/compatible_handler.go
📚 Learning: 2025-08-21T06:31:11.073Z
Learnt from: jiajunly
Repo: QuantumNous/new-api PR: 1629
File: relay/channel/openai/relay-openai.go:170-174
Timestamp: 2025-08-21T06:31:11.073Z
Learning: In relay/channel/openai/relay-openai.go, the streaming logic for the AddThinkFirst feature is designed so that only the first chunk of a stream gets the "<think>\n" prefix. The final flush in the streaming handler intentionally uses addThink=false because the last chunk should never receive the prefix, even in single-chunk streams where the prefix would have been applied during normal processing.
Applied to files:
relay/channel/openai/audio.go
📚 Learning: 2025-08-05T17:14:17.246Z
Learnt from: neotf
Repo: QuantumNous/new-api PR: 1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.
Applied to files:
setting/ratio_setting/model_ratio.go
🧬 Code graph analysis (4)
relay/audio_handler.go (2)
dto/openai_response.go (1)
Usage(222-240)service/quota.go (1)
PostAudioConsumeQuota(377-478)
relay/compatible_handler.go (1)
dto/openai_response.go (1)
Usage(222-240)
dto/audio.go (2)
model/model_meta.go (1)
Model(23-44)types/request_meta.go (2)
TokenType(12-12)TokenTypeTokenizer(16-16)
relay/channel/openai/relay-openai.go (1)
common/json.go (1)
Unmarshal(9-11)
🔇 Additional comments (7)
setting/ratio_setting/model_ratio.go (1)
534-540: LGTM - Allows completion ratio override for gpt-4o models.The change from
truetofalsefor the gpt-4o fallback allows theCompletionRatiomap to override the hardcoded value of 4, providing more flexibility for configuration. The special case forgpt-4o-2024-05-13(ratio 3) is preserved.dto/audio.go (1)
34-36: LGTM - Proper SSE stream detection.The
IsStreammethod now correctly returnstrueonly whenStreamFormat == "sse", aligning with the expected streaming behavior for audio requests.relay/compatible_handler.go (1)
184-188: LGTM - Data-driven audio quota detection is a good improvement.Replacing model-name based gating (
strings.HasPrefix(name, "gpt-4o-audio")) with actual audio token presence detection is more robust and will correctly handle any model that produces audio output, including new GPT TTS models.relay/channel/openai/relay-openai.go (2)
153-154: LGTM - Consistent use of common.Unmarshal.Using
common.Unmarshalinstead of directjson.Unmarshalaligns with the codebase pattern and allows for centralized JSON handling if needed in the future.
627-628: LGTM - Consistent unmarshalling pattern.Same consistency improvement as other locations in this file.
relay/channel/openai/audio.go (2)
80-91: LGTM - PCM duration calculation.The PCM format handling with hardcoded OpenAI TTS parameters (24000 Hz, 16-bit, mono) is correct. The fallback to
GetAudioDurationfor other formats is appropriate.
114-145: LGTM - STT handler with proper usage fallback.The STT handler correctly parses usage from the response and falls back to estimated tokens when not available. The normalization of
InputTokens/OutputTokenstoPromptTokens/CompletionTokenshandles different API response formats.
| if usage.(*dto.Usage).CompletionTokenDetails.AudioTokens > 0 || usage.(*dto.Usage).PromptTokensDetails.AudioTokens > 0 { | ||
| service.PostAudioConsumeQuota(c, info, usage.(*dto.Usage), "") | ||
| } else { | ||
| postConsumeQuota(c, info, usage.(*dto.Usage), "") | ||
| } |
There was a problem hiding this comment.
Add nil check before accessing usage fields to prevent potential panic.
If adaptor.DoResponse returns nil for usage without an error (edge case), the type assertion and field access will cause a panic. Consider adding a nil check.
+ if usage == nil {
+ usage = &dto.Usage{
+ PromptTokens: info.GetEstimatePromptTokens(),
+ TotalTokens: info.GetEstimatePromptTokens(),
+ }
+ }
if usage.(*dto.Usage).CompletionTokenDetails.AudioTokens > 0 || usage.(*dto.Usage).PromptTokensDetails.AudioTokens > 0 {
service.PostAudioConsumeQuota(c, info, usage.(*dto.Usage), "")
} else {
postConsumeQuota(c, info, usage.(*dto.Usage), "")
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if usage.(*dto.Usage).CompletionTokenDetails.AudioTokens > 0 || usage.(*dto.Usage).PromptTokensDetails.AudioTokens > 0 { | |
| service.PostAudioConsumeQuota(c, info, usage.(*dto.Usage), "") | |
| } else { | |
| postConsumeQuota(c, info, usage.(*dto.Usage), "") | |
| } | |
| if usage == nil { | |
| usage = &dto.Usage{ | |
| PromptTokens: info.GetEstimatePromptTokens(), | |
| TotalTokens: info.GetEstimatePromptTokens(), | |
| } | |
| } | |
| if usage.(*dto.Usage).CompletionTokenDetails.AudioTokens > 0 || usage.(*dto.Usage).PromptTokensDetails.AudioTokens > 0 { | |
| service.PostAudioConsumeQuota(c, info, usage.(*dto.Usage), "") | |
| } else { | |
| postConsumeQuota(c, info, usage.(*dto.Usage), "") | |
| } |
🤖 Prompt for AI Agents
In relay/audio_handler.go around lines 70 to 74, the code assumes usage is
non-nil and does a type assertion and field access directly which can panic if
adaptor.DoResponse returns nil; add a nil check after obtaining usage (and
before the type assertion/cast) and handle the nil case (e.g., call
postConsumeQuota with a nil/empty usage or return/skip as appropriate), or
perform a safe type assertion that checks for nil before accessing
CompletionTokenDetails/PromptTokensDetails to avoid dereferencing a nil pointer.
…-tts feat: support gpt tts series model quota calculate
…-tts feat: support gpt tts series model quota calculate
1 participant
对于gpt-4o-mini-tts的价格，按以下设置倍率规则设置：
模型倍率 0.30，补全倍率 20.00，音频倍率 25.00，音频补全倍率 1.00
补全倍率计算规则：
输入价格为$0.6，输出价格为$12，$12/$0.6=20
其中流模式官方会返回token数量，因此无需使用到音频倍率，直接使用补全倍率计算
非流模式官方不返回token，因此使用本地计费，所以需要用到音频倍率
音频计算规则为：
Summary by CodeRabbit
New Features
Improvements
✏️ Tip: You can customize this high-level summary in your review settings.