feat: add tiered token pricing support - #4116
Closed
yyhhyyyyyy wants to merge 2 commits into
Closed
Conversation
Contributor
WalkthroughImplements tiered pricing for models, allowing step-based pricing calculations based on prompt token thresholds. Adds configuration, application logic, and UI controls for tier-based model pricing across backend services, API helpers, logging, and frontend settings. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~28 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
service/text_quota_test.go (1)
287-310: Consider test isolation for global tier pricing state.Both this test and
TestModelPriceHelperAllowsTierOnlyPricingmodify the globalmodelTierPricingMapviaUpdateModelTierPricingByJSONString. Whilet.Cleanupresets the state, if tests run in parallel (e.g., viat.Parallel()), they could interfere with each other.Currently these tests don't call
t.Parallel(), so they're safe. If parallelization is added later, consider using unique model names per test to avoid conflicts.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@service/text_quota_test.go` around lines 287 - 310, The test mutates global modelTierPricingMap via UpdateModelTierPricingByJSONString which can clash with TestModelPriceHelperAllowsTierOnlyPricing when tests run in parallel; to fix, make the JSON use a unique model key per test (e.g., include t.Name() or a random suffix) instead of the shared "google/gemini-3.1-pro-preview", and update assertions to reference that unique model name so each test modifies isolated tier pricing while still calling UpdateModelTierPricingByJSONString and t.Cleanup as before.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@service/quota.go`:
- Around line 140-146: The AudioRatio and AudioCompletionRatio reads currently
trust relayInfo.PriceData unconditionally; update them to use the same fallback
logic used for CompletionRatio (i.e., if
relayInfo.PriceData.AudioRatio/AudioCompletionRatio == 0 &&
relayInfo.PriceData.TierPricing == nil, use the existing fallback value used for
completion/model ratios) so calculateAudioQuota doesn't underbill when
ModelPriceHelper hasn't populated PriceData; apply the same change to the other
occurrences around the blocks that set AudioRatio/AudioCompletionRatio (the
similar sets at the later diff ranges).
In `@setting/ratio_setting/tier_pricing.go`:
- Around line 184-198: The ApplyModelTierPricing function should clamp negative
prompt token counts to zero so callers passing negative values still match the
first tier; at the start of ApplyModelTierPricing (before calling
GetModelTierPricing or iterating config.Tiers) normalize promptTokens = max(0,
promptTokens) (or otherwise set promptTokens to 0 when less than 0) to ensure
tier matching logic that checks tier.MinTokens and tier.MaxTokens behaves
correctly across callers like relay/helper/price.go and service/tier_pricing.go.
In `@web/src/pages/Setting/Ratio/hooks/useModelPricingEditorState.js`:
- Around line 764-766: The i18n call t('Legacy Flat Fields') in
useModelPricingEditorState (key: 'legacy-flat-fields') violates the repo
convention that translation keys must be Chinese source strings; replace the
English key with the corresponding Chinese source string (e.g. t('旧版平铺字段') or
the exact Chinese key from web/src/i18n/locales/{lang}.json) and do the same for
the other occurrences referenced around the same area (the other t(...) calls at
the noted nearby spots). Ensure useTranslation() is used in the component and
update the JSON locale files only if the Chinese source key is missing so
t('中文key') resolves correctly.
- Around line 716-741: The current logic divides dependent prices by inputPrice
and skips setting ratios when inputPrice === 0, which causes non-zero dependent
prices to be erased; update the block that computes CompletionRatio, CacheRatio,
CreateCacheRatio, ImageRatio, and AudioRatio so that if inputPrice === 0 you do
NOT overwrite existing model.rawRatios values and you avoid any division-by-zero
(i.e., when inputPrice === 0 and
completionPrice/cachePrice/createCachePrice/imagePrice/audioInputPrice are
non-null, preserve
model.rawRatios.CompletionRatio/CacheRatio/CreateCacheRatio/ImageRatio/AudioRatio
in result instead of setting them to null or attempting division); refer to
symbols result.CompletionRatio, result.CacheRatio, result.CreateCacheRatio,
result.ImageRatio, result.AudioRatio and model.rawRatios to implement the guard
and copy behavior.
---
Nitpick comments:
In `@service/text_quota_test.go`:
- Around line 287-310: The test mutates global modelTierPricingMap via
UpdateModelTierPricingByJSONString which can clash with
TestModelPriceHelperAllowsTierOnlyPricing when tests run in parallel; to fix,
make the JSON use a unique model key per test (e.g., include t.Name() or a
random suffix) instead of the shared "google/gemini-3.1-pro-preview", and update
assertions to reference that unique model name so each test modifies isolated
tier pricing while still calling UpdateModelTierPricingByJSONString and
t.Cleanup as before.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a56c691c-d7fa-4269-9821-359aa39db6dd
📒 Files selected for processing (24)
controller/channel-test.gocontroller/option.gocontroller/option_test.gomodel/option.gorelay/helper/price.gorelay/helper/price_test.goservice/log_info_generate.goservice/log_info_generate_test.goservice/quota.goservice/text_quota.goservice/text_quota_test.goservice/tier_pricing.gosetting/ratio_setting/exposed_cache.gosetting/ratio_setting/model_ratio.gosetting/ratio_setting/tier_pricing.gosetting/ratio_setting/tier_pricing_test.gotypes/price_data.gotypes/tier_pricing.goweb/src/components/settings/RatioSetting.jsxweb/src/components/table/usage-logs/UsageLogsColumnDefs.jsxweb/src/helpers/log.jsweb/src/hooks/usage-logs/useUsageLogsData.jsxweb/src/pages/Setting/Ratio/components/ModelPricingEditor.jsxweb/src/pages/Setting/Ratio/hooks/useModelPricingEditorState.js
Comment on lines
+140
to
+146
| ModelName: modelName, | ||
| UsePrice: relayInfo.PriceData.UsePrice, | ||
| ModelRatio: modelRatio, | ||
| GroupRatio: actualGroupRatio, | ||
| CompletionRatio: relayInfo.PriceData.CompletionRatio, | ||
| AudioRatio: relayInfo.PriceData.AudioRatio, | ||
| AudioCompletionRatio: relayInfo.PriceData.AudioCompletionRatio, |
Contributor
There was a problem hiding this comment.
Restore the audio-ratio fallback when PriceData is empty.
Unlike modelRatio and completionRatio, these reads now trust relayInfo.PriceData.AudioRatio / AudioCompletionRatio unconditionally. If a realtime/audio path reaches here before ModelPriceHelper has populated them, calculateAudioQuota treats audio tokens as 0x and underbills the request. Apply the same == 0 && TierPricing == nil fallback you already use for CompletionRatio.
💡 Representative fix
audioRatioValue := relayInfo.PriceData.AudioRatio
+if audioRatioValue == 0 && relayInfo.PriceData.TierPricing == nil {
+ audioRatioValue = ratio_setting.GetAudioRatio(modelName)
+}
audioCompletionRatioValue := relayInfo.PriceData.AudioCompletionRatio
+if audioCompletionRatioValue == 0 && relayInfo.PriceData.TierPricing == nil {
+ audioCompletionRatioValue = ratio_setting.GetAudioCompletionRatio(modelName)
+}Also applies to: 179-187, 290-298
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@service/quota.go` around lines 140 - 146, The AudioRatio and
AudioCompletionRatio reads currently trust relayInfo.PriceData unconditionally;
update them to use the same fallback logic used for CompletionRatio (i.e., if
relayInfo.PriceData.AudioRatio/AudioCompletionRatio == 0 &&
relayInfo.PriceData.TierPricing == nil, use the existing fallback value used for
completion/model ratios) so calculateAudioQuota doesn't underbill when
ModelPriceHelper hasn't populated PriceData; apply the same change to the other
occurrences around the blocks that set AudioRatio/AudioCompletionRatio (the
similar sets at the later diff ranges).
Comment on lines
+184
to
+198
| func ApplyModelTierPricing(modelName string, priceData types.PriceData, promptTokens int) (types.PriceData, bool) { | ||
| config, ok := GetModelTierPricing(modelName) | ||
| if !ok || !config.Enabled || config.Basis != TierPricingBasisPromptTokens { | ||
| return priceData, false | ||
| } | ||
|
|
||
| baseCacheRatio := resolveBaseCacheRatio(priceData) | ||
|
|
||
| for index, tier := range config.Tiers { | ||
| if promptTokens < tier.MinTokens { | ||
| continue | ||
| } | ||
| if tier.MaxTokens != nil && promptTokens >= *tier.MaxTokens { | ||
| continue | ||
| } |
Contributor
There was a problem hiding this comment.
Normalize negative prompt counts inside ApplyModelTierPricing.
promptTokens < 0 can never match the first min_tokens == 0 tier, so tier-only callers fall back to the incoming zero-value ratios. service/tier_pricing.go already guards this case before calling here; moving the clamp into the shared helper keeps every caller safe, including relay/helper/price.go.
💡 Proposed fix
func ApplyModelTierPricing(modelName string, priceData types.PriceData, promptTokens int) (types.PriceData, bool) {
+ if promptTokens < 0 {
+ promptTokens = 0
+ }
config, ok := GetModelTierPricing(modelName)
if !ok || !config.Enabled || config.Basis != TierPricingBasisPromptTokens {
return priceData, false
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| func ApplyModelTierPricing(modelName string, priceData types.PriceData, promptTokens int) (types.PriceData, bool) { | |
| config, ok := GetModelTierPricing(modelName) | |
| if !ok || !config.Enabled || config.Basis != TierPricingBasisPromptTokens { | |
| return priceData, false | |
| } | |
| baseCacheRatio := resolveBaseCacheRatio(priceData) | |
| for index, tier := range config.Tiers { | |
| if promptTokens < tier.MinTokens { | |
| continue | |
| } | |
| if tier.MaxTokens != nil && promptTokens >= *tier.MaxTokens { | |
| continue | |
| } | |
| func ApplyModelTierPricing(modelName string, priceData types.PriceData, promptTokens int) (types.PriceData, bool) { | |
| if promptTokens < 0 { | |
| promptTokens = 0 | |
| } | |
| config, ok := GetModelTierPricing(modelName) | |
| if !ok || !config.Enabled || config.Basis != TierPricingBasisPromptTokens { | |
| return priceData, false | |
| } | |
| baseCacheRatio := resolveBaseCacheRatio(priceData) | |
| for index, tier := range config.Tiers { | |
| if promptTokens < tier.MinTokens { | |
| continue | |
| } | |
| if tier.MaxTokens != nil && promptTokens >= *tier.MaxTokens { | |
| continue | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@setting/ratio_setting/tier_pricing.go` around lines 184 - 198, The
ApplyModelTierPricing function should clamp negative prompt token counts to zero
so callers passing negative values still match the first tier; at the start of
ApplyModelTierPricing (before calling GetModelTierPricing or iterating
config.Tiers) normalize promptTokens = max(0, promptTokens) (or otherwise set
promptTokens to 0 when less than 0) to ensure tier matching logic that checks
tier.MinTokens and tier.MaxTokens behaves correctly across callers like
relay/helper/price.go and service/tier_pricing.go.
Comment on lines
+716
to
741
| if (!model.tierPricingEnabled) { | ||
| result.ModelRatio = toNormalizedNumber(inputPrice / 2); | ||
|
|
||
| if (!model.completionRatioLocked && completionPrice !== null) { | ||
| result.CompletionRatio = toNormalizedNumber(completionPrice / inputPrice); | ||
| } else if ( | ||
| model.completionRatioLocked && | ||
| hasValue(model.rawRatios.completionRatio) | ||
| ) { | ||
| result.CompletionRatio = toNormalizedNumber( | ||
| model.rawRatios.completionRatio, | ||
| ); | ||
| } | ||
| if (cachePrice !== null) { | ||
| result.CacheRatio = toNormalizedNumber(cachePrice / inputPrice); | ||
| if (!model.completionRatioLocked && completionPrice !== null) { | ||
| result.CompletionRatio = toNormalizedNumber(completionPrice / inputPrice); | ||
| } else if ( | ||
| model.completionRatioLocked && | ||
| hasValue(model.rawRatios.completionRatio) | ||
| ) { | ||
| result.CompletionRatio = toNormalizedNumber( | ||
| model.rawRatios.completionRatio, | ||
| ); | ||
| } | ||
| if (cachePrice !== null) { | ||
| result.CacheRatio = toNormalizedNumber(cachePrice / inputPrice); | ||
| } | ||
| } | ||
| if (createCachePrice !== null) { | ||
| if (createCachePrice !== null && inputPrice !== 0) { | ||
| result.CreateCacheRatio = toNormalizedNumber(createCachePrice / inputPrice); | ||
| } | ||
| if (imagePrice !== null) { | ||
| if (imagePrice !== null && inputPrice !== 0) { | ||
| result.ImageRatio = toNormalizedNumber(imagePrice / inputPrice); | ||
| } | ||
| if (audioInputPrice !== null) { | ||
| if (audioInputPrice !== null && inputPrice !== 0) { | ||
| result.AudioRatio = toNormalizedNumber(audioInputPrice / inputPrice); | ||
| } |
Contributor
There was a problem hiding this comment.
Don't silently drop dependent prices when the reference input price is 0.
Here completionPrice / inputPrice collapses to null, and the inputPrice !== 0 guards skip CreateCacheRatio / ImageRatio / AudioRatio entirely. Saving a free-input model therefore erases non-zero dependent prices instead of rejecting the unsupported combination. That affects both flat pricing and tier pricing, because tier mode also reuses the first tier's input price as the reference denominator.
💡 Proposed fix
if (!model.tierPricingEnabled) {
+ if (
+ inputPrice === 0 &&
+ ((completionPrice !== null && completionPrice !== 0) ||
+ (cachePrice !== null && cachePrice !== 0))
+ ) {
+ throw new Error(
+ t('模型 {{name}} 的输入价格为 0 时，输出和缓存读取价格也必须为 0', {
+ name: model.name,
+ }),
+ );
+ }
result.ModelRatio = toNormalizedNumber(inputPrice / 2);
if (!model.completionRatioLocked && completionPrice !== null) {
result.CompletionRatio = toNormalizedNumber(completionPrice / inputPrice);
} else if (
@@
if (cachePrice !== null) {
result.CacheRatio = toNormalizedNumber(cachePrice / inputPrice);
}
}
+ if (
+ inputPrice === 0 &&
+ [createCachePrice, imagePrice, audioInputPrice].some(
+ (value) => value !== null && value !== 0,
+ )
+ ) {
+ throw new Error(
+ t('模型 {{name}} 的参考输入价格为 0 时，扩展价格也必须为 0', {
+ name: model.name,
+ }),
+ );
+ }
if (createCachePrice !== null && inputPrice !== 0) {
result.CreateCacheRatio = toNormalizedNumber(createCachePrice / inputPrice);
}Based on learnings, "OpenRouter API can return models with free input pricing (prompt="0") but non-zero completion pricing, requiring division-by-zero guards when computing completion_ratio and cache_ratio in controller/ratio_sync.go."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@web/src/pages/Setting/Ratio/hooks/useModelPricingEditorState.js` around lines
716 - 741, The current logic divides dependent prices by inputPrice and skips
setting ratios when inputPrice === 0, which causes non-zero dependent prices to
be erased; update the block that computes CompletionRatio, CacheRatio,
CreateCacheRatio, ImageRatio, and AudioRatio so that if inputPrice === 0 you do
NOT overwrite existing model.rawRatios values and you avoid any division-by-zero
(i.e., when inputPrice === 0 and
completionPrice/cachePrice/createCachePrice/imagePrice/audioInputPrice are
non-null, preserve
model.rawRatios.CompletionRatio/CacheRatio/CreateCacheRatio/ImageRatio/AudioRatio
in result instead of setting them to null or attempting division); refer to
symbols result.CompletionRatio, result.CacheRatio, result.CreateCacheRatio,
result.ImageRatio, result.AudioRatio and model.rawRatios to implement the guard
and copy behavior.
Comment on lines
+764
to
+766
| key: 'legacy-flat-fields', | ||
| title: t('Legacy Flat Fields'), | ||
| rows: [ |
Contributor
There was a problem hiding this comment.
Use a Chinese source key for this label.
t('Legacy Flat Fields') breaks the repo's i18n key convention and is unlikely to resolve from the existing locale JSON files.
As per coding guidelines, "web/src/**/*.{ts,tsx,js,jsx}: Translation files in web/src/i18n/locales/{lang}.json must be flat JSON with Chinese source strings as keys. Use useTranslation() hook and call t('中文key') in components."
Also applies to: 808-810, 862-864, 928-930
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@web/src/pages/Setting/Ratio/hooks/useModelPricingEditorState.js` around lines
764 - 766, The i18n call t('Legacy Flat Fields') in useModelPricingEditorState
(key: 'legacy-flat-fields') violates the repo convention that translation keys
must be Chinese source strings; replace the English key with the corresponding
Chinese source string (e.g. t('旧版平铺字段') or the exact Chinese key from
web/src/i18n/locales/{lang}.json) and do the same for the other occurrences
referenced around the same area (the other t(...) calls at the noted nearby
spots). Ensure useTranslation() is used in the component and update the JSON
locale files only if the Chinese source key is missing so t('中文key') resolves
correctly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 沟通提示 / Pre-submission
📝 变更描述 / Description
为模型价格配置新增了“阶梯定价”能力，把原来按量计费只能配置一组固定价格/倍率，扩展成可以按
prompt tokens区间配置多档价格。如vertex ai的定价方式。具体实现如下：ModelTierPricing，可以为模型配置多档输入价、输出价和缓存读取价prompt tokens命中对应档位，并把该档位换算成当前请求真正使用的价格/倍率和预扣额度🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
📸 运行证明 / Proof of Work
Summary by CodeRabbit
Release Notes
New Features
Tests