feat: tiered pricing based on prompt-side tokens - #3157
Closed
zuoliangyu wants to merge 3 commits into
Closed
Conversation
Implement context-based tiered pricing (e.g., GPT-4.1 200K threshold, GPT-5 272K threshold). When prompt-side tokens exceed a threshold, the entire request is billed at the higher tier's ratios. Backend: - New tiered_ratio.go with TieredPricingTier struct and resolution logic - PostClaudeConsumeQuota resolves tier based on actual prompt-side tokens - ModelPriceHelper uses highest tier for conservative pre-consumption - TieredPricing option persisted via settings API - Pricing API exposes tiered_pricing per model Frontend: - Admin ratio settings page: new TieredPricing JSON editor - Model card view: green "tiered pricing" tag for applicable models - Model detail side sheet: tiered pricing panel showing per-tier prices Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When tiered pricing is triggered, the log content now shows: "分段计费已触发（prompt tokens: Xk，阈值: Yk）" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The parent component's initial state was missing the TieredPricing key, causing the child ModelRatioSettings to filter it out when loading options from the API. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
WalkthroughThe pull request introduces a comprehensive tiered pricing system that enables pricing ratios to vary based on prompt token thresholds. Changes span backend pricing logic, quota calculations, configuration management, and frontend UI components for setting and displaying tiered pricing tiers per model. Changes
Sequence Diagram(s)sequenceDiagram
actor Admin as Admin/User
participant Frontend as Frontend UI
participant Backend as Backend API
participant PricingEngine as Pricing Engine
participant QuotaService as Quota Service
Admin->>Frontend: Configure tiered pricing tiers (JSON)
Frontend->>Backend: POST tiered pricing config
Backend->>PricingEngine: UpdateTieredPricingByJSONString(jsonStr)
PricingEngine->>PricingEngine: Load tiers into tieredPricingMap
PricingEngine->>Backend: Cache invalidated
Note over Backend: Later: User makes API call
Backend->>PricingEngine: Get ModelPriceHelper for model
PricingEngine->>PricingEngine: GetHighestTier(model name)
alt Tiered pricing exists
PricingEngine->>PricingEngine: ResolveTieredPricing(name, promptTokens)
PricingEngine->>PricingEngine: Select highest tier where Threshold ≤ promptTokens
PricingEngine->>Backend: Return tier's ModelRatio/CompletionRatio/CacheRatio
Backend->>QuotaService: Apply tiered ratios to quota calculation
QuotaService->>QuotaService: Use tiered ratios instead of base ratios
else No tiered pricing
Backend->>QuotaService: Use standard pricing ratios
end
QuotaService->>Backend: Return calculated quota with tier info
Backend->>Frontend: Return PriceData with HasTieredPricing flag
Frontend->>Frontend: Display pricing with tiered breakdown
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~30 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Member
|
非常感谢你的PR！但是这个功能我们已经在内部开发推进中，考虑到核心模块的长期维护、架构一致性以及后续迭代节奏，当前阶段暂不接受核心功能相关的外部 PR，因此这次无法合并。 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (1)
types/price_data.go (1)
28-31: Surface the new tiered fields inToSetting().
PriceDatanow carries tiered-pricing state, butToSetting()still omits it, so the debug log inrelay/helper/price.gocannot show whether tiered pricing was attached or which override ratios were chosen.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@types/price_data.go` around lines 28 - 31, The ToSetting() method on PriceData needs to include the new tiered-pricing fields so logs can surface them: update PriceData.ToSetting() to copy HasTieredPricing, TieredModelRatio, TieredCompletionRatio, and TieredCacheRatio into the returned settings structure (or map) alongside the existing ModelRatio/CompletionRatio/CacheRatio values; ensure the keys/names used match what relay/helper/price.go expects for logging so the debug output will show whether tiered pricing is enabled and the override ratios selected.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@relay/helper/price.go`:
- Around line 93-105: The code uses highestTier.ModelRatio to compute
preConsumedQuota but later still checks "free model" using the base modelRatio,
causing tiered paid requests to be misclassified; fix by introducing an
effectiveRatio (e.g., effectiveRatio := if highestTier != nil then
highestTier.ModelRatio * groupRatioInfo.GroupRatio else modelRatio *
groupRatioInfo.GroupRatio), use effectiveRatio when computing preConsumedQuota
(with preConsumedTokens) and reuse that same effectiveRatio in the subsequent
free-model check instead of modelRatio so the tiered decision is consistent;
update any related flags (hasTieredPricing / tieredModelRatio) only as before
but base free-model logic on effectiveRatio.
In `@setting/ratio_setting/tiered_ratio.go`:
- Around line 45-46: UpdateTieredPricingByJSONString must normalize and validate
tiers before publishing: parse the incoming JSON into the same struct used by
tieredPricingMap, sort the tier slice by Threshold ascending, verify thresholds
are strictly increasing (no duplicates) and that required fields (e.g., Name/ID
and Threshold) are present; if validation fails return an error. After
normalization call types.LoadFromJsonStringWithCallback using the normalized
JSON/structure (or replace tieredPricingMap with the normalized value) so
ResolveTieredPricing and GetHighestTier always see a canonical,
strictly-ascending tier list; keep references to tieredPricingMap and
InvalidateExposedDataCache when wiring the call.
In
`@web/src/components/table/model-pricing/modal/components/ModelPricingTable.jsx`:
- Around line 219-227: The code currently picks a multiplier using the first
enumerable key from availableGroups (availableGroups[0]), which makes tier
prices dependent on object key iteration order; change this to select a
deterministic, explicit group instead: compute the intersection between
modelEnableGroups (preserving its order) and Object.keys(usableGroup) (filtered)
and use the first match from modelEnableGroups, or fall back to a clearly named
default group (e.g., 'default' or 'standard') before falling back to 1; update
the logic around usedGroupRatio, availableGroups, usableGroup,
modelEnableGroups, and groupRatio so the group choice is predictable and
documented in the UI/state rather than relying on object key order.
- Around line 199-217: The UI labels are off-by-one vs backend (backend applies
a tier when promptSideTokens >= threshold), so update the range formatting to
use inclusive upper bounds: for the base band use 0 - (firstThreshold - 1)
(guard with Math.max(0, firstThreshold - 1)), and in the tiers loop treat the
current tier as starting at tier.threshold and ending at nextThreshold - 1 (when
nextThreshold exists) or as `${formatTokenCount(tier.threshold)}+` when it's the
last tier; compute upperEnd = nextThreshold ? nextThreshold - 1 : null and build
labels with formatTokenCount(currentStart) and formatTokenCount(upperEnd)
accordingly, leaving modelRatio/completionRatio assignment
(baseModelRatio/baseCompletionRatio and tier.model_ratio/tier.completion_ratio)
unchanged.
- Around line 264-266: The caption "per {unitLabel} tokens" is hardcoded and
bypasses i18n; update ModelPricingTable.jsx to use the useTranslation() hook and
translate the unit caption by calling t('your.translation.key', { unit:
unitLabel }) (or similar) where the Text showing per {unitLabel} tokens is
rendered; import and invoke useTranslation() at the top of the ModelPricingTable
component and replace the literal string with t('...') referencing a new
translation key so the label renders through the i18n pipeline.
In `@web/src/pages/Setting/Ratio/ModelRatioSettings.jsx`:
- Around line 337-340: The validation error message for the JSON textarea is a
raw string; update the component to use i18n by calling useTranslation() and
replacing the literal message with a call to t('...') (e.g.,
t('modelRatio.invalidJson') or an existing key) in the rules array where
validator uses verifyJSON; ensure useTranslation is imported and invoked in the
ModelRatioSettings component so the message flows through t().
---
Nitpick comments:
In `@types/price_data.go`:
- Around line 28-31: The ToSetting() method on PriceData needs to include the
new tiered-pricing fields so logs can surface them: update PriceData.ToSetting()
to copy HasTieredPricing, TieredModelRatio, TieredCompletionRatio, and
TieredCacheRatio into the returned settings structure (or map) alongside the
existing ModelRatio/CompletionRatio/CacheRatio values; ensure the keys/names
used match what relay/helper/price.go expects for logging so the debug output
will show whether tiered pricing is enabled and the override ratios selected.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: dbc9a55c-f1a5-4cbd-80f1-19d0782802b7
📒 Files selected for processing (12)
model/option.gomodel/pricing.gorelay/helper/price.goservice/quota.gosetting/ratio_setting/exposed_cache.gosetting/ratio_setting/model_ratio.gosetting/ratio_setting/tiered_ratio.gotypes/price_data.goweb/src/components/settings/RatioSetting.jsxweb/src/components/table/model-pricing/modal/components/ModelPricingTable.jsxweb/src/components/table/model-pricing/view/card/PricingCardView.jsxweb/src/pages/Setting/Ratio/ModelRatioSettings.jsx
Comment on lines
+93
to
+105
| // Check tiered pricing: use highest tier for conservative pre-consumption | ||
| if highestTier := ratio_setting.GetHighestTier(info.OriginModelName); highestTier != nil { | ||
| hasTieredPricing = true | ||
| tieredModelRatio = highestTier.ModelRatio | ||
| tieredCompletionRatio = highestTier.CompletionRatio | ||
| tieredCacheRatio = highestTier.CacheRatio | ||
| // Use highest tier's model ratio for pre-consumption to be conservative | ||
| ratio := highestTier.ModelRatio * groupRatioInfo.GroupRatio | ||
| preConsumedQuota = int(float64(preConsumedTokens) * ratio) | ||
| } else { | ||
| ratio := modelRatio * groupRatioInfo.GroupRatio | ||
| preConsumedQuota = int(float64(preConsumedTokens) * ratio) | ||
| } |
Contributor
There was a problem hiding this comment.
Don't base free-model detection on the non-tiered ratio.
This block can pre-consume using highestTier.ModelRatio, but Line 125 still decides "free model" from the base modelRatio. If a model is free below the threshold and paid above it, paid tiered requests will have their pre-consumption zeroed back out.
[suggested fix: track the effective ratio used for pre-consumption and reuse that in the later free-model check.]
💡 Proposed fix
var freeModel bool
var hasTieredPricing bool
var tieredModelRatio, tieredCompletionRatio, tieredCacheRatio float64
+ var effectivePreconsumeModelRatio float64
if !usePrice {
preConsumedTokens := common.Max(promptTokens, common.PreConsumedQuota)
if meta.MaxTokens != 0 {
preConsumedTokens += meta.MaxTokens
@@
modelRatio, success, matchName = ratio_setting.GetModelRatio(info.OriginModelName)
+ effectivePreconsumeModelRatio = modelRatio
if !success {
@@
// Check tiered pricing: use highest tier for conservative pre-consumption
if highestTier := ratio_setting.GetHighestTier(info.OriginModelName); highestTier != nil {
hasTieredPricing = true
tieredModelRatio = highestTier.ModelRatio
tieredCompletionRatio = highestTier.CompletionRatio
tieredCacheRatio = highestTier.CacheRatio
+ effectivePreconsumeModelRatio = highestTier.ModelRatio
// Use highest tier's model ratio for pre-consumption to be conservative
- ratio := highestTier.ModelRatio * groupRatioInfo.GroupRatio
+ ratio := effectivePreconsumeModelRatio * groupRatioInfo.GroupRatio
preConsumedQuota = int(float64(preConsumedTokens) * ratio)
} else {
- ratio := modelRatio * groupRatioInfo.GroupRatio
+ ratio := effectivePreconsumeModelRatio * groupRatioInfo.GroupRatio
preConsumedQuota = int(float64(preConsumedTokens) * ratio)
}
} else {
@@
} else {
- if modelRatio == 0 {
+ if effectivePreconsumeModelRatio == 0 {
preConsumedQuota = 0
freeModel = true
}
}
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@relay/helper/price.go` around lines 93 - 105, The code uses
highestTier.ModelRatio to compute preConsumedQuota but later still checks "free
model" using the base modelRatio, causing tiered paid requests to be
misclassified; fix by introducing an effectiveRatio (e.g., effectiveRatio := if
highestTier != nil then highestTier.ModelRatio * groupRatioInfo.GroupRatio else
modelRatio * groupRatioInfo.GroupRatio), use effectiveRatio when computing
preConsumedQuota (with preConsumedTokens) and reuse that same effectiveRatio in
the subsequent free-model check instead of modelRatio so the tiered decision is
consistent; update any related flags (hasTieredPricing / tieredModelRatio) only
as before but base free-model logic on effectiveRatio.
Comment on lines
+45
to
+46
| func UpdateTieredPricingByJSONString(jsonStr string) error { | ||
| return types.LoadFromJsonStringWithCallback(tieredPricingMap, jsonStr, InvalidateExposedDataCache) |
Contributor
There was a problem hiding this comment.
Normalize and validate tiers before publishing them.
ResolveTieredPricing() and GetHighestTier() both rely on thresholds being strictly ascending, but this accepts arbitrary admin JSON as-is. A misordered or duplicate tier list will select the wrong active tier and can misprice both final settlement and conservative pre-consumption.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@setting/ratio_setting/tiered_ratio.go` around lines 45 - 46,
UpdateTieredPricingByJSONString must normalize and validate tiers before
publishing: parse the incoming JSON into the same struct used by
tieredPricingMap, sort the tier slice by Threshold ascending, verify thresholds
are strictly increasing (no duplicates) and that required fields (e.g., Name/ID
and Threshold) are present; if validation fails return an error. After
normalization call types.LoadFromJsonStringWithCallback using the normalized
JSON/structure (or replace tieredPricingMap with the normalized value) so
ResolveTieredPricing and GetHighestTier always see a canonical,
strictly-ascending tier list; keep references to tieredPricingMap and
InvalidateExposedDataCache when wiring the call.
Comment on lines
+199
to
+217
| // Determine the first tier's upper bound | ||
| const firstThreshold = tiers[0]?.threshold || 0; | ||
| allTiers.push({ | ||
| label: `0 - ${formatTokenCount(firstThreshold)}`, | ||
| modelRatio: baseModelRatio, | ||
| completionRatio: baseCompletionRatio, | ||
| }); | ||
|
|
||
| tiers.forEach((tier, idx) => { | ||
| const nextThreshold = idx < tiers.length - 1 ? tiers[idx + 1].threshold : null; | ||
| const label = nextThreshold | ||
| ? `${formatTokenCount(tier.threshold + 1)} - ${formatTokenCount(nextThreshold)}` | ||
| : `${formatTokenCount(tier.threshold + 1)}+`; | ||
| allTiers.push({ | ||
| label, | ||
| modelRatio: tier.model_ratio, | ||
| completionRatio: tier.completion_ratio, | ||
| }); | ||
| }); |
Contributor
There was a problem hiding this comment.
Align the displayed ranges with the backend's inclusive threshold rule.
The backend switches tiers on promptSideTokens >= threshold, but this card shows the base band as ending at threshold and the next band as starting at threshold + 1. Requests exactly on the threshold will therefore be billed in the higher tier while the UI says they still belong to the lower one.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@web/src/components/table/model-pricing/modal/components/ModelPricingTable.jsx`
around lines 199 - 217, The UI labels are off-by-one vs backend (backend applies
a tier when promptSideTokens >= threshold), so update the range formatting to
use inclusive upper bounds: for the base band use 0 - (firstThreshold - 1)
(guard with Math.max(0, firstThreshold - 1)), and in the tiers loop treat the
current tier as starting at tier.threshold and ending at nextThreshold - 1 (when
nextThreshold exists) or as `${formatTokenCount(tier.threshold)}+` when it's the
last tier; compute upperEnd = nextThreshold ? nextThreshold - 1 : null and build
labels with formatTokenCount(currentStart) and formatTokenCount(upperEnd)
accordingly, leaving modelRatio/completionRatio assignment
(baseModelRatio/baseCompletionRatio and tier.model_ratio/tier.completion_ratio)
unchanged.
Comment on lines
+219
to
+227
| // Use the first available group ratio | ||
| let usedGroupRatio = 1; | ||
| const availableGroups = Object.keys(usableGroup || {}) | ||
| .filter((g) => g !== '' && g !== 'auto') | ||
| .filter((g) => modelEnableGroups.includes(g)); | ||
| if (availableGroups.length > 0) { | ||
| const firstGroup = availableGroups[0]; | ||
| usedGroupRatio = groupRatio?.[firstGroup] ?? 1; | ||
| } |
Contributor
There was a problem hiding this comment.
Don't derive tier prices from the first enumerable group.
This card uses availableGroups[0] as its multiplier, so the numbers depend on object key order rather than an explicit group choice. That can show the wrong tier prices with no indication of which group the card is using.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@web/src/components/table/model-pricing/modal/components/ModelPricingTable.jsx`
around lines 219 - 227, The code currently picks a multiplier using the first
enumerable key from availableGroups (availableGroups[0]), which makes tier
prices dependent on object key iteration order; change this to select a
deterministic, explicit group instead: compute the intersection between
modelEnableGroups (preserving its order) and Object.keys(usableGroup) (filtered)
and use the first match from modelEnableGroups, or fall back to a clearly named
default group (e.g., 'default' or 'standard') before falling back to 1; update
the logic around usedGroupRatio, availableGroups, usableGroup,
modelEnableGroups, and groupRatio so the group choice is predictable and
documented in the UI/state rather than relying on object key order.
Comment on lines
+264
to
+266
| <div className='flex items-center justify-between mb-2'> | ||
| <Text strong className='text-sm'>{tier.label}</Text> | ||
| <Text type='tertiary' className='text-xs'>per {unitLabel} tokens</Text> |
Contributor
There was a problem hiding this comment.
Translate the new unit caption.
per {unitLabel} tokens is hardcoded English, so it skips the i18n pipeline unlike the surrounding labels. As per coding guidelines, "Use useTranslation() hook and call t('中文key') in components."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@web/src/components/table/model-pricing/modal/components/ModelPricingTable.jsx`
around lines 264 - 266, The caption "per {unitLabel} tokens" is hardcoded and
bypasses i18n; update ModelPricingTable.jsx to use the useTranslation() hook and
translate the unit caption by calling t('your.translation.key', { unit:
unitLabel }) (or similar) where the Text showing per {unitLabel} tokens is
rendered; import and invoke useTranslation() at the top of the ModelPricingTable
component and replace the literal string with t('...') referencing a new
translation key so the label renders through the i18n pipeline.
Comment on lines
+337
to
+340
| rules={[ | ||
| { | ||
| validator: (rule, value) => verifyJSON(value), | ||
| message: '不是合法的 JSON 字符串', |
Contributor
There was a problem hiding this comment.
Localize the new validation message.
The new textarea uses t() for its labels and helper copy, but the validation error on Line 340 is still a raw string, so it won't go through the i18n pipeline.
rules={[
{
validator: (rule, value) => verifyJSON(value),
- message: '不是合法的 JSON 字符串',
+ message: t('不是合法的 JSON 字符串'),
},
]}As per coding guidelines, web/src/**/*.{ts,tsx,js,jsx} must "Use useTranslation() hook and call t('中文key') in components."
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| rules={[ | |
| { | |
| validator: (rule, value) => verifyJSON(value), | |
| message: '不是合法的 JSON 字符串', | |
| rules={[ | |
| { | |
| validator: (rule, value) => verifyJSON(value), | |
| message: t('不是合法的 JSON 字符串'), |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@web/src/pages/Setting/Ratio/ModelRatioSettings.jsx` around lines 337 - 340,
The validation error message for the JSON textarea is a raw string; update the
component to use i18n by calling useTranslation() and replacing the literal
message with a call to t('...') (e.g., t('modelRatio.invalidJson') or an
existing key) in the rules array where validator uses verifyJSON; ensure
useTranslation is imported and invoked in the ModelRatioSettings component so
the message flows through t().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implement context-based tiered pricing (like OpenAI's GPT-4.1 / GPT-5 pricing tiers). When prompt-side tokens exceed a configurable threshold, the entire request is billed at the higher tier's ratios.
For example, GPT-5.4:
Changes
Backend (Go):
setting/ratio_setting/tiered_ratio.go— New file: TieredPricingTier struct, default configs for GPT-4.1 (200K) and GPT-5 (272K) families, tier resolution logictypes/price_data.go— PriceData gains HasTieredPricing, TieredModelRatio, TieredCompletionRatio, TieredCacheRatio fieldsrelay/helper/price.go— ModelPriceHelper uses highest tier for conservative pre-consumption estimationservice/quota.go— PostClaudeConsumeQuota resolves tier based on actual prompt-side tokens; adds log notice "分段计费已触发（prompt tokens: Xk，阈值: Yk）"model/option.go— TieredPricing option persistence via settings APImodel/pricing.go— Pricing struct exposes tiered_pricing per model to frontendsetting/ratio_setting/model_ratio.go— InitRatioSettings loads tiered pricing defaultssetting/ratio_setting/exposed_cache.go— Expose tiered_pricing in ratio data cacheFrontend (React):
Design
{"gpt-5.4": [{"threshold": 272000, "model_ratio": 2.5, "completion_ratio": 4.5, "cache_ratio": 0.1}]}Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
UI Updates