Skip to content

fix(volcengine): support tiered Doubao pricing - #34902

Open
kunwl123456 wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
kunwl123456:fix/volcengine-doubao-tiered-pricing
Open

fix(volcengine): support tiered Doubao pricing#34902
kunwl123456 wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
kunwl123456:fix/volcengine-doubao-tiered-pricing

Conversation

@kunwl123456

@kunwl123456 kunwl123456 commented Jul 28, 2026

Copy link
Copy Markdown

Summary

  • add a dedicated Volcengine cost calculator that selects one request-wide tier from total prompt tokens
  • restore non-zero cost tracking for Doubao Seed 2.0 Lite/Pro
  • add Doubao Seed 1.8 pricing, including cache reads, input-length tiers, and its <=200 output-token discount
  • display tiered model prices as min-max ranges in the model dashboard instead of $0.00

Context

The current code contains tiered price data for the Seed 2.0 models, but the cost dispatcher has no Volcengine branch, so requests fall through to flat pricing and are tracked as zero. This reintroduces the routing intent from #30357 while using Volcengine's request-wide, input-length-selected tier semantics instead of graduated bracket math.

Testing

  • 22 passed — Volcengine calculator and model-cost-map unit tests on the litellm_internal_staging base
  • Ruff format and lint checks passed for new Python files
  • dashboard production build passed
  • Prettier checks passed for changed dashboard files
  • tiered dashboard transformation assertion passed

@CLAassistant

CLAassistant commented Jul 28, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@kunwl123456
kunwl123456 changed the base branch from main to litellm_internal_staging July 28, 2026 09:04
@kunwl123456 kunwl123456 reopened this Jul 28, 2026
@kunwl123456
kunwl123456 force-pushed the fix/volcengine-doubao-tiered-pricing branch from 4bfa7bc to a13b4cf Compare July 28, 2026 09:06
@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds request-wide tiered cost calculation for Volcengine models.

  • Routes Volcengine token costs through a dedicated provider calculator.
  • Adds Seed 1.8 pricing, cache-read accounting, and short-output rates while restoring Seed 2.0 tiered costs.
  • Displays token-pricing tiers as minimum–maximum ranges in the model dashboard.
  • Adds calculator, pricing-map, and dashboard transformation tests.

Confidence Score: 5/5

The PR appears safe to merge; no concrete changed-code failure was identified.

The new dispatcher preserves custom-pricing precedence, the calculator applies the repository’s established tier boundaries and flat-pricing fallback, and the dashboard transformation correctly formats the token-tier shapes introduced by this change.

Important Files Changed

Filename Overview
litellm/cost_calculator.py Adds the Volcengine provider dispatch after custom-pricing overrides and before generic pricing fallback.
litellm/llms/volcengine/cost_calculator.py Implements request-wide tier selection, cached-token accounting, short-output rates, and flat-pricing fallback without an identified defect.
model_prices_and_context_window.json Adds synchronized Seed 1.8 pricing metadata and retains the existing Seed 2.0 tier definitions.
litellm/model_prices_and_context_window_backup.json Mirrors the canonical pricing-map changes for the bundled backup map.
tests/test_litellm/llms/volcengine/test_cost_calculator.py Covers tier boundaries, request-wide rates, cached prompts, short outputs, and top-level dispatch.
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/utils/modelDataTransformer.ts Derives formatted minimum–maximum token-price ranges from tier metadata while preserving flat-price behavior.
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/components/ModelsTableColumns.tsx Labels tiered ranges appropriately while retaining the existing currency rendering.
ui/litellm-dashboard/src/components/model_dashboard/types.ts Extends model cost fields to support formatted range strings and identifies tiered rows.

Reviews (1): Last reviewed commit: 4bfa7bc | Re-trigger Greptile

@kunwl123456
kunwl123456 force-pushed the fix/volcengine-doubao-tiered-pricing branch from a13b4cf to 04a6e02 Compare July 28, 2026 09:09
{
"input_cost_per_token": 1.1e-07,
"output_cost_per_token": 2.8e-07,
"output_cost_per_token_above_200_tokens": 1.1e-06,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: Higher output rate bypasses the pre-call budget bound

An authenticated user can request more than 200 output tokens while near their budget limit because _max_cost_for_cost_info() only considers output_cost_per_token and output_cost_per_reasoning_token. For this tier it reserves at 2.8e-07, while post-call accounting charges the entire output at 1.1e-06; concurrent requests can therefore incur substantially more provider spend than the atomic budget gate permits. Update the budget reservation calculation to include this conditional rate when estimating maximum output cost.

@veria-ai

veria-ai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

PR overview

This pull request adds support for tiered pricing for Volcengine Doubao models, including a higher conditional output-token rate.

One issue remains open in the pre-call budget calculation: it reserves output costs using the lower base rate even when the higher tier may apply. An authenticated user near their budget limit could use sufficiently large or concurrent requests to cause provider spending beyond the enforced reservation, though exploitation depends on those budget and concurrency conditions.

Open issues (1)

Fixed/addressed: 0 · PR risk: 4/10

Co-authored-by: Cursor <cursoragent@cursor.com>
@kunwl123456
kunwl123456 force-pushed the fix/volcengine-doubao-tiered-pricing branch from 04a6e02 to 4427f56 Compare July 28, 2026 09:17
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Co-authored-by: Cursor <cursoragent@cursor.com>
@codspeed-hq

codspeed-hq Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing kunwl123456:fix/volcengine-doubao-tiered-pricing (d53f574) with litellm_internal_staging (daf22ec)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants