feat(model_prices): add missing Databricks Foundation Model API models and refresh Gemini 2.5 rates - #39714
Conversation
Greptile SummaryThe PR expands both model-price registries with Databricks Foundation Model metadata, refreshes Gemini 2.5 pricing, and adds prompt-cache pricing and capability fields. The two registry copies are synchronized, but the Databricks cost path does not consume the newly supplied cache rates.
Confidence Score: 4/5The PR should not merge until Databricks cached-token usage is charged using the cache-read and cache-creation rates added by this change. Databricks requests bypass the generic cache-aware calculator, causing cached prompt tokens to remain priced at the full input rate in recorded spend and budget accounting. Files Needing Attention: model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds and updates Databricks pricing metadata, but its cache-rate fields are ineffective in the provider-specific cost path. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the root pricing registry, including the same ineffective Databricks cache-rate metadata. |
Reviews (1): Last reviewed commit: "feat(model_prices): add missing Databric..." | Re-trigger Greptile
| "cache_creation_input_token_cost": 1.250004e-05, | ||
| "cache_read_input_token_cost": 1.00002e-06, | ||
| "input_cost_per_token": 1.000006e-05, |
There was a problem hiding this comment.
Cache rates bypass cost accounting
When a Databricks completion reports cache-read or cache-creation prompt tokens, the provider-specific calculator charges every prompt token at input_cost_per_token instead of the cache rates added here, causing recorded spend and budget accounting to overcharge cached traffic.
There was a problem hiding this comment.
Thanks for the review — this is actually already handled.
The Databricks provider routes through generic_cost_per_token (see litellm/llms/databricks/cost_calculator.py), which calls get_billable_input_tokens() — it subtracts cached tokens from prompt tokens and charges them at cache_read_input_token_cost / cache_creation_input_token_cost. The existing tests in tests/test_litellm/llms/databricks/test_databricks_cost_calculator.py (test_cached_tokens_bill_at_cache_rates) assert exactly this path, and they pass on the latest commit.
The cache rates added by this PR were previously missing for the new models, which meant cached tokens fell back to the full input rate — the fix in 6961442e4b completes those fields (and corrects a few copied-by-mistake values), so cached Databricks traffic is now billed at the published cache DBU rates.
…s and refresh Gemini 2.5 rates Adds 19 model entries from the official Databricks supported models docs that were missing from the price map: - OpenAI: gpt-5-6-sol/terra/luna, gpt-5-5, gpt-5-5-pro - Google: gemini-3-8-flash, gemini-3-7-flash, gemini-3-6-flash, gemini-3-5-flash, gemini-3-5-flash-lite, gemini-3-1-flash-image, gemini-3-pro-image - Alibaba: qwen35-122b-a10b, qwen3-next-80b-a3b-instruct, qwen3-embedding-0-6b - Zhipu: glm-5-3 (new), glm-5-3-flash (DBU rates now published) - xAI: grok-4-6, Thinking Machines: inkling, Anthropic: claude-fable-5-1 All rates derived from the official Databricks Foundation Model Serving DBU tables (USD = DBU x 0.07 / 1M tokens) with cache write/read costs. Also refreshes gemini-2-5-flash/pro DBU rates to current official values (5.357/44.643 and 22.321/178.571 per 1M) and fills the missing DBU cost fields for glm-5-3-flash.
5bfc7a2 to
ee7eed1
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ions Address CI failures and review feedback on the Databricks price map: - gemini-2-5-flash/pro: store the promotional rate (list DBU x 0.8, promo runs to 2027-01-31) as the repo convention requires, with cache-read at 0.1x input - gemini-3-7-flash / gemini-3-8-flash: drop copied rates - Databricks has not published DBU prices for these endpoints yet; keep endpoint metadata only - qwen3-embedding-0-6b: cache rates were copied from the GTE row; embeddings have no published cache DBU, so cache now bills at the input rate like bge/gte - gpt-5-5-pro, qwen3-next-80b-a3b-instruct, qwen35-122b-a10b: declare cache rates at the input rate (official cache columns are n/a) - tests: extend PUBLISHED_DBU_PER_MILLION with the 12 newly published models and update the unpublished-cache-rate tripwire 14 -> 18 Also reverts the gemini-2.5 refresh to the promotional rates upstream already tracked (the published table lists pre-promotion list rates).
|
Superseded by #39388, which carries these Databricks entries with values re-read from the Databricks pricing and supported-models pages. Thanks for the catalog work |
Summary
Adds 19 Databricks Foundation Model API models missing from
model_prices_and_context_window.json(and the packaged backup), based on the official Databricks supported models docs and pricing pages.Added models
databricks-gpt-5-6-sol,databricks-gpt-5-6-terra,databricks-gpt-5-6-luna,databricks-gpt-5-5,databricks-gpt-5-5-prodatabricks-gemini-3-8-flash,databricks-gemini-3-7-flash,databricks-gemini-3-6-flash,databricks-gemini-3-5-flash,databricks-gemini-3-5-flash-lite,databricks-gemini-3-1-flash-image,databricks-gemini-3-pro-imagedatabricks-qwen35-122b-a10b,databricks-qwen3-next-80b-a3b-instruct,databricks-qwen3-embedding-0-6bdatabricks-glm-5-3,databricks-glm-5-3-flash(DBU rates now published on the FMS pricing page)databricks-grok-4-6(xAI),databricks-inkling(Thinking Machines),databricks-claude-fable-5-1(Anthropic)Updates
databricks-gemini-2-5-flash/databricks-gemini-2-5-pro: adds cache write/read rates per the official cache-read DBU columns (rates keep the promotional pricing the repo already tracks)supports_prompt_cachingto newly published models per the official cache DBU columnsgemini-3-7-flash/gemini-3-8-flash: endpoint metadata only — Databricks has not published DBU rates for these yetPUBLISHED_DBU_PER_MILLIONextended with the 12 newly published models; unpublished-cache-rate tripwire 14 → 18Pricing methodology
All rates are derived from the official Databricks Foundation Model Serving DBU tables using the LiteLLM convention USD = DBU × $0.07 per 1M tokens (
*_dbu_cost_per_tokenfields are used in actual cost calculations).Source pages: