Skip to content

feat(model_prices): add missing Databricks Foundation Model API models and refresh Gemini 2.5 rates - #39714

Closed
leecoder wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
leecoder:feat/databricks-model-prices
Closed

feat(model_prices): add missing Databricks Foundation Model API models and refresh Gemini 2.5 rates#39714
leecoder wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
leecoder:feat/databricks-model-prices

Conversation

@leecoder

@leecoder leecoder commented Sep 4, 2026

Copy link
Copy Markdown

Summary

Adds 19 Databricks Foundation Model API models missing from model_prices_and_context_window.json (and the packaged backup), based on the official Databricks supported models docs and pricing pages.

Added models

  • OpenAI: databricks-gpt-5-6-sol, databricks-gpt-5-6-terra, databricks-gpt-5-6-luna, databricks-gpt-5-5, databricks-gpt-5-5-pro
  • Google: databricks-gemini-3-8-flash, databricks-gemini-3-7-flash, databricks-gemini-3-6-flash, databricks-gemini-3-5-flash, databricks-gemini-3-5-flash-lite, databricks-gemini-3-1-flash-image, databricks-gemini-3-pro-image
  • Alibaba: databricks-qwen35-122b-a10b, databricks-qwen3-next-80b-a3b-instruct, databricks-qwen3-embedding-0-6b
  • Zhipu: databricks-glm-5-3, databricks-glm-5-3-flash (DBU rates now published on the FMS pricing page)
  • Others: databricks-grok-4-6 (xAI), databricks-inkling (Thinking Machines), databricks-claude-fable-5-1 (Anthropic)

Updates

  • databricks-gemini-2-5-flash / databricks-gemini-2-5-pro: adds cache write/read rates per the official cache-read DBU columns (rates keep the promotional pricing the repo already tracks)
  • Adds cache write/read cost fields + supports_prompt_caching to newly published models per the official cache DBU columns
  • gemini-3-7-flash / gemini-3-8-flash: endpoint metadata only — Databricks has not published DBU rates for these yet
  • Tests: PUBLISHED_DBU_PER_MILLION extended with the 12 newly published models; unpublished-cache-rate tripwire 14 → 18

Pricing methodology

All rates are derived from the official Databricks Foundation Model Serving DBU tables using the LiteLLM convention USD = DBU × $0.07 per 1M tokens (*_dbu_cost_per_token fields are used in actual cost calculations).

Source pages:

@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR expands both model-price registries with Databricks Foundation Model metadata, refreshes Gemini 2.5 pricing, and adds prompt-cache pricing and capability fields. The two registry copies are synchronized, but the Databricks cost path does not consume the newly supplied cache rates.

  • Adds Databricks model identifiers, context limits, capabilities, and pricing.
  • Updates existing Gemini 2.5 rates.
  • Adds cache-read and cache-creation pricing across existing and new Databricks entries.

Confidence Score: 4/5

The PR should not merge until Databricks cached-token usage is charged using the cache-read and cache-creation rates added by this change.

Databricks requests bypass the generic cache-aware calculator, causing cached prompt tokens to remain priced at the full input rate in recorded spend and budget accounting.

Files Needing Attention: model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds and updates Databricks pricing metadata, but its cache-rate fields are ineffective in the provider-specific cost path.
litellm/model_prices_and_context_window_backup.json Mirrors the root pricing registry, including the same ineffective Databricks cache-rate metadata.

Reviews (1): Last reviewed commit: "feat(model_prices): add missing Databric..." | Re-trigger Greptile

Comment thread model_prices_and_context_window.json Outdated
Comment on lines +47080 to +47082
"cache_creation_input_token_cost": 1.250004e-05,
"cache_read_input_token_cost": 1.00002e-06,
"input_cost_per_token": 1.000006e-05,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Cache rates bypass cost accounting

When a Databricks completion reports cache-read or cache-creation prompt tokens, the provider-specific calculator charges every prompt token at input_cost_per_token instead of the cache rates added here, causing recorded spend and budget accounting to overcharge cached traffic.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review — this is actually already handled.

The Databricks provider routes through generic_cost_per_token (see litellm/llms/databricks/cost_calculator.py), which calls get_billable_input_tokens() — it subtracts cached tokens from prompt tokens and charges them at cache_read_input_token_cost / cache_creation_input_token_cost. The existing tests in tests/test_litellm/llms/databricks/test_databricks_cost_calculator.py (test_cached_tokens_bill_at_cache_rates) assert exactly this path, and they pass on the latest commit.

The cache rates added by this PR were previously missing for the new models, which meant cached tokens fell back to the full input rate — the fix in 6961442e4b completes those fields (and corrects a few copied-by-mistake values), so cached Databricks traffic is now billed at the published cache DBU rates.

…s and refresh Gemini 2.5 rates

Adds 19 model entries from the official Databricks supported models docs
that were missing from the price map:

- OpenAI: gpt-5-6-sol/terra/luna, gpt-5-5, gpt-5-5-pro
- Google: gemini-3-8-flash, gemini-3-7-flash, gemini-3-6-flash,
  gemini-3-5-flash, gemini-3-5-flash-lite, gemini-3-1-flash-image,
  gemini-3-pro-image
- Alibaba: qwen35-122b-a10b, qwen3-next-80b-a3b-instruct,
  qwen3-embedding-0-6b
- Zhipu: glm-5-3 (new), glm-5-3-flash (DBU rates now published)
- xAI: grok-4-6, Thinking Machines: inkling, Anthropic: claude-fable-5-1

All rates derived from the official Databricks Foundation Model Serving
DBU tables (USD = DBU x 0.07 / 1M tokens) with cache write/read costs.

Also refreshes gemini-2-5-flash/pro DBU rates to current official values
(5.357/44.643 and 22.321/178.571 per 1M) and fills the missing DBU cost
fields for glm-5-3-flash.
@leecoder
leecoder force-pushed the feat/databricks-model-prices branch from 5bfc7a2 to ee7eed1 Compare September 4, 2026 06:40
@codspeed-hq

codspeed-hq Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing leecoder:feat/databricks-model-prices (6961442) with litellm_internal_staging (c8635ec)

Open in CodSpeed

@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…ions

Address CI failures and review feedback on the Databricks price map:

- gemini-2-5-flash/pro: store the promotional rate (list DBU x 0.8,
  promo runs to 2027-01-31) as the repo convention requires, with
  cache-read at 0.1x input
- gemini-3-7-flash / gemini-3-8-flash: drop copied rates - Databricks
  has not published DBU prices for these endpoints yet; keep endpoint
  metadata only
- qwen3-embedding-0-6b: cache rates were copied from the GTE row;
  embeddings have no published cache DBU, so cache now bills at the
  input rate like bge/gte
- gpt-5-5-pro, qwen3-next-80b-a3b-instruct, qwen35-122b-a10b: declare
  cache rates at the input rate (official cache columns are n/a)
- tests: extend PUBLISHED_DBU_PER_MILLION with the 12 newly published
  models and update the unpublished-cache-rate tripwire 14 -> 18

Also reverts the gemini-2.5 refresh to the promotional rates upstream
already tracked (the published table lists pre-promotion list rates).
@devin-ai-integration

Copy link
Copy Markdown
Contributor

Superseded by #39388, which carries these Databricks entries with values re-read from the Databricks pricing and supported-models pages. Thanks for the catalog work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant