feat(databricks): price Unity Catalog system.ai model services

[elabbarw]

TLDR

Problem this solves:

• Databricks Unity Catalog serves every foundation model as system.ai.<name>
• databricks/system.ai.claude-opus-5 had no pricing, so spend tracked as $0
• Unity Responses and embedding requests were sent to the legacy /serving-endpoints URL
• Four Databricks Gemini entries had no rates at all

How it solves it:

• Adds a databricks/system.ai.<name> twin for all 62 databricks/databricks-<name> entries
• Twins are byte-identical copies, so published DBU rates stay in one place per model
• Routes Unity Responses and embeddings to /ai-gateway/mlflow/v1 like chat already did
• Prices Gemini 3.7 Flash, 3.8 Flash, 3 Pro Image and 3.1 Flash Image from Databricks' DBU rows
• Tests keep twins in sync, pin the new DBU rows, and cover the three Unity URL paths

User Flow

Before: a proxy admin who points deployments at Unity Catalog model services sees $0 spend and failing Responses and embedding calls

1. They add model: databricks/system.ai.claude-opus-5 and model: databricks/system.ai.bge-large-en to their config and start the proxy
2. They open http://localhost:4000/ui/?page=models and the rows show no input or output cost
3. They send POST http://localhost:4000/v1/chat/completions for the Claude deployment and it succeeds
4. They send POST http://localhost:4000/v1/embeddings for the embedding deployment and the workspace rejects it, since the request went to /serving-endpoints/embeddings
5. They open http://localhost:4000/ui/?page=logs and the chat request shows $0 spend

After: the same deployments show real Databricks pay-per-token pricing and every request reaches the Unity gateway

1. They add the same two deployments to their config and start the proxy
2. They open http://localhost:4000/ui/?page=models and the Claude row shows $5 input and $25 output per 1M tokens
3. They send POST http://localhost:4000/v1/chat/completions for the Claude deployment and it succeeds
4. They send POST http://localhost:4000/v1/embeddings for the embedding deployment and it succeeds, since the request went to /ai-gateway/mlflow/v1/embeddings
5. They open http://localhost:4000/ui/?page=logs and both requests show non-zero spend matching those rates

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup. No Databricks workspace credentials were available, so a tiny local HTTP server on port 4011 stands in for the workspace: it logs the path and model of every POST and returns a minimal OpenAI-shaped body. Everything else is the real proxy. Config used for both runs:

model_list:
  - model_name: claude-opus-5
    litellm_params:
      model: databricks/system.ai.claude-opus-5
      api_base: http://127.0.0.1:4011/serving-endpoints
      api_key: dummy

  - model_name: gemini-3-8-flash
    litellm_params:
      model: databricks/system.ai.gemini-3-8-flash
      api_base: http://127.0.0.1:4011/serving-endpoints
      api_key: dummy

  - model_name: gpt-5-5
    litellm_params:
      model: databricks/system.ai.gpt-5-5
      api_base: http://127.0.0.1:4011/serving-endpoints
      api_key: dummy

  - model_name: bge-large-en
    litellm_params:
      model: databricks/system.ai.bge-large-en
      api_base: http://127.0.0.1:4011/serving-endpoints
      api_key: dummy

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Started with python litellm/proxy/proxy_cli.py --config proxy_config.yaml --port 4010. Before runs from a worktree checked out at the merge base with the default remote cost map. After runs from this branch with LITELLM_LOCAL_MODEL_COST_MAP=True so the proxy reads this branch's JSON.

Before (18c3118)

Model info shows pricing for the four Unity deployments

1. curl -s http://localhost:4010/model/info -H "Authorization: Bearer $LITELLM_MASTER_KEY" (trimmed to the pricing fields)
2. Output. No rates on any row.

[
  {
    "model": "databricks/system.ai.claude-opus-5",
    "input_cost_per_token": null,
    "output_cost_per_token": null,
    "cache_creation_input_token_cost": null,
    "cache_read_input_token_cost": null,
    "max_input_tokens": 200000
  },
  {
    "model": "databricks/system.ai.gemini-3-8-flash",
    "input_cost_per_token": null,
    "output_cost_per_token": null,
    "cache_creation_input_token_cost": null,
    "cache_read_input_token_cost": null,
    "max_input_tokens": null
  },
  {
    "model": "databricks/system.ai.gpt-5-5",
    "input_cost_per_token": null,
    "output_cost_per_token": null,
    "cache_creation_input_token_cost": null,
    "cache_read_input_token_cost": null,
    "max_input_tokens": null
  },
  {
    "model": "databricks/system.ai.bge-large-en",
    "input_cost_per_token": null,
    "output_cost_per_token": null,
    "cache_creation_input_token_cost": null,
    "cache_read_input_token_cost": null,
    "max_input_tokens": null
  }
]

Spend for claude-opus-5

1. curl -s -X POST http://localhost:4010/spend/calculate -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"completion_response":{"model":"databricks/system.ai.claude-opus-5","usage":{"prompt_tokens":1000000,"completion_tokens":1000000,"total_tokens":2000000}}}'
2. Output.

{"cost":0.0}

Spend for gemini-3-8-flash

1. curl -s -X POST http://localhost:4010/spend/calculate -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"completion_response":{"model":"databricks/system.ai.gemini-3-8-flash","usage":{"prompt_tokens":1000000,"completion_tokens":1000000,"total_tokens":2000000}}}'
2. Output.

{"cost":0.0}

Chat, Responses and embeddings reach the right upstream URL

1. curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:4010/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"claude-opus-5","messages":[{"role":"user","content":"hi"}]}' gives HTTP 200
2. curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:4010/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-5","input":"hi"}' gives HTTP 200
3. curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:4010/v1/embeddings -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"bge-large-en","input":"hi"}' gives HTTP 200
4. The fake workspace on port 4011 logs the URL each request arrived at. Chat already reaches the gateway. Responses and embeddings go to the legacy base, which a real workspace rejects for a system.ai name.

UPSTREAM GOT POST /ai-gateway/mlflow/v1/chat/completions model=system.ai.claude-opus-5
UPSTREAM GOT POST /serving-endpoints/responses model=system.ai.gpt-5-5
UPSTREAM GOT POST /serving-endpoints/embeddings model=system.ai.bge-large-en

After (a7a6043)

Model info shows pricing for the four Unity deployments

1. curl -s http://localhost:4010/model/info -H "Authorization: Bearer $LITELLM_MASTER_KEY" (trimmed to the pricing fields)
2. Output. Databricks pay-per-token rates on all four rows. The Gemini row is 10.714 and 53.571 DBU times $0.070, which matches Google's own list price.

[
  {
    "model": "databricks/system.ai.claude-opus-5",
    "input_cost_per_token": 5.00003e-06,
    "output_cost_per_token": 2.500001e-05,
    "cache_creation_input_token_cost": 6.25002e-06,
    "cache_read_input_token_cost": 5.0001e-07,
    "max_input_tokens": 1000000
  },
  {
    "model": "databricks/system.ai.gemini-3-8-flash",
    "input_cost_per_token": 7.4998e-07,
    "output_cost_per_token": 3.74997e-06,
    "cache_creation_input_token_cost": 7.4998e-07,
    "cache_read_input_token_cost": 7.497e-08,
    "max_input_tokens": 1048576
  },
  {
    "model": "databricks/system.ai.gpt-5-5",
    "input_cost_per_token": 5.00003e-06,
    "output_cost_per_token": 2.999997e-05,
    "cache_creation_input_token_cost": 5.00003e-06,
    "cache_read_input_token_cost": 5.0001e-07,
    "max_input_tokens": 922000
  },
  {
    "model": "databricks/system.ai.bge-large-en",
    "input_cost_per_token": 1.0003e-07,
    "output_cost_per_token": 0.0,
    "cache_creation_input_token_cost": 1.0003e-07,
    "cache_read_input_token_cost": 1.0003e-07,
    "max_input_tokens": 512
  }
]

Spend for claude-opus-5

1. curl -s -X POST http://localhost:4010/spend/calculate -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"completion_response":{"model":"databricks/system.ai.claude-opus-5","usage":{"prompt_tokens":1000000,"completion_tokens":1000000,"total_tokens":2000000}}}'
2. Output. 1M prompt tokens at $5 plus 1M completion tokens at $25.

{"cost":30.00004}

Spend for gemini-3-8-flash

1. curl -s -X POST http://localhost:4010/spend/calculate -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"completion_response":{"model":"databricks/system.ai.gemini-3-8-flash","usage":{"prompt_tokens":1000000,"completion_tokens":1000000,"total_tokens":2000000}}}'
2. Output. 1M prompt tokens at $0.75 plus 1M completion tokens at $3.75.

{"cost":4.49995}

Chat, Responses and embeddings reach the right upstream URL

1. curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:4010/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"claude-opus-5","messages":[{"role":"user","content":"hi"}]}' gives HTTP 200
2. curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:4010/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-5","input":"hi"}' gives HTTP 200
3. curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://localhost:4010/v1/embeddings -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"bge-large-en","input":"hi"}' gives HTTP 200
4. The fake workspace on port 4011 logs the URL each request arrived at. All three request types now reach /ai-gateway/mlflow/v1.

UPSTREAM GOT POST /ai-gateway/mlflow/v1/chat/completions model=system.ai.claude-opus-5
UPSTREAM GOT POST /ai-gateway/mlflow/v1/responses model=system.ai.gpt-5-5
UPSTREAM GOT POST /ai-gateway/mlflow/v1/embeddings model=system.ai.bge-large-en

Type

🆕 New Feature
🐛 Bug Fix

Caveats (if any)

Medium

• The proof uses a local stand-in for the workspace, so it shows the URL the proxy calls, not a real Databricks response
• Databricks marks the Gemini 3.6, 3.7 and 3.8 Flash DBU rates as a 50% promotion through 2026-12-31. The stored rate is the promotional one, which equals Google's list price; after the promotion the list DBU rate is double
• Only the system.ai.<name> form is added. One Databricks docs page also shows system.ai.databricks-<name>; add those if a workspace exposes that form

Low

• The documentation and code-quality checks fail on main as well: DEFER_PYDANTIC_BUILD (added in perf(types): defer pydantic schema builds via shared LiteLLMBaseModel #44720) is not yet documented in litellm-docs, which those checks read. Nothing in this PR touches it
• The Responses URL builder reads the model from litellm_params, which the handler now adds at the four call sites that create a response. Get, delete, list and cancel paths have no model and keep the legacy base
• Each price now lives in two rows; the sync test fails if either side is edited alone

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Co-authored-by: Claude noreply@anthropic.com

[CLAassistant]

All committers have signed the CLA.

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds pricing and routing for Databricks Unity Catalog models.

The PR appears safe to merge based on the current review

Summary

The PR adds Unity Catalog pricing entries for Databricks models, prices four Gemini models, and routes Unity Responses and embeddings through the AI Gateway

• The latest change adds sources and dates beside the new DBU assertions

_{Reviews (3) · Last reviewed commit: "test(databricks): cite source and date f..."}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing elabbarw:feat(databricks-unity)--Add-pricing-for-databricks-unity-hosted-models-prefixed-with-system.ai (4563d2f) with main (d4619c4)¹}

Footnotes

1. No successful run was found on main (10444df) during the generation of this report, so d4619c4 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[elabbarw]

@greptileai please re-review. Routing for Unity Responses and embeddings fixed, and Gemini rows now use Databricks DBU rates.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[elabbarw]

@greptileai please re-review. Added source and date beside the new DBU rows; promotional rate kept deliberately, see reply.

[elabbarw]

Any updates on this? The existing Databricks provider does not work.

[elabbarw]

I can workaround this by adding the databricks/databricks-xx model name under "base_model" for now.