fix(databricks): translate reasoning_effort to thinking for Gemini 2.5

[devin-ai-integration]

Replaces #34001 at head c7b607c, moved onto a litellm_ branch with the same commits and authors so CircleCI runs the legacy Databricks suites

TLDR

Problem this solves:

• reasoning_effort on a Databricks Gemini 2.5 model came back as a 400 from Databricks
• The gateway only translated it to thinking when the model name contained claude
• drop_params did not help, since Databricks reports reasoning_effort as supported everywhere

How it solves it:

• A cost-map flag, supports_anthropic_thinking_payload, marks which Databricks models take the Anthropic thinking shape
• Every Databricks Claude entry and both Gemini 2.5 entries carry the flag, so all of them get translated
• Gemini 3, GPT-5 and GPT-OSS keep passing reasoning_effort through untouched, as Databricks documents
• A Claude serving endpoint missing from the cost map still translates by name, as before

Also in this PR, one CI line: the ocr_testing CircleCI job listed tests/test_litellm/ocr/test_rust_bridge.py next to its glob, and #40734 removed that file from the base branch without touching the job, so pytest crashed at collection on every worker ("4 workers [0 items]", exit 123) for any PR that reached the job. This PR was the first to run it since; dropping the stale path lets the job run the four tests/ocr_tests suites for real again

User Flow

Before: a developer calling a Databricks-hosted Gemini 2.5 model through the gateway with reasoning_effort gets a 400 back, while the same request against a Databricks Claude model works

1. They send POST https://litellm-domain/v1/chat/completions with "model": "databricks-gemini-2-5-flash", a user message, and "reasoning_effort": "low"
2. The response is 400 with Invalid JSON payload received. Unknown name "reasoning_effort" at 'generation_config': Cannot find field.
3. They retry without reasoning_effort and get a 200, so they cannot control thinking on that model through the gateway

After: the same request returns 200 and the model reasons at the requested effort

1. They send the same POST https://litellm-domain/v1/chat/completions with "model": "databricks-gemini-2-5-flash", a user message, and "reasoning_effort": "low"
2. The response is 200 with the model's answer
3. The same request against Databricks Claude, Gemini 3, GPT-5 and GPT-OSS models keeps returning 200 as before

Relevant issues

Replacement for #34001, which was the reopened copy of #32786 with the merge conflicts against litellm_internal_staging resolved. The fix and its tests come from #32786 (head 2bff743); its author could not carry it further. #34001 sat on a devin/ head that CircleCI's branch filter skips, so this PR carries the same commits on a litellm_ branch

Linear ticket

Resolves LIT-7665

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both legs below ran at c7b607c, the last commit that touches proxy code. The commit after it (a7ebc12) changes only .circleci/config.yml, which cannot change what the proxy sends, so the proof stands at that hash

Shared setup for both legs. One proxy per leg runs from a worktree checked out at that leg's commit, with the config below, a random free port, --detailed_debug so the payload the proxy sends to Databricks shows up in the log as a -d '{...}' line, and LITELLM_LOCAL_MODEL_COST_MAP=True so the cost map on the checkout is the one loaded. The Databricks workspace used here serves Claude and GPT-OSS but has no Gemini serving endpoint, so the Gemini 2.5 cases show the payload the proxy sends upstream (the part Databricks rejects with the 400) and the workspace answers 404 on both legs; the Claude and GPT-OSS cases are full round trips against the real endpoints

model_list:
  - model_name: databricks-gemini-2-5-flash
    litellm_params:
      model: databricks/databricks-gemini-2-5-flash
      api_key: os.environ/DATABRICKS_API_KEY
      api_base: os.environ/DATABRICKS_API_BASE
  - model_name: databricks-claude-sonnet-4-5
    litellm_params:
      model: databricks/databricks-claude-sonnet-4-5
      api_key: os.environ/DATABRICKS_API_KEY
      api_base: os.environ/DATABRICKS_API_BASE
  - model_name: databricks-claude-opus-4-7
    litellm_params:
      model: databricks/databricks-claude-opus-4-7
      api_key: os.environ/DATABRICKS_API_KEY
      api_base: os.environ/DATABRICKS_API_BASE
  - model_name: databricks-gpt-oss-120b
    litellm_params:
      model: databricks/databricks-gpt-oss-120b
      api_key: os.environ/DATABRICKS_API_KEY
      api_base: os.environ/DATABRICKS_API_BASE

general_settings:
  master_key: sk-1234

LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config proxy_config.yaml --port $PORT --detailed_debug 2>&1 | tee litellm.log

The same request loop runs against each leg, followed by the grep that pulls the upstream payloads out of the log:

for pair in databricks-gemini-2-5-flash:low databricks-claude-sonnet-4-5:low databricks-claude-opus-4-7:high databricks-gpt-oss-120b:low; do
  m=${pair%%:*}; e=${pair##*:}
  echo "=== $m reasoning_effort=$e ==="
  curl -s -w '\nHTTP %{http_code}\n' "http://localhost:$PORT/v1/chat/completions" \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"Why is the sky blue? Answer in one sentence.\"}],\"reasoning_effort\":\"$e\"}"
done
for m in databricks-gemini-2-5-flash databricks-claude-sonnet-4-5; do
  echo "=== /v1/responses $m reasoning.effort=low ==="
  curl -s -w '\nHTTP %{http_code}\n' "http://localhost:$PORT/v1/responses" \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d "{\"model\":\"$m\",\"input\":\"Why is the sky blue? Answer in one sentence.\",\"reasoning\":{\"effort\":\"low\"}}"
done
grep "^-d '{" litellm.log

Before (559247f)

Gemini 2.5 Flash, /v1/chat/completions, reasoning_effort low

1. Send the request from the loop above with "model": "databricks-gemini-2-5-flash" and "reasoning_effort": "low"
2. Response: HTTP 404 from this workspace, litellm.NotFoundError: DatabricksException - {"error_code":"ENDPOINT_NOT_FOUND", ...}
3. Upstream payload from the log, with reasoning_effort sent as is, which is what a workspace that serves the model rejects with Unknown name "reasoning_effort" at 'generation_config':

-d '{'model': 'databricks-gemini-2-5-flash', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'reasoning_effort': 'low'}'

Claude Sonnet 4.5, /v1/chat/completions, reasoning_effort low

1. Send the request with "model": "databricks-claude-sonnet-4-5" and "reasoning_effort": "low"
2. Response: HTTP 200, "content": "The sky is blue because molecules in Earth's atmosphere scatter short-wavelength blue light ...", with "reasoning_content" filled in
3. Upstream payload from the log:

-d '{'model': 'databricks-claude-sonnet-4-5', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'thinking': {'type': 'enabled', 'budget_tokens': 1024}, 'max_tokens': 5120}'

Claude Opus 4.7, /v1/chat/completions, reasoning_effort high

1. Send the request with "model": "databricks-claude-opus-4-7" and "reasoning_effort": "high"
2. Response: HTTP 200, "content": "The sky appears blue because molecules in Earth's atmosphere scatter shorter wavelengths of sunlight (blue) ..."
3. Upstream payload from the log:

-d '{'model': 'databricks-claude-opus-4-7', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'thinking': {'type': 'adaptive', 'display': 'summarized'}, 'output_config': {'effort': 'high'}}'

GPT-OSS 120B, /v1/chat/completions, reasoning_effort low

1. Send the request with "model": "databricks-gpt-oss-120b" and "reasoning_effort": "low"
2. Response: HTTP 200, "content": "The sky appears blue because molecules in Earth's atmosphere scatter shorter (blue) wavelengths ...", "reasoning_content": "Provide concise answer."
3. Upstream payload from the log:

-d '{'model': 'databricks-gpt-oss-120b', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'reasoning_effort': 'low'}'

Gemini 2.5 Flash, /v1/responses, reasoning.effort low

1. Send the request with "model": "databricks-gemini-2-5-flash" and "reasoning": {"effort": "low"}
2. Response: HTTP 404 from this workspace, same ENDPOINT_NOT_FOUND body
3. Upstream payload from the log, again carrying reasoning_effort:

-d '{'model': 'databricks-gemini-2-5-flash', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'reasoning_effort': 'low'}'

Claude Sonnet 4.5, /v1/responses, reasoning.effort low

1. Send the request with "model": "databricks-claude-sonnet-4-5" and "reasoning": {"effort": "low"}
2. Response: HTTP 200, a resp_... object with the answer
3. Upstream payload from the log: 'thinking': {'type': 'enabled', 'budget_tokens': 1024}, 'max_tokens': 5120

After (c7b607c)

Gemini 2.5 Flash, /v1/chat/completions, reasoning_effort low

1. Send the same request with "model": "databricks-gemini-2-5-flash" and "reasoning_effort": "low"
2. Response: HTTP 404 from this workspace, same ENDPOINT_NOT_FOUND body, since it has no Gemini endpoint
3. Upstream payload from the log now carries thinking and no reasoning_effort, the shape Databricks accepts for Gemini 2.5:

-d '{'model': 'databricks-gemini-2-5-flash', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'thinking': {'type': 'enabled', 'budget_tokens': 1024}, 'max_tokens': 5120}'

Claude Sonnet 4.5, /v1/chat/completions, reasoning_effort low

1. Send the same request with "model": "databricks-claude-sonnet-4-5" and "reasoning_effort": "low"
2. Response: HTTP 200, "content": "The sky appears blue because molecules in Earth's atmosphere scatter shorter blue wavelengths ...", with "reasoning_content" filled in
3. Upstream payload from the log, byte for byte the Before payload:

-d '{'model': 'databricks-claude-sonnet-4-5', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'thinking': {'type': 'enabled', 'budget_tokens': 1024}, 'max_tokens': 5120}'

Claude Opus 4.7, /v1/chat/completions, reasoning_effort high

1. Send the same request with "model": "databricks-claude-opus-4-7" and "reasoning_effort": "high"
2. Response: HTTP 200, "content": "The sky appears blue because molecules in Earth's atmosphere scatter shorter blue wavelengths of sunlight more strongly ..."
3. Upstream payload from the log, byte for byte the Before payload, so the adaptive thinking path for the newer Claude models is unchanged:

-d '{'model': 'databricks-claude-opus-4-7', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'thinking': {'type': 'adaptive', 'display': 'summarized'}, 'output_config': {'effort': 'high'}}'

GPT-OSS 120B, /v1/chat/completions, reasoning_effort low

1. Send the same request with "model": "databricks-gpt-oss-120b" and "reasoning_effort": "low"
2. Response: HTTP 200, "content": "The sky appears blue because molecules in Earth's atmosphere scatter shorter (blue) wavelengths ...", "reasoning_content": "Provide concise answer."
3. Upstream payload from the log, byte for byte the Before payload, so reasoning_effort still passes through for the models that take it natively:

-d '{'model': 'databricks-gpt-oss-120b', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'stream': False, 'reasoning_effort': 'low'}'

Gemini 2.5 Flash, /v1/responses, reasoning.effort low

1. Send the same request with "model": "databricks-gemini-2-5-flash" and "reasoning": {"effort": "low"}
2. Response: HTTP 404 from this workspace, same ENDPOINT_NOT_FOUND body
3. Upstream payload from the log now carries thinking instead of reasoning_effort, so the Responses API route gets the same fix:

-d '{'model': 'databricks-gemini-2-5-flash', 'messages': [{'role': 'user', 'content': 'Why is the sky blue? Answer in one sentence.'}], 'thinking': {'type': 'enabled', 'budget_tokens': 1024}, 'max_tokens': 5120}'

Claude Sonnet 4.5, /v1/responses, reasoning.effort low

1. Send the same request with "model": "databricks-claude-sonnet-4-5" and "reasoning": {"effort": "low"}
2. Response: HTTP 200, a resp_... object with the answer
3. Upstream payload from the log, byte for byte the Before payload: 'thinking': {'type': 'enabled', 'budget_tokens': 1024}, 'max_tokens': 5120

Type

🐛 Bug Fix

Caveats (if any)

Medium

• The Gemini 2.5 leg of the proof stops at the payload the proxy sends upstream. The QA workspace serves 22 endpoints and no Gemini one (GET /api/2.0/serving-endpoints), so both legs get a 404 there; the Claude and GPT-OSS cases are full round trips through the same gate and mapping. A workspace whose region serves databricks-gemini-2-5-flash is what closes it
• reasoning_effort: "none" on a flagged non-Claude model follows the shared Anthropic mapping: the proxy sends no thinking key and the model's own default applies, which for Gemini 2.5 means thinking stays on. The alternative, sending thinking: {"type": "disabled"} for those models, was not taken because the QA workspace cannot confirm Databricks accepts it for Gemini 2.5 and a wrong guess turns none back into the 400 this PR removes. A caller who needs thinking off passes "thinking": {"type": "disabled"} explicitly, which the proxy forwards unchanged

Low

• A request without max_tokens gets max_tokens set to the thinking budget plus 4096 (5120 for low), the same default the Anthropic, Bedrock and Vertex Claude paths apply and Databricks Claude already had; Databricks requires budget_tokens below max_tokens for Claude and documents nothing for Gemini 2.5. Passing max_tokens overrides it
• A Databricks endpoint that is neither flagged in the cost map nor named claude still passes reasoning_effort through unchanged. That is the documented behavior for Gemini 3, GPT-5 and GPT-OSS. A custom-named endpoint that needs the translation gets model_info: {supports_anthropic_thinking_payload: true} on its proxy config deployment, which registers the flag through the router
• Databricks retires Gemini 2.5 Flash and Gemini 2.5 Pro on October 2, 2026, and Gemini 3.x on Databricks takes reasoning_effort natively, so the Gemini part of this fix has a short window; the flag keeps working for Claude endpoints
• The thinking tokens guard from fix(databricks): translate reasoning_effort to thinking for Gemini 2.5 #32786 already landed on the base branch on its own, so it is not part of this diff

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/aacc049712434d50888cf457809a50b1

• a7ebc12 passes /live-pr-risk

---

Note

Medium Risk
Changes upstream request shaping for Databricks chat completions; wrong flags could mis-translate reasoning params for some endpoints, though behavior for unflagged models is unchanged.

Overview
Fixes Databricks chat requests that sent reasoning_effort on models that only accept the Anthropic-style thinking payload (notably Gemini 2.5), which caused upstream 400s.

Databricks mapping now decides translation via _uses_anthropic_thinking_param(): Claude names still match, and any model with supports_anthropic_thinking_payload in the cost map (or deploy model_info) gets reasoning_effort mapped through AnthropicConfig._map_reasoning_effort and stripped from the outbound request. Gemini 3, GPT-5, and GPT-OSS stay on passthrough reasoning_effort.

Adds the new capability flag end-to-end (supports_anthropic_thinking_payload helper, types, schema, and entries on Databricks Claude + Gemini 2.5 models in the price/context JSON), plus unit tests for Claude, adaptive Claude, Gemini 2.5/2.0/3, and GPT paths.

CI: the ocr_testing job drops a hard-coded reference to removed tests/test_litellm/ocr/test_rust_bridge.py so pytest collection succeeds again.

^{Reviewed by Cursor Bugbot for commit a7ebc12. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

The PR translates reasoning_effort into Anthropic-style thinking parameters for Databricks models explicitly marked as supporting that payload, while preserving existing Claude endpoint compatibility and native pass-through behavior for other model families.

• Adds a model-metadata capability flag for Databricks Claude and Gemini 2.5 entries.
• Wires the capability through model information types, utilities, and the cost-map schema.
• Adds regression coverage for Gemini 2.5, Claude legacy/adaptive behavior, unmapped Claude endpoints, and pass-through models.
• Removes an obsolete CircleCI reference to an already-deleted Rust OCR bridge test.

Confidence Score: 5/5

The PR appears safe to merge with no outstanding actionable findings.

The metadata-driven translation is consistently wired through the cost maps, schema, model information, request transformation, and focused tests. The previous hardcoded-capability finding was withdrawn after mateo-berri explained that the Claude name check preserves compatibility for user-named endpoints absent from the cost map, and the subsequent Greptile reply conceded the point.

Important Files Changed

Filename	Overview
litellm/llms/databricks/chat/transformation.py	Uses model metadata plus the established Claude compatibility fallback to select Anthropic-style reasoning translation.
model_prices_and_context_window.json	Marks Databricks models that require the Anthropic thinking payload.
litellm/utils.py	Exposes and propagates the new model capability through existing model-info infrastructure.
model_prices_and_context_window.schema.json	Adds schema validation for the new boolean capability.
tests/test_litellm/llms/databricks/chat/test_databricks_chat_transformation.py	Covers translated and pass-through reasoning behavior across relevant Databricks model families.
.circleci/config.yml	Removes a stale OCR test path whose referenced file no longer exists.

_{Reviews (3): Last reviewed commit: "ci: drop the removed rust bridge test pa..." | Re-trigger Greptile}

[greptile-apps]

Hardcoded capability check

The literal Claude check hardcodes model capability detection. Repository rules require model-specific capabilities to come from model metadata before merging

Rule Used: What: Do not hardcode model-specific flags in the codebase. Instead, put them in model_prices_and_context_window.json and then read them in via get_model_info Why: Prevents need for users to upgrade litellm each time a new model supports this featu... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[mateo-berri]

The name check predates this PR and covers user-named Databricks Claude endpoints that have no cost-map entry. Removing it would 400 requests that work today

[greptile-apps]

You're right — removing the name check would be a backwards-incompatible change for user-named Databricks Claude endpoints that are absent from the cost map. The metadata flag is the preferred path for mapped models, while "claude" in normalized is a necessary compatibility fallback for unmapped endpoints. I withdraw this comment.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_databricks_reasoning_effort_thinking (a7ebc12) with litellm_internal_staging (426e675)}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a7ebc12. Configure here.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

LGTM