Skip to content

fix(anthropic): drop non-1 temperature when downgrading adaptive thinking - #33207

Closed
axelray-dev wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
axelray-dev:litellm_fix_anthropic_adaptive_temp_33203
Closed

fix(anthropic): drop non-1 temperature when downgrading adaptive thinking#33207
axelray-dev wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
axelray-dev:litellm_fix_anthropic_adaptive_temp_33203

Conversation

@axelray-dev

Copy link
Copy Markdown

Summary

When adaptive thinking is downgraded to legacy extended thinking for pre-4.6 Anthropic models, drop a non-1 temperature so the request is not rejected by Anthropic.

Changes

  • In _translate_adaptive_effort_for_non_adaptive_model, after setting legacy thinking.type=enabled, remove temperature when it is present and not 1
  • Leave temperature untouched for adaptive-capable models and when thinking is dropped entirely
  • Add regression tests for Haiku 4.5 (drop/preserve) and Sonnet 4.6 (preserve temperature with adaptive)

Verification

  • Focused tests in tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_messages_effort.py

Backwards compatibility

No public API changes. Only strips an invalid temperature combination after adaptive-to-legacy translation.

Fixes #33203

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing axelray-dev:litellm_fix_anthropic_adaptive_temp_33203 (a74b353) with litellm_internal_staging (b200d66)

Open in CodSpeed

@axelray-dev
axelray-dev marked this pull request as ready for review July 14, 2026 18:34
@greptile-apps

greptile-apps Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a rejection from Anthropic's API that occurs when a non-1 temperature is sent alongside legacy extended thinking (thinking.type=enabled) on pre-4.6 models. When the adaptive-to-legacy downgrade path produces a type=enabled thinking block, the fix now strips temperature if it is not 1.

  • Two lines added in _translate_adaptive_effort_for_non_adaptive_model to drop non-1 temperature after setting legacy thinking.type=enabled; adaptive-capable models (4.6+) and the "drop thinking entirely" branch are deliberately unaffected.
  • Three new unit tests cover the three distinct cases: temperature dropped for pre-4.6 legacy thinking, temperature=1 preserved, and temperature untouched for adaptive thinking on 4.6.

Confidence Score: 5/5

Safe to merge — the change is a two-line guard in a single downgrade path that can only remove a parameter Anthropic would reject anyway, and the new tests confirm all three temperature cases behave correctly.

The change is narrowly scoped to the adaptive-to-legacy translation path and cannot affect adaptive models or requests without thinking. The condition is logically correct for all temperature values (absent, 0, 1, 1.0), the pop default makes the absent-temperature case a no-op, and the three new tests directly exercise the regression being fixed. The only note is a minor readability improvement available on the condition guard.

No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/anthropic/experimental_pass_through/messages/transformation.py Adds two lines to drop temperature when it is not 1 during adaptive-to-legacy thinking downgrade; minor: the guard fires pop even when temperature is absent (harmless no-op due to default argument)
tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_messages_effort.py Adds three focused unit tests (temperature=0 dropped, temperature=1 preserved for legacy thinking, temperature=0 preserved for adaptive thinking on 4.6); pure mock tests with no real network calls

Reviews (1): Last reviewed commit: "fix(anthropic): drop non-1 temperature w..." | Re-trigger Greptile

Comment on lines +367 to +368
if capped_thinking.get("type") == "enabled" and optional_params.get("temperature") != 1:
optional_params.pop("temperature", None)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The condition optional_params.get("temperature") != 1 evaluates to True when temperature is absent (None != 1), so pop("temperature", None) is called on every legacy-thinking translation even when no temperature was set. The pop default prevents any error, but the intent — "only strip an explicitly-provided non-1 temperature" — is clearer with an explicit membership check.

Suggested change
if capped_thinking.get("type") == "enabled" and optional_params.get("temperature") != 1:
optional_params.pop("temperature", None)
if (
capped_thinking.get("type") == "enabled"
and "temperature" in optional_params
and optional_params["temperature"] != 1
):
optional_params.pop("temperature")

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@axelray-dev

Copy link
Copy Markdown
Author

Closing this as superseded. The adaptive-to-legacy temperature drop already landed in #33244 (merged by krrish-berri-2), and #33203 is closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant