Skip to content

feat(anthropic): add opt-out flag for default reasoning summary#22904

Merged
Chesars merged 2 commits intoBerriAI:litellm_oss_staging_03_05_2026from
Chesars:claude/add-claude-param-default-M4Yic
Mar 5, 2026
Merged

feat(anthropic): add opt-out flag for default reasoning summary#22904
Chesars merged 2 commits intoBerriAI:litellm_oss_staging_03_05_2026from
Chesars:claude/add-claude-param-default-M4Yic

Conversation

@Chesars
Copy link
Copy Markdown
Contributor

@Chesars Chesars commented Mar 5, 2026

Relevant issues

Allows users to disable the automatic "summary": "detailed" injection when LiteLLM translates Anthropic thinking parameters to OpenAI reasoning_effort.

Pre-Submission checklist

  • I have Added testing in the tests/test_litellm/ directory
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai

Type

🆕 New Feature

Changes

  • Add litellm.disable_default_reasoning_summary flag (default False) in litellm/__init__.py
  • Support LITELLM_DISABLE_DEFAULT_REASONING_SUMMARY env var as alternative
  • Update adapters/handler.py to skip injecting "summary": "detailed" when flag is active (both string and dict reasoning_effort)
  • Update responses_adapters/transformation.py to respect the flag in translate_thinking_to_reasoning
  • Add 6 unit tests covering: flag on/off, env var, user-provided summary preservation

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 5, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Error Error Mar 5, 2026 2:59pm

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 5, 2026

Greptile Summary

This PR adds a litellm.disable_default_reasoning_summary flag (and LITELLM_DISABLE_DEFAULT_REASONING_SUMMARY env var) to let users suppress the automatic injection of "summary": "detailed" when Anthropic thinking parameters are translated to OpenAI reasoning_effort. The shared helper is_default_reasoning_summary_disabled() is correctly extracted into utils.py and the flag is applied in two of the three conversion code paths.

Critical issues found:

  • Backwards-incompatible default: The flag defaults to False, which means summary: "detailed" is now injected for every request that previously produced no summary. This silently changes the API payload sent downstream for all existing users of the experimental pass-through. Following the project's backwards-compatibility guidelines, this behavioral change should be opt-in — either default the flag to True, or rename it to enable_default_reasoning_summary defaulting to False.
  • translate_thinking_for_model not updated: adapters/transformation.py lines 692–700 is a third code path that converts thinkingreasoning_effort, but it was not modified to inject summary: "detailed" by default. This creates inconsistent behavior depending on which code path a request follows.

Confidence Score: 1/5

  • Not safe to merge: breaks backwards compatibility for all existing users by changing default API payloads, and one code path is not updated creating inconsistent behavior.
  • Two blocking issues prevent merging: (1) The flag defaults to False, which silently injects summary: "detailed" into requests that previously never included it. This breaks existing users without their consent and violates the backwards-compatibility rule. (2) The function translate_thinking_for_model in adapters/transformation.py was not updated to match the new default behavior, creating inconsistent outputs depending on which code path handles a request. Both issues require code changes to resolve.
  • litellm/init.py (flag default must change), litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py (function not updated)

Comments Outside Diff (1)

  1. General comment

    translate_thinking_for_model not updated — inconsistent default behavior

    translate_thinking_for_model is a third code path that converts Anthropic thinking to an OpenAI reasoning_effort value, but it does not check the disable_default_reasoning_summary flag. It only includes summary when the user explicitly provides one, unlike the two updated paths (_route_openai_thinking_to_responses_api_if_needed and translate_thinking_to_reasoning) which inject summary: "detailed" by default when the flag is False.

    This creates inconsistent behavior depending on which code path a request follows. To align the three code paths, translate_thinking_for_model should also consult is_default_reasoning_summary_disabled():

    This requires importing the helper at the top of the file and updating the corresponding test at line 224 to expect {"reasoning_effort": {"effort": "minimal", "summary": "detailed"}} by default.

Last reviewed commit: 3e9ea6f

Chesars added 2 commits March 5, 2026 11:58
Add `litellm.disable_default_reasoning_summary` flag (default False) and
env var `LITELLM_DISABLE_DEFAULT_REASONING_SUMMARY` to allow users to
opt out of the automatic `summary="detailed"` injection when routing
Anthropic thinking requests to OpenAI's Responses API.

Default behavior is preserved (summary="detailed" is always added),
but users who don't want to pay for summary tokens can now disable it.

https://claude.ai/code/session_01VJU9EwVvgvmeCe3Yu1aULa
…issing env var test

- Extract duplicated summary_disabled evaluation from handler.py and
  transformation.py into a shared is_default_reasoning_summary_disabled()
  helper in utils.py to prevent future divergence.
- Add test_summary_excluded_when_env_var_set to handler test class to
  close env-var test coverage gap flagged by Greptile.
@Chesars Chesars force-pushed the claude/add-claude-param-default-M4Yic branch from 0e59ad8 to 3e9ea6f Compare March 5, 2026 14:58
@Chesars Chesars merged commit c9e60d9 into BerriAI:litellm_oss_staging_03_05_2026 Mar 5, 2026
3 of 4 checks passed
@Chesars Chesars deleted the claude/add-claude-param-default-M4Yic branch March 5, 2026 14:59
guardrail_name_config_map: Dict[str, GuardrailItem] = {}
include_cost_in_streaming_usage: bool = False
reasoning_auto_summary: bool = False
disable_default_reasoning_summary: bool = False
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backwards-incompatible default behavior change

Setting disable_default_reasoning_summary to False (the default) means summary: "detailed" is now injected into every request that previously received no summary. Before this PR, translate_thinking_to_reasoning({"type": "enabled", "budget_tokens": 5000}) returned {"effort": "medium"}; after this PR it returns {"effort": "medium", "summary": "detailed"} by default, silently altering all existing callers' API payloads.

Per the repository guidelines on backwards compatibility (rule b48b7341), behavioral changes like this should be opt-in, not opt-out — meaning the old behavior should remain the default and users explicitly set the flag to enable the new behavior. The flag should either:

Option 1: Default to True to preserve old behavior:

Suggested change
disable_default_reasoning_summary: bool = False
disable_default_reasoning_summary: bool = True

Option 2: Rename to make the opt-in nature explicit:

enable_default_reasoning_summary: bool = False

and update the helper in utils.py accordingly.

Context Used: Rule from dashboard - What: avoid backwards-incompatible changes without user-controlled flags

Why: This breaks current ... (source)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant