feat(anthropic): per-deployment opt-out for legacy thinking → adaptive rewrite (port of #77) - #80
Merged
songkuan-zheng merged 1 commit intoJun 10, 2026
Conversation
…e rewrite
Add `disable_adaptive_thinking_rewrite` to `GenericLiteLLMParams`. When
set on a model_list deployment, the /v1/messages passthrough preserves
the caller's legacy `thinking={"type":"enabled","budget_tokens":N}`
verbatim instead of rewriting it to the adaptive form
(`{"type":"adaptive"}` + `output_config.effort`).
Motivation: on Claude Sonnet/Opus 4.6, the legacy thinking shape is
deprecated upstream but still functional. LiteLLM rewrites it
proactively, which discretizes `budget_tokens` into a 4-bucket effort
band — callers that need byte-exact passthrough (precise cache pricing,
A/B parity with non-LiteLLM clients) have no way to opt out today.
The flag is duck-typed: the gate reads from either a dict-shaped
`litellm_params` or a `GenericLiteLLMParams` pydantic instance,
mirroring the `additional_drop_params` pattern. It only short-circuits
the rewrite path — it does NOT bypass upstream Anthropic's 400 rejection
of the legacy shape on Opus 4.7+ (those models reject it at the
provider, not in LiteLLM).
Usage:
model_list:
- model_name: claude-sonnet-4-6-raw
litellm_params:
model: anthropic/claude-sonnet-4-6
api_key: os.environ/ANTHROPIC_API_KEY
disable_adaptive_thinking_rewrite: true
Tests cover: default rewrite preserved, flag-via-dict opt-out, flag-via-
pydantic-model opt-out, no-op on non-adaptive models, and default-False
on a fresh pydantic instance.
Tier: D (universal mechanism + opinionated default of False preserves
existing behavior). To be filed upstream as a passthrough hook.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tier classification
litellm/coreTried upstream first?
fix/billing-accuracy-phase-1's sibling row inUPSTREAM_PR_QUEUE.md(Wave 6d —disable_adaptive_thinking_rewriteopt-in flag).Summary
Port of #77 from
ship/v1.83.10toship/v1.87.0. The two ship lines diverge at the upstream pin and are maintained in parallel — landing the fix on v1.87.0 keeps them in sync.Adds
disable_adaptive_thinking_rewritetoGenericLiteLLMParams. When set on a model_list deployment, the /v1/messages passthrough preserves the caller's legacythinking={"type":"enabled","budget_tokens":N}verbatim instead of rewriting it to the adaptive form ({"type":"adaptive"}+output_config.effort).Motivation
On Claude Sonnet/Opus 4.6, the legacy thinking shape is deprecated upstream but still functional. LiteLLM rewrites it proactively, which discretizes
budget_tokensinto a 4-bucket effort band — callers that need byte-exact passthrough (precise cache pricing, A/B parity with non-LiteLLM clients) have no way to opt out today.The flag is duck-typed: the gate reads from either a dict-shaped
litellm_paramsor aGenericLiteLLMParamspydantic instance, mirroring theadditional_drop_paramspattern. It only short-circuits the rewrite path — it does NOT bypass upstream Anthropic's 400 rejection of the legacy shape on Opus 4.7+ (those models reject it at the provider, not in LiteLLM).Files
litellm/llms/anthropic/experimental_pass_through/messages/transformation.py— rewrite gate.litellm/types/router.py—disable_adaptive_thinking_rewrite: Optional[bool]onGenericLiteLLMParams.tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_legacy_thinking_passthrough.py— 5 tests (default behavior, dict-shape opt-out, pydantic-model opt-out, no-op on non-adaptive models, default-False on a fresh pydantic instance).Verification
fix/anthropic-passthrough-thinking(v1.83.10 base) onto v1.87.0 base resolved cleanly (auto-merge handled both production files).tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_legacy_thinking_passthrough.py).False— existing rewrite behavior is preserved for all deployments that don't opt in.Conflict resolutions
N/A —
git cherry-pick 31f189c7b7reportedAuto-mergingon both source files and produced no conflict markers.Pre-submission
tests/test_litellm/(5 tests in the per-file passthrough test)make test-unitequivalent passes