Skip to content

fix(anthropic): fix clear_thinking retry body after thinking strip - #27514

Closed
mchtech wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
mchtech:fix/anthropic-clear-thinking-retry-official
Closed

fix(anthropic): fix clear_thinking retry body after thinking strip#27514
mchtech wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
mchtech:fix/anthropic-clear-thinking-retry-official

Conversation

@mchtech

@mchtech mchtech commented May 9, 2026

Copy link
Copy Markdown

Summary

  • restore top-level thinking when invalid-signature retry keeps clear_thinking_20251015
  • keep the fix scoped to Anthropic Messages passthrough retry handling
  • avoid carrying manual budget_tokens into adaptive thinking retries
  • add behavior coverage for enabled, adaptive, disabled, missing, and non-adaptive cases

Fixes #27512

Related: #25867 added the same clear_thinking_20251015 request-contract guard for Bedrock. This PR covers the Anthropic Messages passthrough retry path, where the retry cleanup can remove top-level thinking while leaving clear_thinking_20251015 in place.

Tests

.venv/bin/python -m pytest tests/test_litellm/llms/anthropic/test_anthropic_common_utils.py -q

Result: 72 passed

- restore top-level thinking when invalid-signature retry keeps clear_thinking_20251015
- keep the fix scoped to Anthropic Messages passthrough retry handling
- avoid carrying manual budget_tokens into adaptive thinking retries
- add behavior coverage for enabled, adaptive, disabled, missing, and non-adaptive cases
@CLAassistant

CLAassistant commented May 9, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented May 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented May 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR overrides transform_anthropic_messages_request_on_http_error in AnthropicMessagesConfig to restore the caller's original top-level thinking parameter after the base class strips it during an invalid-signature retry, but only when a clear_thinking_20251015 context-management edit is present and the target model supports adaptive thinking.

  • The new method captures thinking before the base call, then conditionally re-adds it post-strip, preserving budget_tokens for enabled mode and dropping it for adaptive mode (server-controlled).
  • Five mock-only test cases are added covering the enabled, adaptive, disabled, missing, no-clear_thinking, and non-adaptive-model paths; all make no real network calls, satisfying the test-isolation rule.

Confidence Score: 4/5

The change is narrow and well-guarded: the restore only fires for adaptive-thinking models with a clear_thinking edit present, and the base-class strip still runs unconditionally before any restore attempt.

The logic correctly handles all edge cases tested (enabled, adaptive without budget, disabled, missing, no clear-thinking edit, non-adaptive model). The not has_active_thinking guard will always be True in the primary retry path because the base class always removes thinking for invalid-signature errors, making it a purely defensive check. The call site discards the return value and relies on in-place mutation, which is consistent with the base class but is an implicit contract.

No files require special attention; the transformation file is the only meaningful change and its scope is tightly controlled.

Important Files Changed

Filename Overview
litellm/llms/anthropic/experimental_pass_through/messages/transformation.py Adds transform_anthropic_messages_request_on_http_error override that restores top-level thinking config after the base class strips it, restricted to adaptive-thinking models with a clear_thinking edit present.
tests/test_litellm/llms/anthropic/test_anthropic_common_utils.py Adds five new mock-only tests and a shared helper covering all branch outcomes of the new retry transform: enabled/adaptive restore, disabled/missing no-restore, no-clear-thinking no-restore, and non-adaptive-model no-restore.

Reviews (1): Last reviewed commit: "fix(anthropic): fix clear_thinking retry..." | Re-trigger Greptile

@krrish-berri-2

Copy link
Copy Markdown
Contributor

@mchtech — could you add a screenshot or short video showing that this change works as expected? It really helps reviewers verify the fix quickly. Thanks!

@mchtech

mchtech commented May 12, 2026

Copy link
Copy Markdown
Author
image

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions Bot added the stale label Aug 11, 2026
@github-actions github-actions Bot closed this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic Messages retry drops thinking while keeping clear_thinking_20251015

4 participants