fix(anthropic): allow effort="max" on Claude Opus 4.7 - #25958
fix(anthropic): allow effort="max" on Claude Opus 4.7#25958shang309073819 wants to merge 3 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR fixes #25957 by allowing Confidence Score: 5/5Safe to merge; all remaining findings are P2 style nits and the core logic is correct. The hardcoded model check is fully replaced with the data-driven approach required by the team's custom rule; supports_max_reasoning_effort is wired through all layers (TypedDict → helper → model map); tests are strengthened not weakened. The only open finding is a trivial indentation inconsistency in two JSON entries. model_prices_and_context_window.json and its backup — minor indentation inconsistency on the vertex_ai/claude-sonnet-4-6 and vertex_ai/claude-sonnet-4-6@default entries.
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/chat/transformation.py | Switches the effort=max guard from a hardcoded _is_opus_4_6_model substring check to the data-driven _supports_effort_level(model, max) helper, matching the team's model-map rule and the existing xhigh pattern. |
| model_prices_and_context_window.json | Adds supports_max_reasoning_effort: true to Opus 4.6/4.7 and Sonnet 4.6 entries (including regional/vertex variants); vertex_ai/claude-sonnet-4-6 and its @default variant have a minor indentation inconsistency on the new key. |
| litellm/model_prices_and_context_window_backup.json | Mirror of model_prices_and_context_window.json changes; same indentation inconsistency on vertex_ai/claude-sonnet-4-6 entries. |
| litellm/types/utils.py | Adds supports_max_reasoning_effort: Optional[bool] to ProviderSpecificModelInfo TypedDict, completing the type coverage for the new model-map key. |
| litellm/utils.py | Threads supports_max_reasoning_effort through _get_model_info_helper so the new TypedDict field is populated from the model cost map. |
| tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py | Updates error-message assertions to match the new generic wording, inverts test_max_effort_rejected_for_sonnet_46 to test_max_effort_accepted_for_sonnet_46 per Anthropic docs, and adds test_max_effort_accepted_for_opus_47 as a regression test for #25957. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["_apply_output_config(model, effort)"] --> B{effort == max?}
B -- No --> E[Pass through to request]
B -- Yes --> C["_supports_effort_level(model, 'max')"]
C --> D["_get_model_info_helper looks up\nsupports_max_reasoning_effort\nin model cost map JSON"]
D -- true --> E
D -- "false or not found" --> F["raise ValueError: effort=max not supported"]
E --> G[Set output_config on request data]
subgraph "Entries with supports_max_reasoning_effort: true"
M1["claude-opus-4-6 and dated variants"]
M2["claude-opus-4-7 and dated variants"]
M3["claude-sonnet-4-6 and dated variants"]
M4["All regional and vertex_ai variants"]
end
Reviews (7): Last reviewed commit: "test(anthropic): force LITELLM_LOCAL_MOD..." | Re-trigger Greptile
| if effort == "max" and not ( | ||
| self._is_opus_4_6_model(model) or self._is_opus_4_7_model(model) | ||
| ): | ||
| raise ValueError( | ||
| f"effort='max' is only supported by Claude Opus 4.6. " | ||
| f"effort='max' is only supported by Claude Opus 4.6 and 4.7. " | ||
| f"Got model: {model}" | ||
| ) |
There was a problem hiding this comment.
Hardcoded model-flag violates the "use model map" rule
The team rule requires that model-capability flags live in model_prices_and_context_window.json and be queried via _supports_effort_level / get_model_info, not hardcoded substring checks. The xhigh effort level already follows this pattern (one line below); max should too.
The Opus 4.7 entries (claude-opus-4-7, claude-opus-4-7-20260416) already carry supports_xhigh_reasoning_effort: true but are missing supports_max_reasoning_effort: true. The complete fix is a two-part change:
- Add
"supports_max_reasoning_effort": trueto the Opus 4.7 entries inmodel_prices_and_context_window.json(and its bedrock/azure_ai/regional variants). - Replace the expanded substring check with the existing data-driven helper:
if effort == "max" and not self._supports_effort_level(model, "max"):
raise ValueError(
f"effort='max' is only supported by Claude Opus 4.6 and 4.7. "
f"Got model: {model}"
)This makes enabling max for any future model a pure JSON change—exactly the pattern the project enforces.
Rule Used: What: Do not hardcode model-specific flags in the ... (source)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Note on the failing After the Concretely, The reason previous PRs against Happy to either:
Let me know which you'd prefer. The actual bugfix in this PR is small and self-contained, and |
| # Opus 4.6 and 4.7, so we keep both substring checks here for | ||
| # date-variant tolerance (e.g. claude-opus-4-7-20260408). See #25957. | ||
| if effort == "max" and not ( | ||
| self._is_opus_4_6_model(model) or self._is_opus_4_7_model(model) |
There was a problem hiding this comment.
Claude Sonnet 4.6 also supports max. https://platform.claude.com/docs/en/build-with-claude/effort#effort-levels
|
Thanks both — addressed in 532c47a. @greptile-apps (P2): swapped the hardcoded @congqiao: good catch — confirmed against Anthropic's effort docs1 that
Plumbed Tests:
All 137 tests in Footnotes
|
b9c7542 to
fa54191
Compare
|
Rebased onto latest Rebase summary:
Re: the lint check — same situation as before (the
|
Anthropic's Messages API accepts `output_config.effort="max"` on `claude-opus-4-7-*` models, but `_apply_output_config` hardcoded the gate to `_is_opus_4_6_model`, so any request hitting LiteLLM with Opus 4.7 + effort=max was rejected with a 400 before reaching Anthropic. This was preserved on purpose in BerriAI#25867 (day-0 4.7 PR) under the old assumption that `max` was Opus-4.6-only. The Anthropic API has since extended `max` support to Opus 4.7. Allow the effort=max guard to also accept Opus 4.7. Update the two existing rejection tests to match the new error message and add a new regression test for Opus 4.7. Fixes BerriAI#25957
Address review feedback on BerriAI#25958: 1. @greptile-apps (P2): replace hardcoded `_is_opus_4_6_model OR _is_opus_4_7_model` substring check with the same data-driven `_supports_effort_level(model, 'max')` pattern already used for `xhigh` one line below. Adding support for a new model is now a pure model-map change. 2. @congqiao: per Anthropic's effort docs[1], `max` is supported by Claude Opus 4.6, Opus 4.7, AND Sonnet 4.6 — the previous patch only enabled 4.7. This commit also enables max on Sonnet 4.6. Changes: * litellm/llms/anthropic/chat/transformation.py - swap substring guard for `_supports_effort_level(model, 'max')` * model_prices_and_context_window.json (+ backup) - add `supports_max_reasoning_effort: true` to the 10 Opus 4.7 entries (anthropic / bedrock_converse / azure_ai / vertex_ai) - add `supports_max_reasoning_effort: true` to the 9 Sonnet 4.6 entries (anthropic / bedrock_converse / azure_ai / vertex_ai) * litellm/types/utils.py + litellm/utils.py - add `supports_max_reasoning_effort` to ModelInfoBase / ModelInfo so `get_model_info()` surfaces the new flag (mirrors the existing xhigh handling) * tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py - flip `test_max_effort_rejected_for_sonnet_46` to `test_max_effort_accepted_for_sonnet_46` - update opus-4.5 rejection test to match new error message - point the opus-4.6 acceptance test at a SKU that exists in the model map (claude-opus-4-6-20260205) [1] https://platform.claude.com/docs/en/build-with-claude/effort "The effort parameter is supported by ... Claude Opus 4.7, Claude Opus 4.6, and Claude Sonnet 4.6."
fa54191 to
cb38100
Compare
|
Rebased onto latest Rebase summary:
Re: the previous Re: the PR is otherwise green: @krrishdholakia @ishaan-jaff — could one of you take a look when you have a moment? The bug is small (just lets |
…t tests CI fetches model_prices_and_context_window.json from `main` at runtime, so the new `supports_max_reasoning_effort` flags added in this PR are not yet visible to the live CI run. Mirror the established pattern from `tests/litellm_utils_tests/test_utils.py::test_supports_reasoning`: set LITELLM_LOCAL_MODEL_COST_MAP=True and reload `litellm.model_cost` inside each new accept-test, so they read the model-map version that is in this same PR. Affected tests: * test_max_effort_accepted_for_opus_46 * test_max_effort_accepted_for_opus_47 * test_max_effort_accepted_for_sonnet_46 Locally verified: 137 / 137 pass in test_anthropic_chat_transformation.py
cb38100 to
1625adc
Compare
|
Any updates on this PR? |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
Title
fix(anthropic): allow
effort="max"on Claude Opus 4.7Relevant issues
Fixes #25957
Pre-Submission Checklist (Click to expand)
make test-unitType
🐛 Bug Fix
Changes
AnthropicConfig._apply_output_configpreviously rejectedeffort="max"for any model that wasn't Claude Opus 4.6:This guard was added in #22234 (back when 4.6 was the only model accepting
max) and was intentionally kept in the day-0 4.7 PR (#25867). However, Anthropic's Messages API acceptsoutput_config.effort="max"onclaude-opus-4-7-*as well — calling the API directly withmodel=claude-opus-4-7-*,output_config={"effort":"max"}, andthinking={"type":"adaptive"}returns a normal 200 completion.So LiteLLM was the only thing in the chain rejecting a perfectly valid configuration, breaking flows like Claude Code via
ANTHROPIC_BASE_URLand any direct SDK call against Opus 4.7 withreasoning_effort="max".This PR allows
effort="max"on both Opus 4.6 and Opus 4.7._is_opus_4_7_modelalready exists onAnthropicConfig(added in #25867), so the diff is minimal:I kept the substring-style check (rather than switching to a model-map
supports_max_reasoning_effortlookup) so date-variant model names (e.g.claude-opus-4-7-20260408) keep working without requiring every dated entry to be present in the cost map. Switching to a data-driven lookup would be a reasonable follow-up but is out of scope for this fix.Tests
test_max_effort_rejected_for_opus_45andtest_max_effort_rejected_for_sonnet_46to match the new error message.test_max_effort_accepted_for_opus_47(regression for [Bug]: effort="max" hardcoded as Opus 4.6 only — Opus 4.7 rejected even though Anthropic API accepts it #25957).test_max_effort_accepted_for_opus_46still passes unchanged.