fix(anthropic-adapter): exclude output_config from extra_kwargs in Anthropic adapter - #23706
Conversation
…ic-to-OpenAI adapter When Claude Code (or any Anthropic Messages API client) sends requests through the LiteLLM proxy to a non-Anthropic provider (e.g., Azure OpenAI, Fireworks), the Anthropic adapter translates the request to OpenAI format. The output_config parameter (Anthropic-specific, used for extended thinking/effort configuration) was being passed through via extra_kwargs to non-Anthropic providers, which reject it with: 400: Extra inputs are not permitted, field: output_config Note: PR BerriAI#22990 addresses this in the Azure transformation layer, but requests routed through the Anthropic adapter path (POST /v1/messages) bypass that fix entirely. This change addresses the adapter-specific code path. Added output_config to excluded_keys set in _prepare_completion_kwargs() to properly strip it during Anthropic-to-OpenAI translation. Related: BerriAI#22963
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR fixes a bug where Anthropic-specific Key changes and observations:
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/experimental_pass_through/adapters/handler.py | Single-line fix adding "output_config" to excluded_keys set; prevents Anthropic-specific parameter from being forwarded to non-Anthropic providers. The fix stops the 400 error but silently drops reasoning effort semantics without any translation to target-provider equivalents. |
| tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_output_config_excluded.py | Three unit tests validating output_config exclusion, anthropic_messages exclusion, and valid extra_kwargs passthrough. Tests are pure unit tests with no network calls, consistent with repo conventions. Uses fragile relative sys.path.insert anchored to CWD rather than file. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["POST /v1/messages\n(Anthropic Messages API)"] --> B{Is target\nAnthropic?}
B -- Yes --> C["AnthropicConfig.transform_request()\ngpt_transformation.py\noutput_config handled natively"]
B -- No --> D["LiteLLMMessagesToCompletionTransformationHandler\n_prepare_completion_kwargs()"]
D --> E["Iterate extra_kwargs\n(includes output_config from Claude Code)"]
E --> F{key in\nexcluded_keys?}
F -- Yes\n'output_config'\n'anthropic_messages' --> G["❌ Skip / Drop parameter\n(was causing 400 errors\nbefore this PR)"]
F -- No --> H["✅ Add to completion_kwargs"]
H --> I["litellm.acompletion(**completion_kwargs)\nForwarded to Azure / Fireworks / etc."]
G --> I
Last reviewed commit: a60c7a5
| } | ||
|
|
||
| excluded_keys = {"anthropic_messages"} | ||
| excluded_keys = {"anthropic_messages", "output_config"} |
There was a problem hiding this comment.
Hardcoded exclusion may not scale as Anthropic adds new parameters
output_config is now hardcoded as a single entry in excluded_keys. The broader problem is that any future Anthropic-specific parameter that ends up in extra_kwargs will silently be forwarded to non-Anthropic providers until another one-off exclusion is added here.
More critically, output_config carries semantically meaningful information about reasoning effort. When the adapter silently drops it, the downstream model receives no instruction about reasoning effort — which may result in incorrect or degraded output quality without any error being raised.
A more robust approach would be to also translate known Anthropic-specific parameters (like output_config) into their target-provider equivalents (e.g., reasoning_effort) rather than unconditionally dropping them. However, if the intent here is purely to prevent 400 errors rather than preserve semantics, this fix is consistent with what was done in gpt_transformation.py for PR #22990.
At minimum, consider whether output_config should be handled as a named parameter in _prepare_completion_kwargs (like thinking, tool_choice, etc.) and translated to a provider-agnostic form, rather than being silently discarded through the exclusion list.
|
|
||
| import pytest | ||
|
|
||
| sys.path.insert(0, os.path.abspath("../../../../..")) |
There was a problem hiding this comment.
Fragile relative path in sys.path.insert
os.path.abspath("../../../../..") resolves relative to the current working directory at the time the test is run, not relative to the file itself. This is consistent with the existing test file in this directory (test_anthropic_experimental_pass_through_adapters_transformation.py), but the pattern is fragile — it silently resolves to the wrong path if the test is invoked from a directory other than the file's own directory.
The safer approach is to anchor the path to the file's location:
| sys.path.insert(0, os.path.abspath("../../../../..")) | |
| sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../../.."))) |
…ept it Resolves the silent strip of Anthropic Structured Outputs across the Vertex AI Claude transformation paths and the Anthropic-adapter re-merge. Consolidates and supersedes four stalled community PRs addressing overlapping aspects of the same root bug: - #23475 (Vertex AI Claude blanket-strip removal) - #23396 (Vertex AI Claude conditional passthrough) - #23706 (Anthropic adapter exclude output_config from non-Anthropic backends) - #22727 (Anthropic adapter strip output_config for non-Anthropic backends) Closes / addresses: #23380 (Vertex AI Claude output_config drop), related: #26423, #25079, #24549, #25971, #25957, #26163, #24856. What was broken --------------- * Vertex AI Claude paths called ``data.pop("output_config")`` and ``data.pop("output_format")`` unconditionally even when Vertex accepted those fields. Callers asking for Structured Outputs got a 200 with prose and never knew the schema constraints had been silently dropped (often masked for months by permissive fallback parsers). * The ``/v1/messages`` -> ``/chat/completions`` adapter (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the raw Anthropic-shaped ``output_config`` into ``completion_kwargs`` AFTER the translator already mapped its meaningful parts to ``response_format`` / ``reasoning_effort``. Non-Anthropic backends (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with "Extra inputs are not permitted". Approach -------- Vertex AI Claude (chat-completion + experimental_pass_through paths): Replace the unconditional pop with a sanitizer ``_sanitize_vertex_anthropic_output_params`` that strips only the Vertex-unsupported keys (today: ``effort``) from ``output_config`` while forwarding ``format`` and the legacy top-level ``output_format``. Defensive: non-dict ``output_config`` values are dropped to avoid sending malformed payloads downstream. Greptile P1 from PR #23396 addressed: when ``output_config`` carries both ``format`` and ``effort``, the prior conditional pass-through forwarded ``effort`` and reproduced the 400. The new helper filters per-key. Anthropic ``/v1/messages`` adapter: Add ``output_config`` to a named module-level constant ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so the post-translation re-merge skips re-adding the raw key. This fixes the 400 on non-Anthropic backends and avoids the conflicting duplicate (``response_format`` + raw ``output_config``) on Anthropic-family backends. Greptile P2 from PR #23706 addressed: the constant gives reviewers one grep target instead of an inline literal that silently grows. Greptile P2 from PR #22727 addressed: ``extra_kwargs or {}`` is replaced with explicit ``is None`` checks so empty-dict callers no longer skip the fallback path. Tests ----- * tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/ test_vertex_ai_partner_models_anthropic_transformation.py: - 5 new/updated cases plus a direct unit test for ``_sanitize_vertex_anthropic_output_params``. - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix`` so its mock-injected ``output_format`` is asserted to FLOW THROUGH (the original test asserted the now-buggy strip behavior). * tests/test_litellm/llms/anthropic/experimental_pass_through/ adapters/test_handler_output_config_passthrough.py (new): - Constant export sanity, output_config strip with ``effort`` only, output_config strip with ``format`` only, regression guard that unrelated extras still flow, explicit-empty-dict path, and the ``extra_kwargs=None`` no-crash path. Test-quality fixes incorporated from Greptile review on the superseded PRs: * No ``inspect.getsource`` source-text assertions (PR #24114 / #23475). * ``sys.path`` insertion is anchored to ``__file__`` (PR #23706). * Assertion messages are positional, not tuple (PR #24114-class bug). * No ``or {}`` masking explicit empty dicts in helper signatures (PR #22727). Verified locally: 26/26 pass with this commit. The new tests fail (or fail to import) on ``main`` without it. Out of scope ------------ * The ``max_tokens`` capping logic from PR #22727 — independent concern, deserves its own PR with a focused test plan. * Architectural rework of the ``excluded_keys`` mechanism (Greptile P2 on PR #23706 noted point-fix growth). The named constant gives maintainers a clear place to extend; a registry-based approach would be a follow-up. Co-Authored-By: netbrah <netbrah> Co-Authored-By: s-zx <s-zx> Co-Authored-By: invoicepulse <invoicepulse> Co-Authored-By: cfdude <cfdude> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ept it Resolves the silent strip of Anthropic Structured Outputs across the Vertex AI Claude transformation paths and the Anthropic-adapter re-merge. Consolidates and supersedes four stalled community PRs addressing overlapping aspects of the same root bug: - BerriAI#23475 (Vertex AI Claude blanket-strip removal) - BerriAI#23396 (Vertex AI Claude conditional passthrough) - BerriAI#23706 (Anthropic adapter exclude output_config from non-Anthropic backends) - BerriAI#22727 (Anthropic adapter strip output_config for non-Anthropic backends) Closes / addresses: BerriAI#23380 (Vertex AI Claude output_config drop), related: BerriAI#26423, BerriAI#25079, BerriAI#24549, BerriAI#25971, BerriAI#25957, BerriAI#26163, BerriAI#24856. What was broken --------------- * Vertex AI Claude paths called ``data.pop("output_config")`` and ``data.pop("output_format")`` unconditionally even when Vertex accepted those fields. Callers asking for Structured Outputs got a 200 with prose and never knew the schema constraints had been silently dropped (often masked for months by permissive fallback parsers). * The ``/v1/messages`` -> ``/chat/completions`` adapter (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the raw Anthropic-shaped ``output_config`` into ``completion_kwargs`` AFTER the translator already mapped its meaningful parts to ``response_format`` / ``reasoning_effort``. Non-Anthropic backends (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with "Extra inputs are not permitted". Approach -------- Vertex AI Claude (chat-completion + experimental_pass_through paths): Replace the unconditional pop with a sanitizer ``_sanitize_vertex_anthropic_output_params`` that strips only the Vertex-unsupported keys (today: ``effort``) from ``output_config`` while forwarding ``format`` and the legacy top-level ``output_format``. Defensive: non-dict ``output_config`` values are dropped to avoid sending malformed payloads downstream. Greptile P1 from PR BerriAI#23396 addressed: when ``output_config`` carries both ``format`` and ``effort``, the prior conditional pass-through forwarded ``effort`` and reproduced the 400. The new helper filters per-key. Anthropic ``/v1/messages`` adapter: Add ``output_config`` to a named module-level constant ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so the post-translation re-merge skips re-adding the raw key. This fixes the 400 on non-Anthropic backends and avoids the conflicting duplicate (``response_format`` + raw ``output_config``) on Anthropic-family backends. Greptile P2 from PR BerriAI#23706 addressed: the constant gives reviewers one grep target instead of an inline literal that silently grows. Greptile P2 from PR BerriAI#22727 addressed: ``extra_kwargs or {}`` is replaced with explicit ``is None`` checks so empty-dict callers no longer skip the fallback path. Tests ----- * tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/ test_vertex_ai_partner_models_anthropic_transformation.py: - 5 new/updated cases plus a direct unit test for ``_sanitize_vertex_anthropic_output_params``. - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix`` so its mock-injected ``output_format`` is asserted to FLOW THROUGH (the original test asserted the now-buggy strip behavior). * tests/test_litellm/llms/anthropic/experimental_pass_through/ adapters/test_handler_output_config_passthrough.py (new): - Constant export sanity, output_config strip with ``effort`` only, output_config strip with ``format`` only, regression guard that unrelated extras still flow, explicit-empty-dict path, and the ``extra_kwargs=None`` no-crash path. Test-quality fixes incorporated from Greptile review on the superseded PRs: * No ``inspect.getsource`` source-text assertions (PR BerriAI#24114 / BerriAI#23475). * ``sys.path`` insertion is anchored to ``__file__`` (PR BerriAI#23706). * Assertion messages are positional, not tuple (PR BerriAI#24114-class bug). * No ``or {}`` masking explicit empty dicts in helper signatures (PR BerriAI#22727). Verified locally: 26/26 pass with this commit. The new tests fail (or fail to import) on ``main`` without it. Out of scope ------------ * The ``max_tokens`` capping logic from PR BerriAI#22727 — independent concern, deserves its own PR with a focused test plan. * Architectural rework of the ``excluded_keys`` mechanism (Greptile P2 on PR BerriAI#23706 noted point-fix growth). The named constant gives maintainers a clear place to extend; a registry-based approach would be a follow-up. Co-Authored-By: netbrah <netbrah> Co-Authored-By: s-zx <s-zx> Co-Authored-By: invoicepulse <invoicepulse> Co-Authored-By: cfdude <cfdude> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
FIx for #22963 was merged. Closing this |
…ept it Resolves the silent strip of Anthropic Structured Outputs across the Vertex AI Claude transformation paths and the Anthropic-adapter re-merge. Consolidates and supersedes four stalled community PRs addressing overlapping aspects of the same root bug: - BerriAI#23475 (Vertex AI Claude blanket-strip removal) - BerriAI#23396 (Vertex AI Claude conditional passthrough) - BerriAI#23706 (Anthropic adapter exclude output_config from non-Anthropic backends) - BerriAI#22727 (Anthropic adapter strip output_config for non-Anthropic backends) Closes / addresses: BerriAI#23380 (Vertex AI Claude output_config drop), related: BerriAI#26423, BerriAI#25079, BerriAI#24549, BerriAI#25971, BerriAI#25957, BerriAI#26163, BerriAI#24856. What was broken --------------- * Vertex AI Claude paths called ``data.pop("output_config")`` and ``data.pop("output_format")`` unconditionally even when Vertex accepted those fields. Callers asking for Structured Outputs got a 200 with prose and never knew the schema constraints had been silently dropped (often masked for months by permissive fallback parsers). * The ``/v1/messages`` -> ``/chat/completions`` adapter (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the raw Anthropic-shaped ``output_config`` into ``completion_kwargs`` AFTER the translator already mapped its meaningful parts to ``response_format`` / ``reasoning_effort``. Non-Anthropic backends (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with "Extra inputs are not permitted". Approach -------- Vertex AI Claude (chat-completion + experimental_pass_through paths): Replace the unconditional pop with a sanitizer ``_sanitize_vertex_anthropic_output_params`` that strips only the Vertex-unsupported keys (today: ``effort``) from ``output_config`` while forwarding ``format`` and the legacy top-level ``output_format``. Defensive: non-dict ``output_config`` values are dropped to avoid sending malformed payloads downstream. Greptile P1 from PR BerriAI#23396 addressed: when ``output_config`` carries both ``format`` and ``effort``, the prior conditional pass-through forwarded ``effort`` and reproduced the 400. The new helper filters per-key. Anthropic ``/v1/messages`` adapter: Add ``output_config`` to a named module-level constant ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so the post-translation re-merge skips re-adding the raw key. This fixes the 400 on non-Anthropic backends and avoids the conflicting duplicate (``response_format`` + raw ``output_config``) on Anthropic-family backends. Greptile P2 from PR BerriAI#23706 addressed: the constant gives reviewers one grep target instead of an inline literal that silently grows. Greptile P2 from PR BerriAI#22727 addressed: ``extra_kwargs or {}`` is replaced with explicit ``is None`` checks so empty-dict callers no longer skip the fallback path. Tests ----- * tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/ test_vertex_ai_partner_models_anthropic_transformation.py: - 5 new/updated cases plus a direct unit test for ``_sanitize_vertex_anthropic_output_params``. - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix`` so its mock-injected ``output_format`` is asserted to FLOW THROUGH (the original test asserted the now-buggy strip behavior). * tests/test_litellm/llms/anthropic/experimental_pass_through/ adapters/test_handler_output_config_passthrough.py (new): - Constant export sanity, output_config strip with ``effort`` only, output_config strip with ``format`` only, regression guard that unrelated extras still flow, explicit-empty-dict path, and the ``extra_kwargs=None`` no-crash path. Test-quality fixes incorporated from Greptile review on the superseded PRs: * No ``inspect.getsource`` source-text assertions (PR BerriAI#24114 / BerriAI#23475). * ``sys.path`` insertion is anchored to ``__file__`` (PR BerriAI#23706). * Assertion messages are positional, not tuple (PR BerriAI#24114-class bug). * No ``or {}`` masking explicit empty dicts in helper signatures (PR BerriAI#22727). Verified locally: 26/26 pass with this commit. The new tests fail (or fail to import) on ``main`` without it. Out of scope ------------ * The ``max_tokens`` capping logic from PR BerriAI#22727 — independent concern, deserves its own PR with a focused test plan. * Architectural rework of the ``excluded_keys`` mechanism (Greptile P2 on PR BerriAI#23706 noted point-fix growth). The named constant gives maintainers a clear place to extend; a registry-based approach would be a follow-up. Co-Authored-By: netbrah <netbrah> Co-Authored-By: s-zx <s-zx> Co-Authored-By: invoicepulse <invoicepulse> Co-Authored-By: cfdude <cfdude>
…sthrough-consolidated fix(adapters,vertex): pass output_config through to backends that accept it (closes BerriAI#23380, supersedes BerriAI#23475/BerriAI#23396/BerriAI#23706/BerriAI#22727)
Relevant issues
Related to #22963
Pre-Submission checklist
tests/test_litellm/directory@greptileaiType
🐛 Bug Fix
Changes
I ran into this bug while setting up Claude Code CLI with LiteLLM proxy routing to an Azure-hosted Fireworks model (GLM-5). Requests kept failing with:
400: Extra inputs are not permitted, field: 'output_config'PR #22990 fixes this in the Azure transformation layer (
gpt_transformation.py), but I found that requests coming through the Anthropic adapter path (POST /v1/messages→adapters/handler.py) bypass that fix entirely.In
_prepare_completion_kwargs(), theexcluded_keysset only contains"anthropic_messages". All other Anthropic-specific params (like output_config) pass through extra_kwargs to non-Anthropic providers, which reject them.Fix: Added
"output_config"toexcluded_keysinadapters/handler.pyline 171.Files changed:
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py— added output_config toexcluded_keystests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_output_config_excluded.py— 3 unit testsTest plan