Skip to content

fix(anthropic-adapter): exclude output_config from extra_kwargs in Anthropic adapter - #23706

Closed
invoicepulse wants to merge 1 commit into
BerriAI:mainfrom
invoicepulse:fix/anthropic-adapter-output-config-excluded
Closed

fix(anthropic-adapter): exclude output_config from extra_kwargs in Anthropic adapter#23706
invoicepulse wants to merge 1 commit into
BerriAI:mainfrom
invoicepulse:fix/anthropic-adapter-output-config-excluded

Conversation

@invoicepulse

@invoicepulse invoicepulse commented Mar 16, 2026

Copy link
Copy Markdown

Relevant issues

Related to #22963

Pre-Submission checklist

  • I have Added testing in the tests/test_litellm/ directory
  • My PR passes all unit tests
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai

Type

🐛 Bug Fix

Changes

I ran into this bug while setting up Claude Code CLI with LiteLLM proxy routing to an Azure-hosted Fireworks model (GLM-5). Requests kept failing with:

400: Extra inputs are not permitted, field: 'output_config'

PR #22990 fixes this in the Azure transformation layer (gpt_transformation.py), but I found that requests coming through the Anthropic adapter path (POST /v1/messagesadapters/handler.py) bypass that fix entirely.

In _prepare_completion_kwargs(), the excluded_keys set only contains "anthropic_messages". All other Anthropic-specific params (like output_config) pass through extra_kwargs to non-Anthropic providers, which reject them.

Fix: Added "output_config" to excluded_keys in adapters/handler.py line 171.

Files changed:

  • litellm/llms/anthropic/experimental_pass_through/adapters/handler.py — added output_config to excluded_keys
  • tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_output_config_excluded.py — 3 unit tests

Test plan

  • 3 unit tests pass — verifying output_config exclusion, existing anthropic_messages exclusion preserved, and valid kwargs still pass through

…ic-to-OpenAI adapter

When Claude Code (or any Anthropic Messages API client) sends requests
through the LiteLLM proxy to a non-Anthropic provider (e.g., Azure OpenAI,
Fireworks), the Anthropic adapter translates the request to OpenAI format.

The output_config parameter (Anthropic-specific, used for extended
thinking/effort configuration) was being passed through via extra_kwargs
to non-Anthropic providers, which reject it with:
  400: Extra inputs are not permitted, field: output_config

Note: PR BerriAI#22990 addresses this in the Azure transformation layer, but
requests routed through the Anthropic adapter path (POST /v1/messages)
bypass that fix entirely. This change addresses the adapter-specific
code path.

Added output_config to excluded_keys set in _prepare_completion_kwargs()
to properly strip it during Anthropic-to-OpenAI translation.

Related: BerriAI#22963
@vercel

vercel Bot commented Mar 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Mar 16, 2026 2:00am

Request Review

@invoicepulse

Copy link
Copy Markdown
Author

@greptileai

@codspeed-hq

codspeed-hq Bot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing invoicepulse:fix/anthropic-adapter-output-config-excluded (a60c7a5) with main (548e7eb)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a bug where Anthropic-specific output_config parameter was leaking through the adapter path (POST /v1/messagesadapters/handler.py) to non-Anthropic providers (e.g., Azure OpenAI, Fireworks), causing 400: Extra inputs are not permitted errors. The fix adds "output_config" to the excluded_keys set in _prepare_completion_kwargs, mirroring the fix previously applied to the Azure gpt_transformation.py path in PR #22990.

Key changes and observations:

  • handler.py: One-line fix adding "output_config" to the excluded_keys set alongside the existing "anthropic_messages" exclusion
  • test_output_config_excluded.py: Three new unit tests that are correctly isolated (no network calls), covering the exclusion, backward-compatibility of anthropic_messages, and valid extra kwargs passthrough
  • The fix prevents the 400 error, but output_config (which encodes reasoning effort) is silently dropped rather than being translated to an equivalent parameter for the target provider — the behavior is consistent with how anthropic_messages is treated, but worth noting that reasoning effort intent is lost
  • The excluded_keys set is growing as a point-fix pattern; future Anthropic-specific parameters introduced in extra_kwargs will need to be added manually one by one

Confidence Score: 4/5

  • Safe to merge — the change is minimal, well-tested, and stops a real 400 error without affecting the Anthropic-native path.
  • The one-line fix is correct and well-scoped. The tests are purely in-process unit tests with no network calls, consistent with the repository's testing policy. The only concern is that output_config is silently dropped rather than translated, which may silently degrade reasoning quality for users who set it — but this is consistent with the existing treatment of anthropic_messages and is an acceptable tradeoff for a bug-fix PR. No backwards-incompatible changes are introduced.
  • No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py Single-line fix adding "output_config" to excluded_keys set; prevents Anthropic-specific parameter from being forwarded to non-Anthropic providers. The fix stops the 400 error but silently drops reasoning effort semantics without any translation to target-provider equivalents.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_output_config_excluded.py Three unit tests validating output_config exclusion, anthropic_messages exclusion, and valid extra_kwargs passthrough. Tests are pure unit tests with no network calls, consistent with repo conventions. Uses fragile relative sys.path.insert anchored to CWD rather than file.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["POST /v1/messages\n(Anthropic Messages API)"] --> B{Is target\nAnthropic?}
    B -- Yes --> C["AnthropicConfig.transform_request()\ngpt_transformation.py\noutput_config handled natively"]
    B -- No --> D["LiteLLMMessagesToCompletionTransformationHandler\n_prepare_completion_kwargs()"]
    D --> E["Iterate extra_kwargs\n(includes output_config from Claude Code)"]
    E --> F{key in\nexcluded_keys?}
    F -- Yes\n'output_config'\n'anthropic_messages' --> G["❌ Skip / Drop parameter\n(was causing 400 errors\nbefore this PR)"]
    F -- No --> H["✅ Add to completion_kwargs"]
    H --> I["litellm.acompletion(**completion_kwargs)\nForwarded to Azure / Fireworks / etc."]
    G --> I
Loading

Last reviewed commit: a60c7a5

}

excluded_keys = {"anthropic_messages"}
excluded_keys = {"anthropic_messages", "output_config"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded exclusion may not scale as Anthropic adds new parameters

output_config is now hardcoded as a single entry in excluded_keys. The broader problem is that any future Anthropic-specific parameter that ends up in extra_kwargs will silently be forwarded to non-Anthropic providers until another one-off exclusion is added here.

More critically, output_config carries semantically meaningful information about reasoning effort. When the adapter silently drops it, the downstream model receives no instruction about reasoning effort — which may result in incorrect or degraded output quality without any error being raised.

A more robust approach would be to also translate known Anthropic-specific parameters (like output_config) into their target-provider equivalents (e.g., reasoning_effort) rather than unconditionally dropping them. However, if the intent here is purely to prevent 400 errors rather than preserve semantics, this fix is consistent with what was done in gpt_transformation.py for PR #22990.

At minimum, consider whether output_config should be handled as a named parameter in _prepare_completion_kwargs (like thinking, tool_choice, etc.) and translated to a provider-agnostic form, rather than being silently discarded through the exclusion list.


import pytest

sys.path.insert(0, os.path.abspath("../../../../.."))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fragile relative path in sys.path.insert

os.path.abspath("../../../../..") resolves relative to the current working directory at the time the test is run, not relative to the file itself. This is consistent with the existing test file in this directory (test_anthropic_experimental_pass_through_adapters_transformation.py), but the pattern is fragile — it silently resolves to the wrong path if the test is invoked from a directory other than the file's own directory.

The safer approach is to anchor the path to the file's location:

Suggested change
sys.path.insert(0, os.path.abspath("../../../../.."))
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../../..")))

mateo-berri pushed a commit that referenced this pull request Apr 25, 2026
…ept it

Resolves the silent strip of Anthropic Structured Outputs across the
Vertex AI Claude transformation paths and the Anthropic-adapter
re-merge. Consolidates and supersedes four stalled community PRs
addressing overlapping aspects of the same root bug:

- #23475 (Vertex AI Claude blanket-strip removal)
- #23396 (Vertex AI Claude conditional passthrough)
- #23706 (Anthropic adapter exclude output_config from non-Anthropic
  backends)
- #22727 (Anthropic adapter strip output_config for non-Anthropic
  backends)

Closes / addresses: #23380 (Vertex AI Claude output_config drop),
related: #26423, #25079, #24549, #25971, #25957, #26163, #24856.

What was broken
---------------
* Vertex AI Claude paths called ``data.pop("output_config")`` and
  ``data.pop("output_format")`` unconditionally even when Vertex
  accepted those fields. Callers asking for Structured Outputs got a
  200 with prose and never knew the schema constraints had been
  silently dropped (often masked for months by permissive fallback
  parsers).
* The ``/v1/messages`` -> ``/chat/completions`` adapter
  (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the
  raw Anthropic-shaped ``output_config`` into ``completion_kwargs``
  AFTER the translator already mapped its meaningful parts to
  ``response_format`` / ``reasoning_effort``. Non-Anthropic backends
  (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with
  "Extra inputs are not permitted".

Approach
--------
Vertex AI Claude (chat-completion + experimental_pass_through paths):
  Replace the unconditional pop with a sanitizer
  ``_sanitize_vertex_anthropic_output_params`` that strips only the
  Vertex-unsupported keys (today: ``effort``) from ``output_config``
  while forwarding ``format`` and the legacy top-level
  ``output_format``. Defensive: non-dict ``output_config`` values are
  dropped to avoid sending malformed payloads downstream.
  Greptile P1 from PR #23396 addressed: when ``output_config`` carries
  both ``format`` and ``effort``, the prior conditional pass-through
  forwarded ``effort`` and reproduced the 400. The new helper filters
  per-key.

Anthropic ``/v1/messages`` adapter:
  Add ``output_config`` to a named module-level constant
  ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so
  the post-translation re-merge skips re-adding the raw key. This
  fixes the 400 on non-Anthropic backends and avoids the conflicting
  duplicate (``response_format`` + raw ``output_config``) on
  Anthropic-family backends.
  Greptile P2 from PR #23706 addressed: the constant gives reviewers
  one grep target instead of an inline literal that silently grows.
  Greptile P2 from PR #22727 addressed: ``extra_kwargs or {}`` is
  replaced with explicit ``is None`` checks so empty-dict callers no
  longer skip the fallback path.

Tests
-----
* tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/
  test_vertex_ai_partner_models_anthropic_transformation.py:
  - 5 new/updated cases plus a direct unit test for
    ``_sanitize_vertex_anthropic_output_params``.
  - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix``
    so its mock-injected ``output_format`` is asserted to FLOW THROUGH
    (the original test asserted the now-buggy strip behavior).
* tests/test_litellm/llms/anthropic/experimental_pass_through/
  adapters/test_handler_output_config_passthrough.py (new):
  - Constant export sanity, output_config strip with ``effort`` only,
    output_config strip with ``format`` only, regression guard that
    unrelated extras still flow, explicit-empty-dict path, and the
    ``extra_kwargs=None`` no-crash path.

Test-quality fixes incorporated from Greptile review on the
superseded PRs:
* No ``inspect.getsource`` source-text assertions (PR #24114 / #23475).
* ``sys.path`` insertion is anchored to ``__file__`` (PR #23706).
* Assertion messages are positional, not tuple (PR #24114-class bug).
* No ``or {}`` masking explicit empty dicts in helper signatures
  (PR #22727).

Verified locally: 26/26 pass with this commit. The new tests
fail (or fail to import) on ``main`` without it.

Out of scope
------------
* The ``max_tokens`` capping logic from PR #22727 — independent
  concern, deserves its own PR with a focused test plan.
* Architectural rework of the ``excluded_keys`` mechanism (Greptile
  P2 on PR #23706 noted point-fix growth). The named constant gives
  maintainers a clear place to extend; a registry-based approach
  would be a follow-up.

Co-Authored-By: netbrah <netbrah>
Co-Authored-By: s-zx <s-zx>
Co-Authored-By: invoicepulse <invoicepulse>
Co-Authored-By: cfdude <cfdude>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mateo-berri added a commit that referenced this pull request Apr 25, 2026
…h-consolidated

fix(adapters,vertex): pass output_config through to backends that accept it (closes #23380, supersedes #23475/#23396/#23706/#22727)
yugborana pushed a commit to yugborana/litellm that referenced this pull request Jun 2, 2026
…ept it

Resolves the silent strip of Anthropic Structured Outputs across the
Vertex AI Claude transformation paths and the Anthropic-adapter
re-merge. Consolidates and supersedes four stalled community PRs
addressing overlapping aspects of the same root bug:

- BerriAI#23475 (Vertex AI Claude blanket-strip removal)
- BerriAI#23396 (Vertex AI Claude conditional passthrough)
- BerriAI#23706 (Anthropic adapter exclude output_config from non-Anthropic
  backends)
- BerriAI#22727 (Anthropic adapter strip output_config for non-Anthropic
  backends)

Closes / addresses: BerriAI#23380 (Vertex AI Claude output_config drop),
related: BerriAI#26423, BerriAI#25079, BerriAI#24549, BerriAI#25971, BerriAI#25957, BerriAI#26163, BerriAI#24856.

What was broken
---------------
* Vertex AI Claude paths called ``data.pop("output_config")`` and
  ``data.pop("output_format")`` unconditionally even when Vertex
  accepted those fields. Callers asking for Structured Outputs got a
  200 with prose and never knew the schema constraints had been
  silently dropped (often masked for months by permissive fallback
  parsers).
* The ``/v1/messages`` -> ``/chat/completions`` adapter
  (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the
  raw Anthropic-shaped ``output_config`` into ``completion_kwargs``
  AFTER the translator already mapped its meaningful parts to
  ``response_format`` / ``reasoning_effort``. Non-Anthropic backends
  (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with
  "Extra inputs are not permitted".

Approach
--------
Vertex AI Claude (chat-completion + experimental_pass_through paths):
  Replace the unconditional pop with a sanitizer
  ``_sanitize_vertex_anthropic_output_params`` that strips only the
  Vertex-unsupported keys (today: ``effort``) from ``output_config``
  while forwarding ``format`` and the legacy top-level
  ``output_format``. Defensive: non-dict ``output_config`` values are
  dropped to avoid sending malformed payloads downstream.
  Greptile P1 from PR BerriAI#23396 addressed: when ``output_config`` carries
  both ``format`` and ``effort``, the prior conditional pass-through
  forwarded ``effort`` and reproduced the 400. The new helper filters
  per-key.

Anthropic ``/v1/messages`` adapter:
  Add ``output_config`` to a named module-level constant
  ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so
  the post-translation re-merge skips re-adding the raw key. This
  fixes the 400 on non-Anthropic backends and avoids the conflicting
  duplicate (``response_format`` + raw ``output_config``) on
  Anthropic-family backends.
  Greptile P2 from PR BerriAI#23706 addressed: the constant gives reviewers
  one grep target instead of an inline literal that silently grows.
  Greptile P2 from PR BerriAI#22727 addressed: ``extra_kwargs or {}`` is
  replaced with explicit ``is None`` checks so empty-dict callers no
  longer skip the fallback path.

Tests
-----
* tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/
  test_vertex_ai_partner_models_anthropic_transformation.py:
  - 5 new/updated cases plus a direct unit test for
    ``_sanitize_vertex_anthropic_output_params``.
  - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix``
    so its mock-injected ``output_format`` is asserted to FLOW THROUGH
    (the original test asserted the now-buggy strip behavior).
* tests/test_litellm/llms/anthropic/experimental_pass_through/
  adapters/test_handler_output_config_passthrough.py (new):
  - Constant export sanity, output_config strip with ``effort`` only,
    output_config strip with ``format`` only, regression guard that
    unrelated extras still flow, explicit-empty-dict path, and the
    ``extra_kwargs=None`` no-crash path.

Test-quality fixes incorporated from Greptile review on the
superseded PRs:
* No ``inspect.getsource`` source-text assertions (PR BerriAI#24114 / BerriAI#23475).
* ``sys.path`` insertion is anchored to ``__file__`` (PR BerriAI#23706).
* Assertion messages are positional, not tuple (PR BerriAI#24114-class bug).
* No ``or {}`` masking explicit empty dicts in helper signatures
  (PR BerriAI#22727).

Verified locally: 26/26 pass with this commit. The new tests
fail (or fail to import) on ``main`` without it.

Out of scope
------------
* The ``max_tokens`` capping logic from PR BerriAI#22727 — independent
  concern, deserves its own PR with a focused test plan.
* Architectural rework of the ``excluded_keys`` mechanism (Greptile
  P2 on PR BerriAI#23706 noted point-fix growth). The named constant gives
  maintainers a clear place to extend; a registry-based approach
  would be a follow-up.

Co-Authored-By: netbrah <netbrah>
Co-Authored-By: s-zx <s-zx>
Co-Authored-By: invoicepulse <invoicepulse>
Co-Authored-By: cfdude <cfdude>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Sameerlite

Copy link
Copy Markdown
Contributor

FIx for #22963 was merged. Closing this

@Sameerlite Sameerlite closed this Jun 3, 2026
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…ept it

Resolves the silent strip of Anthropic Structured Outputs across the
Vertex AI Claude transformation paths and the Anthropic-adapter
re-merge. Consolidates and supersedes four stalled community PRs
addressing overlapping aspects of the same root bug:

- BerriAI#23475 (Vertex AI Claude blanket-strip removal)
- BerriAI#23396 (Vertex AI Claude conditional passthrough)
- BerriAI#23706 (Anthropic adapter exclude output_config from non-Anthropic
  backends)
- BerriAI#22727 (Anthropic adapter strip output_config for non-Anthropic
  backends)

Closes / addresses: BerriAI#23380 (Vertex AI Claude output_config drop),
related: BerriAI#26423, BerriAI#25079, BerriAI#24549, BerriAI#25971, BerriAI#25957, BerriAI#26163, BerriAI#24856.

What was broken
---------------
* Vertex AI Claude paths called ``data.pop("output_config")`` and
  ``data.pop("output_format")`` unconditionally even when Vertex
  accepted those fields. Callers asking for Structured Outputs got a
  200 with prose and never knew the schema constraints had been
  silently dropped (often masked for months by permissive fallback
  parsers).
* The ``/v1/messages`` -> ``/chat/completions`` adapter
  (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the
  raw Anthropic-shaped ``output_config`` into ``completion_kwargs``
  AFTER the translator already mapped its meaningful parts to
  ``response_format`` / ``reasoning_effort``. Non-Anthropic backends
  (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with
  "Extra inputs are not permitted".

Approach
--------
Vertex AI Claude (chat-completion + experimental_pass_through paths):
  Replace the unconditional pop with a sanitizer
  ``_sanitize_vertex_anthropic_output_params`` that strips only the
  Vertex-unsupported keys (today: ``effort``) from ``output_config``
  while forwarding ``format`` and the legacy top-level
  ``output_format``. Defensive: non-dict ``output_config`` values are
  dropped to avoid sending malformed payloads downstream.
  Greptile P1 from PR BerriAI#23396 addressed: when ``output_config`` carries
  both ``format`` and ``effort``, the prior conditional pass-through
  forwarded ``effort`` and reproduced the 400. The new helper filters
  per-key.

Anthropic ``/v1/messages`` adapter:
  Add ``output_config`` to a named module-level constant
  ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so
  the post-translation re-merge skips re-adding the raw key. This
  fixes the 400 on non-Anthropic backends and avoids the conflicting
  duplicate (``response_format`` + raw ``output_config``) on
  Anthropic-family backends.
  Greptile P2 from PR BerriAI#23706 addressed: the constant gives reviewers
  one grep target instead of an inline literal that silently grows.
  Greptile P2 from PR BerriAI#22727 addressed: ``extra_kwargs or {}`` is
  replaced with explicit ``is None`` checks so empty-dict callers no
  longer skip the fallback path.

Tests
-----
* tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/
  test_vertex_ai_partner_models_anthropic_transformation.py:
  - 5 new/updated cases plus a direct unit test for
    ``_sanitize_vertex_anthropic_output_params``.
  - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix``
    so its mock-injected ``output_format`` is asserted to FLOW THROUGH
    (the original test asserted the now-buggy strip behavior).
* tests/test_litellm/llms/anthropic/experimental_pass_through/
  adapters/test_handler_output_config_passthrough.py (new):
  - Constant export sanity, output_config strip with ``effort`` only,
    output_config strip with ``format`` only, regression guard that
    unrelated extras still flow, explicit-empty-dict path, and the
    ``extra_kwargs=None`` no-crash path.

Test-quality fixes incorporated from Greptile review on the
superseded PRs:
* No ``inspect.getsource`` source-text assertions (PR BerriAI#24114 / BerriAI#23475).
* ``sys.path`` insertion is anchored to ``__file__`` (PR BerriAI#23706).
* Assertion messages are positional, not tuple (PR BerriAI#24114-class bug).
* No ``or {}`` masking explicit empty dicts in helper signatures
  (PR BerriAI#22727).

Verified locally: 26/26 pass with this commit. The new tests
fail (or fail to import) on ``main`` without it.

Out of scope
------------
* The ``max_tokens`` capping logic from PR BerriAI#22727 — independent
  concern, deserves its own PR with a focused test plan.
* Architectural rework of the ``excluded_keys`` mechanism (Greptile
  P2 on PR BerriAI#23706 noted point-fix growth). The named constant gives
  maintainers a clear place to extend; a registry-based approach
  would be a follow-up.

Co-Authored-By: netbrah <netbrah>
Co-Authored-By: s-zx <s-zx>
Co-Authored-By: invoicepulse <invoicepulse>
Co-Authored-By: cfdude <cfdude>
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…sthrough-consolidated

fix(adapters,vertex): pass output_config through to backends that accept it (closes BerriAI#23380, supersedes BerriAI#23475/BerriAI#23396/BerriAI#23706/BerriAI#22727)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants