fix(adapters,vertex): pass output_config through to backends that accept it (closes #23380, supersedes #23475/#23396/#23706/#22727)

[dkindlund]

Summary

Pass output_config through to backends that accept it (Vertex AI Claude, Anthropic-direct) and stop forwarding the raw Anthropic-shaped key into OpenAI-format completion calls where it triggers 400 "Extra inputs are not permitted." Consolidates four stalled community PRs that addressed overlapping aspects of the same root bug, incorporates every Greptile review concern that was outstanding on those PRs, and closes one functional gap they all missed (format + effort mixed payload on Vertex).

Closes / supersedes

ID	Title	Status
Closes #23380	[Bug]: Dropped output_config parameter in Messages API prevents schema and effort constraints	OPEN
Supersedes #23475	fix(vertex): stop stripping output_config and output_format from VertexAI Claude requests	OPEN, stalled since Mar 19
Supersedes #23396	fix: pass output_config for VertexAI Claude structured output	OPEN, stalled since Mar 25
Supersedes #23706	fix(anthropic-adapter): exclude output_config from extra_kwargs in Anthropic adapter	OPEN, stalled since Mar 16
Supersedes #22727	fix(anthropic-adapter): strip output_config for non-Anthropic backends	OPEN, stalled since Mar 4

Original authors credited via Co-Authored-By trailers: @netbrah, @s-zx, @invoicepulse, @cfdude.

Related issues (not closed by this PR but reference the same cluster)

#26423, #25079, #24549, #25971, #25957, #26163, #24856

What was broken

                ┌──────────────────────────────────────────────────────┐
                │ output_config is the Anthropic Messages API field    │
                │ that controls structured outputs (`.format`) and     │
                │ reasoning effort (`.effort`). LiteLLM was silently   │
                │ stripping it on multiple paths, hiding Structured    │
                │ Outputs from callers who explicitly requested them.  │
                └────────────────────────┬─────────────────────────────┘
                                         │
        ┌────────────────────────────────┼────────────────────────────────────┐
        ▼                                ▼                                    ▼
 Vertex AI Claude                Vertex AI Claude               Anthropic /v1/messages
 (Chat Completions path)         (Messages path)                → /chat/completions adapter
        │                                │                                    │
        │  unconditional pop()           │  unconditional pop()               │  raw output_config
        │  of output_config              │  of output_config                  │  re-merged into
        │  AND output_format             │  AND output_format                 │  completion_kwargs
        ▼                                ▼                                    │  AFTER translator
 200 with prose,                 200 with prose,                              │  already mapped its
 schema silently                 schema silently                              │  meaningful parts
 dropped                         dropped                                      ▼
                                                                       400 "Extra inputs
                                                                       are not permitted"
                                                                       on Azure / Fireworks /
                                                                       Bedrock Nova

Anthropic accepts output_config and validates the schema. The bug reporter's empirical confirmation:

# Same payload to Anthropic-direct → 400 invalid_request_error (validates the field is being received and parsed)
# Same payload through LiteLLM proxy → 200 with prose (silent strip)

Approach

Vertex AI Claude (both transformation paths)

Replace data.pop("output_config", None) with a sanitizer that strips only the Vertex-unsupported keys (today: effort) from output_config, forwarding format and the legacy top-level output_format. Helper lives in the chat-completion module and is imported by the Messages module so there's one source of truth.

# Old (silent strip):
data.pop("output_config", None)
data.pop("output_format", None)

# New (per-key sanitization):
_sanitize_vertex_anthropic_output_params(data)
# - output_config containing only ``effort`` → dropped (Vertex 400s on it)
# - output_config containing ``format`` → forwarded
# - output_config containing both → ``effort`` filtered, ``format`` kept
# - output_format → forwarded as-is
# - non-dict output_config → dropped defensively

The _VERTEX_UNSUPPORTED_OUTPUT_CONFIG_KEYS constant gives maintainers one place to extend as Vertex parity drifts.

Anthropic adapter (/v1/messages → /chat/completions)

Add output_config to a named module-level constant ANTHROPIC_ONLY_REQUEST_KEYS and wire it into excluded_keys. The translator above the re-merge has already mapped output_config.format → response_format and output_config.effort → reasoning_effort for non-Claude targets. Re-adding the raw key was either redundant (Anthropic-family target sees both response_format AND a duplicate output_config) or harmful (non-Anthropic target rejects unknown field).

ANTHROPIC_ONLY_REQUEST_KEYS: frozenset[str] = frozenset({"output_config"})
# ...
excluded_keys = ANTHROPIC_ONLY_REQUEST_KEYS | {"anthropic_messages"}

Also replaces extra_kwargs or {} with extra_kwargs if extra_kwargs is not None else {} so callers passing an explicit empty dict don't get a default substituted (Greptile P2 on PR #22727 — the masked-empty-dict bug that hid the fallback inference path from being tested).

Greptile feedback addressed (every concern from the four superseded PRs)

Source PR	Greptile concern	How this PR addresses it
#23475	"Misleading comment on output_format pass-through" — said the field flowed through OpenAI-compat when actually only via mock injection	The Sonnet 4.5 test now asserts output_format flows through the mock path explicitly and notes that map_openai_params still uses tools for the OpenAI-compat path
#23396 (P1)	"effort key leaks through when format is also present" — combined payload would still 400 on Vertex	_sanitize_vertex_anthropic_output_params filters per-key, so format + effort becomes format only. Dedicated test: test_vertex_ai_anthropic_output_config_format_plus_effort_strips_only_effort
#23396	"Existing tests now fail after this change" (the two *_dropped tests asserted the old strip behavior)	Both old assertions removed/rewritten; new tests assert the corrected per-key behavior
#23706	"Hardcoded exclusion may not scale"	Replaced inline literal with named module-level constant ANTHROPIC_ONLY_REQUEST_KEYS + comment instructing future maintainers where to extend
#23706	"Fragile sys.path.insert working-directory-relative path"	New test file uses os.path.join(os.path.dirname(__file__), ...) anchored to file location
#22727 (P0)	"Pre-setting async_complete_streaming_response triggers unconditional early return"	Not applicable here — this PR doesn't touch streaming logging. The author addressed it in their final commit on #22727
#22727 (P1)	"Mock hides regression — test mocks async_success_handler"	This PR's tests don't mock the function under test. Adapter tests drive _prepare_completion_kwargs directly and inspect its return value
#22727 (P2)	"Missing Vertex AI Claude detection" — _is_anthropic_claude only checked anthropic/bedrock	Approach changed: instead of conditional strip based on target provider detection, the strip is unconditional because the translator has already extracted the meaningful parts. No detection logic needed; correctness no longer depends on getting Vertex AI Claude detection right
#22727 (P2)	"Missing tests for output_config stripping"	New file test_handler_output_config_passthrough.py covers strip with effort-only, strip with format-only, regression guard for unrelated extras, empty-extra-kwargs, None-extra-kwargs
#22727 (P2)	"Helper masks empty dict argument" — extra_kwargs or {default} substitutes default for {}	Replaced with if extra_kwargs is not None else {} and dedicated test test_explicit_empty_dict_does_not_substitute_default
(PR #24114, related class of bug from another cluster)	"inspect.getsource source-text tests are fragile"	All new tests are runtime behavior — no source-text inspection
(PR #24114, related)	"Custom assertion messages are dead code (tuple comma)"	All new assertions use positional message argument

Tests

tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py
  test_vertex_ai_anthropic_output_config_effort_only_dropped                                    PASSED
  test_vertex_ai_anthropic_output_config_format_passes_through                                  PASSED
  test_vertex_ai_anthropic_output_config_format_plus_effort_strips_only_effort                  PASSED
  test_vertex_ai_anthropic_output_config_non_dict_dropped                                       PASSED
  test_vertex_ai_anthropic_output_format_preserved_output_config_effort_dropped                 PASSED
  test_sanitize_vertex_anthropic_output_params_unit                                             PASSED
  test_vertex_ai_claude_sonnet_4_5_structured_output_fix                                        PASSED  (updated to assert flow-through, not strip)
  + 12 unrelated existing tests PASSED (regression guard)

tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_handler_output_config_passthrough.py (new file)
  TestAnthropicOnlyRequestKeysExport::test_constant_exposed                                     PASSED
  TestAnthropicOnlyRequestKeysExport::test_contains_output_config                               PASSED
  TestEmptyExtraKwargsPath::test_explicit_empty_dict_does_not_substitute_default                PASSED
  TestEmptyExtraKwargsPath::test_none_extra_kwargs_handled_safely                               PASSED
  TestOutputConfigStrippedFromCompletionKwargs::test_other_extra_kwargs_still_passed_through    PASSED
  TestOutputConfigStrippedFromCompletionKwargs::test_output_config_with_effort_is_stripped      PASSED
  TestOutputConfigStrippedFromCompletionKwargs::test_output_config_with_format_is_stripped_format_already_translated  PASSED

26/26 pass with this commit. The new tests fail (or fail to import) on main without it.

Manual reproduction

# Before the fix
curl -X POST "https://<litellm-proxy>/v1/messages" \
  -H "Authorization: Bearer <litellm-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex_ai/claude-sonnet-4@20250514",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Return a person object."}],
    "output_config": {"format": {"type": "json_schema", "schema": {"type": "object", "additionalProperties": false, "properties": {"name": {"type": "string"}, "age": {"type": "integer"}}}}}
  }'
# → 200 with prose, schema silently dropped
# → /spend/logs shows no indication of the strip
# → client believes Structured Outputs is in effect

# After the fix
# → Vertex receives the schema
# → Validates per-Anthropic semantics
# → Returns structured JSON OR a clear 400 explaining the schema problem

Out of scope (deliberately deferred)

• max_tokens capping from PR fix(anthropic-adapter): strip output_config for non-Anthropic backends #22727 — independent concern. Should land as its own PR with focused tests for Bedrock Nova / strict-limit providers.
• Architectural rework of excluded_keys — Greptile flagged the point-fix-growth pattern. Named constant addresses the maintainability concern; a registry-based approach can come as a follow-up.
• Anthropic-direct /v1/messages strip investigation — the bug report mentions a strip on Anthropic-direct that I could not reproduce via code reading. The transformation path explicitly forwards output_config. If users still see the strip on Anthropic-direct after this PR lands, it's a separate downstream issue worth a fresh bug report.

Test plan

• make test-unit for the touched test files
• Verify tests fail on main without this commit
• black --check clean
• CI green (will check after push)

🤖 Generated with Claude Code

[veria-ai]

This PR adds parameter filtering and translation logic for Anthropic's output_config field when routing requests through the adapter to non-Anthropic backends (Vertex AI, Azure OpenAI, etc.). The changes are confined to request parameter sanitization — filtering known-unsupported keys, translating between API formats, and adding type checks. No injection surfaces, auth changes, secrets handling, or other security-relevant patterns.

---

Status: 0 open
Risk: 1/10

Posted by Veria AI · 2026-04-24T16:05:06.898Z

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing dkindlund:fix/output-config-passthrough-consolidated (79517bc) with main (7b47dff)}

[greptile-apps]

Greptile Summary

This PR fixes silent stripping of output_config on two Vertex AI Claude transformation paths (chat-completion and Messages pass-through) and on the Anthropic-to-OpenAI adapter, consolidating four stalled community PRs. The core fix replaces unconditional data.pop(\"output_config\") calls with a shared sanitize_vertex_anthropic_output_params helper that strips only Vertex-unsupported sub-keys (currently effort) while forwarding supported ones (format); on the adapter path a new ANTHROPIC_ONLY_REQUEST_KEYS constant prevents the raw Anthropic-shaped field from leaking into completion_kwargs after translation.

Confidence Score: 5/5

Safe to merge; fixes a confirmed silent-drop bug with correct per-key sanitization, good test coverage, and only P2 findings

No P0 or P1 issues found. The two P2 findings (None/absent key ambiguity in the sanitizer, and falsy-vs-None check in the translator) have no realistic impact on production inputs. The PR includes 26 passing tests, reversed test assertions are intentional and correctly reflect the fixed behavior, and the shared helper avoids the duplicate-logic problem of the superseded PRs.

output_params_utils.py (None vs absent key edge case) and transformation.py adapter (falsy output_format check semantics)

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/output_params_utils.py	New shared sanitization helper for Vertex AI Claude; correctly filters unsupported sub-keys (effort) while preserving supported ones (format); minor edge case: .get() cannot distinguish absent key from key set to None, leaving null values in the dict
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py	Adds ANTHROPIC_ONLY_REQUEST_KEYS constant and wires output_config into excluded_keys to prevent the raw Anthropic field from leaking into completion_kwargs; None-check fix for extra_kwargs is correct
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py	Extends _translate_output_format_to_openai to accept output_config.format as a fallback when output_format is absent; uses falsy check (if not output_format:) which silently treats empty-dict output_format the same as absent
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/transformation.py	Replaces unconditional pop() of output_config/output_format with call to sanitize_vertex_anthropic_output_params; clean delegation to shared helper
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py	Mirrors the chat-completion path change — replaces unconditional pop() with sanitize_vertex_anthropic_output_params; one source of truth for both transformation paths
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_handler_output_config_passthrough.py	New test file covering all key scenarios: strip, translate, precedence, empty/None extra_kwargs, and constant export; behavioral tests rather than source inspection; no real network calls
tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py	Existing tests updated to reflect corrected behavior (output_format now forwarded, effort dropped); assertions reversed from prior strip-everything behavior — valid since the stripping was the bug; new tests added for per-key sanitization scenarios

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Client sends output_config] --> B{Which path?}
    B --> C[Vertex AI Claude\nchat-completion path]
    B --> D[Vertex AI Claude\nMessages pass-through path]
    B --> E[Anthropic adapter\n/v1/messages → /chat/completions]
    C --> F[sanitize_vertex_anthropic_output_params]
    D --> F
    F --> G{output_config sub-keys}
    G -->|effort only| H[Drop output_config entirely\navoid Vertex 400]
    G -->|format only| I[Forward output_config as-is\nVertex accepts format]
    G -->|format + effort| J[Strip effort, keep format\nForward sanitized output_config]
    G -->|non-dict| K[Drop defensively]
    E --> L[Extract output_config\ninto request_data]
    L --> M[translate_anthropic_to_openai\nmaps format→response_format\neffort→reasoning_effort]
    M --> N[excluded_keys filter\nANTHROPIC_ONLY_REQUEST_KEYS\nprevents raw output_config\nre-merge into completion_kwargs]
    N --> O[completion_kwargs has\nresponse_format / reasoning_effort\nbut NOT raw output_config]

Loading

_{Reviews (2): Last reviewed commit: "fix: address Greptile review feedback on..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 84.84043% with 57 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...y/management_endpoints/mcp_management_endpoints.py	80.95%	12 Missing ⚠️
...opic/experimental_pass_through/adapters/handler.py	52.17%	11 Missing ⚠️
litellm/proxy/auth/auth_checks.py	75.00%	7 Missing ⚠️
...rtex_ai/image_edit/vertex_imagen_transformation.py	58.33%	5 Missing ⚠️
.../llms/dashscope/image_generation/transformation.py	93.10%	4 Missing ⚠️
...llm/litellm_core_utils/prompt_templates/factory.py	25.00%	3 Missing ⚠️
...ass_through_endpoints/llm_passthrough_endpoints.py	62.50%	3 Missing ⚠️
...lms/black_forest_labs/image_edit/transformation.py	50.00%	2 Missing ⚠️
litellm/llms/vertex_ai/common_utils.py	71.42%	2 Missing ⚠️
...y/_experimental/mcp_server/semantic_tool_filter.py	94.11%	2 Missing ⚠️
... and 6 more

📢 Thoughts on this report? Let us know!

[dkindlund]

@greptile-apps, can you please re-review?

[krrish-berri-2]

@dkindlund — could you add a screenshot or short video showing that this change works as expected? It really helps reviewers verify the fix quickly. Thanks!

[krrish-berri-2]

@mateo-berri didn't we recently work on this?

is this still a relevant PR?

[mateo-berri]

We merged a PR for Bedrock but not Vertex. This is a legit value add. I can take over reviewing

[krrish-berri-2]

@mateo-berri assigned to you

[mateo-berri]

LGTM. Thank you for the contribution!