fix(vertex): stop stripping output_config and output_format from VertexAI Claude requests by netbrah · Pull Request #23475 · BerriAI/litellm

netbrah · 2026-03-12T20:04:06Z

Summary

VertexAI Claude now supports output_config (effort control) and output_format (structured JSON outputs) — these parameters were being silently stripped, causing structured output requests to return unstructured text and effort constraints to be ignored.

Root cause: Two explicit .pop() calls in the VertexAI Anthropic transformation code that were added when Vertex didn't support these params. Vertex has since added support (confirmed via Anthropic docs and Google Cloud docs).

Fix: Remove the .pop("output_config") and .pop("output_format") calls in both:

Messages API path (experimental_pass_through/transformation.py)
Chat Completions path (transformation.py)

Changes

litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py — remove output_config and output_format stripping
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/transformation.py — same
Updated tests from "verify dropped" to "verify preserved"

Test plan

test_vertex_ai_anthropic_output_config_preserved — verifies output_config with effort passes through
test_vertex_ai_anthropic_output_format_and_output_config_both_preserved — verifies both params pass through
test_vertex_ai_claude_sonnet_4_5_structured_output_fix — existing test still passes
test_vertex_ai_anthropic_structured_output_header_not_added — existing test still passes
All 4 related tests pass, no regressions

Made with Cursor

vercel · 2026-03-12T20:04:12Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 19, 2026 9:32pm

CLAassistant · 2026-03-12T20:04:16Z

All committers have signed the CLA.

greptile-apps · 2026-03-12T20:06:40Z

Greptile Summary

This PR fixes a silent parameter-stripping bug in the VertexAI Claude integration: two .pop() calls that discarded output_config (effort control) and output_format (structured JSON outputs) before sending requests to Vertex AI are removed, since Vertex AI now supports both parameters. The OpenAI-compatible path is unaffected — VertexAIAnthropicConfig.map_openai_params still forces a tool-based structured-output approach for response_format inputs, so output_format is never added to optional_params in that code path and the Vertex AI request remains clean.

Key changes:

transformation.py (Chat Completions path): removes data.pop("output_format") and data.pop("output_config")
experimental_pass_through/transformation.py (Messages API path): same removals
Updated tests to assert parameters are preserved rather than dropped; two minor test-quality issues noted (outdated docstring, trivially-true model assertion after model was removed from mock data)
All four related tests pass per the PR description

Confidence Score: 4/5

Safe to merge; the change is minimal, targeted, and backed by passing mock tests.
The core change (removing two .pop() calls in each of two files) is exactly right given Vertex AI's new support for these parameters. The OpenAI-compat path is correctly insulated via the map_openai_params override. One point is withheld because the test for test_vertex_ai_claude_sonnet_4_5_structured_output_fix now has an outdated docstring and the companion test_vertex_ai_anthropic_output_format_and_output_config_both_preserved test has a weakened model-stripping assertion after model was removed from the mock data — both are minor quality issues, not correctness problems.
No files require special attention; the two transformation files are straightforward deletions of obsolete .pop() calls.

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/transformation.py	Removes `.pop("output_config")` and `.pop("output_format")` stripping from `transform_request`; the `map_openai_params` override still forces tool-based structured outputs for the OpenAI-compat path, so the OpenAI → Vertex flow is unaffected.
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py	Removes the same two `.pop()` calls from the Messages API pass-through path; only the `model` field is still stripped, which is correct per Vertex AI's API contract.
tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py	Tests correctly flipped from "verify dropped" to "verify preserved"; one assertion in `test_vertex_ai_anthropic_output_format_and_output_config_both_preserved` becomes trivially true because `model` is no longer injected into `test_data` before the mock, mildly weakening coverage of the `model`-stripping behaviour.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant VertexAIAnthropicConfig
    participant AnthropicConfig (parent)
    participant VertexAI

    Note over Caller,VertexAI: OpenAI-compat path (response_format)
    Caller->>VertexAIAnthropicConfig: map_openai_params(response_format=...)
    VertexAIAnthropicConfig->>AnthropicConfig (parent): map_openai_params(model="claude-3-sonnet-20240229")
    AnthropicConfig (parent)-->>VertexAIAnthropicConfig: optional_params with tools+tool_choice (tool-based, no output_format)
    VertexAIAnthropicConfig-->>Caller: optional_params (tools, no output_format)
    Caller->>VertexAIAnthropicConfig: transform_request(optional_params)
    VertexAIAnthropicConfig->>AnthropicConfig (parent): super().transform_request()
    AnthropicConfig (parent)-->>VertexAIAnthropicConfig: data (no output_format since not in optional_params)
    VertexAIAnthropicConfig->>VertexAIAnthropicConfig: data.pop("model")
    VertexAIAnthropicConfig-->>VertexAI: request body (tools-based, no output_format)

    Note over Caller,VertexAI: Native Anthropic API path (pass-through)
    Caller->>VertexAIAnthropicConfig: transform_request(optional_params with output_config/output_format)
    VertexAIAnthropicConfig->>AnthropicConfig (parent): super().transform_request()
    AnthropicConfig (parent)-->>VertexAIAnthropicConfig: data (includes output_config + output_format)
    VertexAIAnthropicConfig->>VertexAIAnthropicConfig: data.pop("model")
    Note over VertexAIAnthropicConfig: output_config/output_format NO LONGER stripped (this PR)
    VertexAIAnthropicConfig-->>VertexAI: request body (includes output_config + output_format)

_{Last reviewed commit: "fix(vertex): stop st..."}

…exAI Claude requests VertexAI Claude now supports output_config (effort control) and output_format (structured JSON outputs) as of early 2026. The previous behavior silently dropped these parameters, causing structured output requests to return unstructured text. Removes the .pop() calls in both the Messages API path (experimental_pass_through/transformation.py) and the Chat Completions path (transformation.py). Updates tests to verify the parameters are preserved. Fixes BerriAI#23380 Made-with: Cursor

codspeed-hq · 2026-03-19T21:32:42Z

Merging this PR will not alter performance

✅ 16 untouched benchmarks

_{Comparing netbrah:fix/vertex-anthropic-output-config (93d04b2) with main (2df9655)}

…ept it Resolves the silent strip of Anthropic Structured Outputs across the Vertex AI Claude transformation paths and the Anthropic-adapter re-merge. Consolidates and supersedes four stalled community PRs addressing overlapping aspects of the same root bug: - #23475 (Vertex AI Claude blanket-strip removal) - #23396 (Vertex AI Claude conditional passthrough) - #23706 (Anthropic adapter exclude output_config from non-Anthropic backends) - #22727 (Anthropic adapter strip output_config for non-Anthropic backends) Closes / addresses: #23380 (Vertex AI Claude output_config drop), related: #26423, #25079, #24549, #25971, #25957, #26163, #24856. What was broken --------------- * Vertex AI Claude paths called ``data.pop("output_config")`` and ``data.pop("output_format")`` unconditionally even when Vertex accepted those fields. Callers asking for Structured Outputs got a 200 with prose and never knew the schema constraints had been silently dropped (often masked for months by permissive fallback parsers). * The ``/v1/messages`` -> ``/chat/completions`` adapter (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the raw Anthropic-shaped ``output_config`` into ``completion_kwargs`` AFTER the translator already mapped its meaningful parts to ``response_format`` / ``reasoning_effort``. Non-Anthropic backends (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with "Extra inputs are not permitted". Approach -------- Vertex AI Claude (chat-completion + experimental_pass_through paths): Replace the unconditional pop with a sanitizer ``_sanitize_vertex_anthropic_output_params`` that strips only the Vertex-unsupported keys (today: ``effort``) from ``output_config`` while forwarding ``format`` and the legacy top-level ``output_format``. Defensive: non-dict ``output_config`` values are dropped to avoid sending malformed payloads downstream. Greptile P1 from PR #23396 addressed: when ``output_config`` carries both ``format`` and ``effort``, the prior conditional pass-through forwarded ``effort`` and reproduced the 400. The new helper filters per-key. Anthropic ``/v1/messages`` adapter: Add ``output_config`` to a named module-level constant ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so the post-translation re-merge skips re-adding the raw key. This fixes the 400 on non-Anthropic backends and avoids the conflicting duplicate (``response_format`` + raw ``output_config``) on Anthropic-family backends. Greptile P2 from PR #23706 addressed: the constant gives reviewers one grep target instead of an inline literal that silently grows. Greptile P2 from PR #22727 addressed: ``extra_kwargs or {}`` is replaced with explicit ``is None`` checks so empty-dict callers no longer skip the fallback path. Tests ----- * tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/ test_vertex_ai_partner_models_anthropic_transformation.py: - 5 new/updated cases plus a direct unit test for ``_sanitize_vertex_anthropic_output_params``. - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix`` so its mock-injected ``output_format`` is asserted to FLOW THROUGH (the original test asserted the now-buggy strip behavior). * tests/test_litellm/llms/anthropic/experimental_pass_through/ adapters/test_handler_output_config_passthrough.py (new): - Constant export sanity, output_config strip with ``effort`` only, output_config strip with ``format`` only, regression guard that unrelated extras still flow, explicit-empty-dict path, and the ``extra_kwargs=None`` no-crash path. Test-quality fixes incorporated from Greptile review on the superseded PRs: * No ``inspect.getsource`` source-text assertions (PR #24114 / #23475). * ``sys.path`` insertion is anchored to ``__file__`` (PR #23706). * Assertion messages are positional, not tuple (PR #24114-class bug). * No ``or {}`` masking explicit empty dicts in helper signatures (PR #22727). Verified locally: 26/26 pass with this commit. The new tests fail (or fail to import) on ``main`` without it. Out of scope ------------ * The ``max_tokens`` capping logic from PR #22727 — independent concern, deserves its own PR with a focused test plan. * Architectural rework of the ``excluded_keys`` mechanism (Greptile P2 on PR #23706 noted point-fix growth). The named constant gives maintainers a clear place to extend; a registry-based approach would be a follow-up. Co-Authored-By: netbrah <netbrah> Co-Authored-By: s-zx <s-zx> Co-Authored-By: invoicepulse <invoicepulse> Co-Authored-By: cfdude <cfdude> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…h-consolidated fix(adapters,vertex): pass output_config through to backends that accept it (closes #23380, supersedes #23475/#23396/#23706/#22727)

…ept it Resolves the silent strip of Anthropic Structured Outputs across the Vertex AI Claude transformation paths and the Anthropic-adapter re-merge. Consolidates and supersedes four stalled community PRs addressing overlapping aspects of the same root bug: - BerriAI#23475 (Vertex AI Claude blanket-strip removal) - BerriAI#23396 (Vertex AI Claude conditional passthrough) - BerriAI#23706 (Anthropic adapter exclude output_config from non-Anthropic backends) - BerriAI#22727 (Anthropic adapter strip output_config for non-Anthropic backends) Closes / addresses: BerriAI#23380 (Vertex AI Claude output_config drop), related: BerriAI#26423, BerriAI#25079, BerriAI#24549, BerriAI#25971, BerriAI#25957, BerriAI#26163, BerriAI#24856. What was broken --------------- * Vertex AI Claude paths called ``data.pop("output_config")`` and ``data.pop("output_format")`` unconditionally even when Vertex accepted those fields. Callers asking for Structured Outputs got a 200 with prose and never knew the schema constraints had been silently dropped (often masked for months by permissive fallback parsers). * The ``/v1/messages`` -> ``/chat/completions`` adapter (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the raw Anthropic-shaped ``output_config`` into ``completion_kwargs`` AFTER the translator already mapped its meaningful parts to ``response_format`` / ``reasoning_effort``. Non-Anthropic backends (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with "Extra inputs are not permitted". Approach -------- Vertex AI Claude (chat-completion + experimental_pass_through paths): Replace the unconditional pop with a sanitizer ``_sanitize_vertex_anthropic_output_params`` that strips only the Vertex-unsupported keys (today: ``effort``) from ``output_config`` while forwarding ``format`` and the legacy top-level ``output_format``. Defensive: non-dict ``output_config`` values are dropped to avoid sending malformed payloads downstream. Greptile P1 from PR BerriAI#23396 addressed: when ``output_config`` carries both ``format`` and ``effort``, the prior conditional pass-through forwarded ``effort`` and reproduced the 400. The new helper filters per-key. Anthropic ``/v1/messages`` adapter: Add ``output_config`` to a named module-level constant ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so the post-translation re-merge skips re-adding the raw key. This fixes the 400 on non-Anthropic backends and avoids the conflicting duplicate (``response_format`` + raw ``output_config``) on Anthropic-family backends. Greptile P2 from PR BerriAI#23706 addressed: the constant gives reviewers one grep target instead of an inline literal that silently grows. Greptile P2 from PR BerriAI#22727 addressed: ``extra_kwargs or {}`` is replaced with explicit ``is None`` checks so empty-dict callers no longer skip the fallback path. Tests ----- * tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/ test_vertex_ai_partner_models_anthropic_transformation.py: - 5 new/updated cases plus a direct unit test for ``_sanitize_vertex_anthropic_output_params``. - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix`` so its mock-injected ``output_format`` is asserted to FLOW THROUGH (the original test asserted the now-buggy strip behavior). * tests/test_litellm/llms/anthropic/experimental_pass_through/ adapters/test_handler_output_config_passthrough.py (new): - Constant export sanity, output_config strip with ``effort`` only, output_config strip with ``format`` only, regression guard that unrelated extras still flow, explicit-empty-dict path, and the ``extra_kwargs=None`` no-crash path. Test-quality fixes incorporated from Greptile review on the superseded PRs: * No ``inspect.getsource`` source-text assertions (PR BerriAI#24114 / BerriAI#23475). * ``sys.path`` insertion is anchored to ``__file__`` (PR BerriAI#23706). * Assertion messages are positional, not tuple (PR BerriAI#24114-class bug). * No ``or {}`` masking explicit empty dicts in helper signatures (PR BerriAI#22727). Verified locally: 26/26 pass with this commit. The new tests fail (or fail to import) on ``main`` without it. Out of scope ------------ * The ``max_tokens`` capping logic from PR BerriAI#22727 — independent concern, deserves its own PR with a focused test plan. * Architectural rework of the ``excluded_keys`` mechanism (Greptile P2 on PR BerriAI#23706 noted point-fix growth). The named constant gives maintainers a clear place to extend; a registry-based approach would be a follow-up. Co-Authored-By: netbrah <netbrah> Co-Authored-By: s-zx <s-zx> Co-Authored-By: invoicepulse <invoicepulse> Co-Authored-By: cfdude <cfdude> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

github-actions · 2026-06-18T00:28:21Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

…ept it Resolves the silent strip of Anthropic Structured Outputs across the Vertex AI Claude transformation paths and the Anthropic-adapter re-merge. Consolidates and supersedes four stalled community PRs addressing overlapping aspects of the same root bug: - BerriAI#23475 (Vertex AI Claude blanket-strip removal) - BerriAI#23396 (Vertex AI Claude conditional passthrough) - BerriAI#23706 (Anthropic adapter exclude output_config from non-Anthropic backends) - BerriAI#22727 (Anthropic adapter strip output_config for non-Anthropic backends) Closes / addresses: BerriAI#23380 (Vertex AI Claude output_config drop), related: BerriAI#26423, BerriAI#25079, BerriAI#24549, BerriAI#25971, BerriAI#25957, BerriAI#26163, BerriAI#24856. What was broken --------------- * Vertex AI Claude paths called ``data.pop("output_config")`` and ``data.pop("output_format")`` unconditionally even when Vertex accepted those fields. Callers asking for Structured Outputs got a 200 with prose and never knew the schema constraints had been silently dropped (often masked for months by permissive fallback parsers). * The ``/v1/messages`` -> ``/chat/completions`` adapter (``LiteLLMMessagesToCompletionTransformationHandler``) re-merged the raw Anthropic-shaped ``output_config`` into ``completion_kwargs`` AFTER the translator already mapped its meaningful parts to ``response_format`` / ``reasoning_effort``. Non-Anthropic backends (Azure OpenAI, Fireworks, Bedrock Nova, etc.) then 400'd with "Extra inputs are not permitted". Approach -------- Vertex AI Claude (chat-completion + experimental_pass_through paths): Replace the unconditional pop with a sanitizer ``_sanitize_vertex_anthropic_output_params`` that strips only the Vertex-unsupported keys (today: ``effort``) from ``output_config`` while forwarding ``format`` and the legacy top-level ``output_format``. Defensive: non-dict ``output_config`` values are dropped to avoid sending malformed payloads downstream. Greptile P1 from PR BerriAI#23396 addressed: when ``output_config`` carries both ``format`` and ``effort``, the prior conditional pass-through forwarded ``effort`` and reproduced the 400. The new helper filters per-key. Anthropic ``/v1/messages`` adapter: Add ``output_config`` to a named module-level constant ``ANTHROPIC_ONLY_REQUEST_KEYS`` and wire it into ``excluded_keys`` so the post-translation re-merge skips re-adding the raw key. This fixes the 400 on non-Anthropic backends and avoids the conflicting duplicate (``response_format`` + raw ``output_config``) on Anthropic-family backends. Greptile P2 from PR BerriAI#23706 addressed: the constant gives reviewers one grep target instead of an inline literal that silently grows. Greptile P2 from PR BerriAI#22727 addressed: ``extra_kwargs or {}`` is replaced with explicit ``is None`` checks so empty-dict callers no longer skip the fallback path. Tests ----- * tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/ test_vertex_ai_partner_models_anthropic_transformation.py: - 5 new/updated cases plus a direct unit test for ``_sanitize_vertex_anthropic_output_params``. - Updated ``test_vertex_ai_claude_sonnet_4_5_structured_output_fix`` so its mock-injected ``output_format`` is asserted to FLOW THROUGH (the original test asserted the now-buggy strip behavior). * tests/test_litellm/llms/anthropic/experimental_pass_through/ adapters/test_handler_output_config_passthrough.py (new): - Constant export sanity, output_config strip with ``effort`` only, output_config strip with ``format`` only, regression guard that unrelated extras still flow, explicit-empty-dict path, and the ``extra_kwargs=None`` no-crash path. Test-quality fixes incorporated from Greptile review on the superseded PRs: * No ``inspect.getsource`` source-text assertions (PR BerriAI#24114 / BerriAI#23475). * ``sys.path`` insertion is anchored to ``__file__`` (PR BerriAI#23706). * Assertion messages are positional, not tuple (PR BerriAI#24114-class bug). * No ``or {}`` masking explicit empty dicts in helper signatures (PR BerriAI#22727). Verified locally: 26/26 pass with this commit. The new tests fail (or fail to import) on ``main`` without it. Out of scope ------------ * The ``max_tokens`` capping logic from PR BerriAI#22727 — independent concern, deserves its own PR with a focused test plan. * Architectural rework of the ``excluded_keys`` mechanism (Greptile P2 on PR BerriAI#23706 noted point-fix growth). The named constant gives maintainers a clear place to extend; a registry-based approach would be a follow-up. Co-Authored-By: netbrah <netbrah> Co-Authored-By: s-zx <s-zx> Co-Authored-By: invoicepulse <invoicepulse> Co-Authored-By: cfdude <cfdude>

…sthrough-consolidated fix(adapters,vertex): pass output_config through to backends that accept it (closes BerriAI#23380, supersedes BerriAI#23475/BerriAI#23396/BerriAI#23706/BerriAI#22727)

vercel Bot deployed to Preview March 12, 2026 20:05 View deployment

greptile-apps Bot reviewed Mar 12, 2026

View reviewed changes

Comment thread ...vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py Outdated

netbrah force-pushed the fix/vertex-anthropic-output-config branch 2 times, most recently from f06e8d9 to 703145a Compare March 12, 2026 20:41

vercel Bot deployed to Preview March 12, 2026 20:43 View deployment

netbrah force-pushed the fix/vertex-anthropic-output-config branch from 703145a to 93d04b2 Compare March 19, 2026 21:30

vercel Bot deployed to Preview March 19, 2026 21:32 View deployment

dkindlund mentioned this pull request Apr 24, 2026

fix(adapters,vertex): pass output_config through to backends that accept it (closes #23380, supersedes #23475/#23396/#23706/#22727) #26439

Merged

4 tasks

mateo-berri mentioned this pull request Apr 25, 2026

chore(staging): roll oss_staging_04_25_2026 into internal staging (output_config fix + 4 upstream sync fixes) #26530

Merged

4 tasks

github-actions Bot added the stale label Jun 18, 2026

github-actions Bot closed this Jun 26, 2026

Navigation Menu

Uh oh!

Uh oh!

fix(vertex): stop stripping output_config and output_format from VertexAI Claude requests - #23475

fix(vertex): stop stripping output_config and output_format from VertexAI Claude requests#23475
netbrah wants to merge 1 commit into
BerriAI:mainfrom
netbrah:fix/vertex-anthropic-output-config

netbrah commented Mar 12, 2026

Uh oh!

vercel Bot commented Mar 12, 2026 •

edited

Loading

Uh oh!

CLAassistant commented Mar 12, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Mar 12, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

codspeed-hq Bot commented Mar 19, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

netbrah commented Mar 12, 2026

Summary

Changes

Test plan

Uh oh!

vercel Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

codspeed-hq Bot commented Mar 19, 2026

Merging this PR will not alter performance

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel Bot commented Mar 12, 2026 •

edited

Loading

CLAassistant commented Mar 12, 2026 •

edited

Loading

greptile-apps Bot commented Mar 12, 2026 •

edited

Loading