fix(anthropic-adapter): strip output_config for non-Anthropic backends

[cfdude]

Summary

• output_config is an Anthropic Claude-specific parameter with no equivalent in other model providers
• When routing via the experimental pass-through adapter to non-Anthropic backends (e.g. Amazon Nova Pro on Bedrock, Llama, Mistral), the parameter was forwarded verbatim causing HTTP 400 extraneous key [output_config] is not permitted errors
• Strip output_config from the request when the target provider is not an Anthropic Claude model (direct Anthropic API or Bedrock-hosted Anthropic Claude)
• thinking is intentionally not stripped — some non-Anthropic models (e.g. Qwen) support reasoning natively and the adapter already handles translation for those

Test plan

• Tested locally: Claude Code routing to bedrock/converse/us.amazon.nova-pro-v1:0 via LiteLLM proxy — previously failed with extraneous key [output_config], now succeeds
• output_config passthrough to native Anthropic Claude models is unaffected
• Unit tests to be added covering strip/no-strip behavior per provider

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 4, 2026 1:38am

[greptile-apps]

Greptile Summary

This PR adds two defensive features to the Anthropic experimental pass-through adapter: (1) capping max_tokens against the target model's known output token limit to prevent HTTP 400 errors from providers with stricter limits, and (2) stripping the Anthropic-specific output_config parameter when routing to non-Anthropic backends. Both changes use litellm.get_model_info() for model lookups, which aligns well with the project's convention of deriving capabilities from the model registry.

• max_tokens capping (lines 124-154): Gracefully caps max_tokens using model info, with a broad except Exception: pass fallback so unknown models aren't blocked.
• output_config stripping (lines 198-216): Strips Anthropic-only params for non-Anthropic targets. The _is_anthropic_claude detection covers direct Anthropic API and Bedrock-hosted Claude; Vertex AI coverage gap was flagged in a prior review thread.
• Test file: Good coverage for both features, but the _call helper has a Python falsy-dict bug that prevents the provider-inference fallback test from actually exercising the intended code path.

Confidence Score: 4/5

• This PR is safe to merge — it fixes a real HTTP 400 regression for non-Anthropic backends with minimal blast radius
• The handler changes are well-scoped and defensively coded (graceful fallback on errors). The main concern is a test bug where the provider-inference fallback path isn't actually exercised, but this doesn't affect production behavior.
• The test file needs the _call helper fix to properly validate the fallback inference path

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py	Adds max_tokens capping against model limits and strips output_config for non-Anthropic backends. Logic is sound for covered providers; Vertex AI gap already flagged in prior review.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_handler.py	New unit tests for max_tokens capping and output_config stripping. The _call helper has a falsy-dict bug that prevents the provider-inference fallback test from exercising the intended code path.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["_prepare_completion_kwargs()"] --> B["Cap max_tokens via get_model_info()"]
    B --> C["Translate request to OpenAI format"]
    C --> D{"Is target Anthropic Claude?"}
    D -->|"provider == 'anthropic'"| E["Keep output_config"]
    D -->|"provider == 'bedrock' && model contains 'anthropic.claude'"| E
    D -->|"Non-Anthropic backend"| F["Strip output_config from excluded_keys"]
    E --> G["Merge extra_kwargs into completion_kwargs"]
    F --> G
    G --> H["Route to litellm.completion()"]

Loading

_{Last reviewed commit: 3647cac}

[greptile-apps]

Missing Vertex AI Claude detection

The _is_anthropic_claude check only covers the "anthropic" and "bedrock" providers, but Vertex AI also hosts Anthropic Claude models (e.g. vertex_ai/claude-sonnet-4). For those, custom_llm_provider would be "vertex_ai" and the model string would contain "claude" but not "anthropic.claude". This means output_config would be incorrectly stripped for Vertex AI Claude models.

There is already an existing helper LiteLLMAnthropicMessagesAdapter.is_anthropic_claude_model() in the adjacent transformation.py (line 653) that correctly handles all three providers by checking for "anthropic" or "claude" in the model string. Consider reusing that helper, or extending this check to cover Vertex AI:

Suggested change

	_anthropic_only_params = {"output_config"}
	_target_provider = (extra_kwargs or {}).get("custom_llm_provider", "")
	_is_anthropic_claude = _target_provider in (
	"anthropic",
	) or (
	_target_provider == "bedrock"
	and "anthropic.claude" in completion_kwargs.get("model", "")
	)
	_anthropic_only_params = {"output_config"}
	_target_provider = (extra_kwargs or {}).get("custom_llm_provider", "")
	_is_anthropic_claude = _target_provider in (
	"anthropic",
	) or (
	_target_provider == "bedrock"
	and "anthropic.claude" in completion_kwargs.get("model", "")
	) or (
	_target_provider == "vertex_ai"
	and "claude" in completion_kwargs.get("model", "").lower()
	)

[greptile-apps]

Missing tests for output_config stripping

The PR title and description focus on stripping output_config for non-Anthropic backends, but the test file only covers the max_tokens capping logic. The PR description itself notes "Unit tests to be added covering strip/no-strip behavior per provider" as unchecked. Please add tests verifying:

1. output_config is stripped when routing to non-Anthropic backends (e.g. Bedrock Nova Pro)
2. output_config is preserved when routing to Anthropic Claude (direct API)
3. output_config is preserved when routing to Bedrock-hosted Anthropic Claude
4. output_config is preserved when routing to Vertex AI Claude

Context Used: Rule from dashboard - What: Ensure that any PR claiming to fix an issue includes evidence that the issue is resolved, such... (source)

[greptile-apps]

Helper masks empty dict argument

extra_kwargs or {default} treats {} as falsy in Python, so passing extra_kwargs={} silently substitutes the default dict that includes the provider. This means test_fallback_infers_provider_when_not_in_extra_kwargs (line 62) never exercises the fallback inference path — get_llm_provider is never invoked because the provider key is always present. The test passes only because get_model_info is mocked to return the capped value regardless of how the provider was resolved.

Use an explicit None check instead:

Suggested change

	extra_kwargs=extra_kwargs or {"custom_llm_provider": PROVIDER},
	extra_kwargs=extra_kwargs if extra_kwargs is not None else {"custom_llm_provider": PROVIDER},

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.