chore(staging): roll oss_staging_04_25_2026 into internal staging (output_config fix + 4 upstream sync fixes)

[mateo-berri]

Summary

Merge litellm-oss-staging-04-25-2026 into litellm_internal_staging. Headline change is the output_config passthrough fix (PR #26439); four unrelated upstream OSS staging fixes ride along in the same merge.

Headline change: output_config passthrough (PR #26439)

fix(adapters,vertex): pass output_config through to backends that accept it

Closes #23380. Supersedes #23475/#23396/#23706/#22727.

Two silent-drop bugs:

• Vertex AI Claude (both chat-completion and Messages paths) was unconditionally stripping output_config, hiding Anthropic Structured Outputs from callers who explicitly requested them.
• /v1/messages → /chat/completions adapter was re-merging the raw Anthropic-shaped output_config key into completion kwargs after translation already mapped its meaningful parts to response_format / reasoning_effort, causing 400 "Extra inputs are not permitted" on non-Anthropic backends (Azure, Fireworks, Bedrock Nova).

Fix introduces:

• Shared leaf-module helper sanitize_vertex_anthropic_output_params (in output_params_utils.py) that strips Vertex-unsupported keys (effort) while preserving format. Lives in its own module to avoid a CodeQL-flagged cyclic import through the parent transformation module.
• ANTHROPIC_ONLY_REQUEST_KEYS named constant gating the adapter re-merge so output_config doesn't leak past the translator.
• Adapter translator extension: _translate_output_format_to_openai now reads both top-level output_format and output_config.format with explicit precedence rules.
• Side fix: extra_kwargs or {} → extra_kwargs if extra_kwargs is not None else {} so callers passing an explicit empty dict don't get a default substituted.

Full design notes, Greptile feedback table, and reproduction steps live on PR #26439.

Sync-pulled OSS staging fixes (no design changes here, just rolling forward)

Commit	Change	Origin PR
98a9005c76	arize._set_usage_outputs handles raw OpenAI Pydantic CompletionUsage (previously assumed dict shape)	#26506
f63a6f1b26	Proxy honors LITELLM_LOG=INFO by setting verbose_logger level (was previously a no-op above WARNING)	#26401
334aedf2d4	UI Add-Model dropdown shows zai (Z.AI / Zhipu AI) provider	#26419 (re-fix of #25482)
e2d0fd9eac	Remove duplicate MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB definition (kept the 1024-default one) + Cloudflare Workers AI response_text key fallback for newer Nemotron models	#26385

These are individual upstream PRs that already passed their own review; this merge just rolls them into internal staging alongside the output_config fix.

Heads-up for reviewers

• Greptile P0 on MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB is a false alarm. The e2d0fd9eac commit removed only one of two duplicate definitions; the surviving definition is at litellm/constants.py:85 with default 1024. from litellm.constants import MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB and from litellm.caching.in_memory_cache import InMemoryCache both import cleanly. Verified locally.
• Greptile P1 on Vertex output_format is legitimate and worth confirming. PR fix(adapters,vertex): pass output_config through to backends that accept it (closes #23380, supersedes #23475/#23396/#23706/#22727) #26439 reverses an assertion in test_vertex_ai_claude_sonnet_4_5_structured_output_fix from "stripped" to "forwarded," based on the claim that Vertex AI now accepts output_format alongside tools/tool_choice. Issue [Bug]: Structured output with claude-sonnet 4.5 VertexAI endpoint #18625 is the only data point and was negative. If Vertex still 400s on the combined payload, the sanitizer needs to be extended to also strip output_format (the helper docstring says it does — the code currently doesn't). Recommend confirming against a live Vertex Claude endpoint before this merges further.

Test plan

• make test-unit for touched test files (PR fix(adapters,vertex): pass output_config through to backends that accept it (closes #23380, supersedes #23475/#23396/#23706/#22727) #26439 tests: 26/26 pass)
• All four sync-pulled fixes already CI-green on their original PRs
• CI green on this consolidated branch
• (Optional but recommended) live-call check against Vertex Claude with output_format + tools payload to validate the P1 assumption

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
6 out of 7 committers have signed the CLA.

✅ Jah-yee
✅ MackDing
✅ dkindlund
✅ alvinttang
✅ mateo-berri
✅ he-yufeng
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[veria-ai]

Anthropic adapter output_config passthrough fix and upstream sync

This PR fixes parameter translation between Anthropic's output_config/output_format and OpenAI-format response_format for non-Anthropic backends, adds a safe attribute accessor for Pydantic models in the Arize integration, supports a new Cloudflare response key, and adds the Z.AI provider to the UI. The changes are confined to provider translation layers and telemetry utilities — no authentication, authorization, or input validation boundaries are affected. The parameter sanitization logic correctly filters to an allowlist of known-supported keys rather than forwarding arbitrary user input to downstream providers.

---

Status: 0 open
Risk: 1/10

[greptile-apps]

Greptile Summary

This merge rolls the output_config passthrough fix (PR #26439) and four upstream OSS staging patches into internal staging. The headline change replaces unconditional output_format/output_config drops on Vertex AI Claude with a targeted sanitizer that forwards format while stripping effort, and prevents output_config from leaking past the /v1/messages → /chat/completions adapter into non-Anthropic backends.

• P1 (Cloudflare streaming): The response_text fallback key added in transform_response for Nemotron models was not applied to CloudflareChatResponseIterator.chunk_parser, so streaming requests to those models will silently return empty content.

Confidence Score: 4/5

Safe to merge with one P1 streaming gap in the Cloudflare path and the acknowledged unconfirmed Vertex output_format parity assumption.

One confirmed P1: the Cloudflare response_text streaming gap produces silent empty content for Nemotron models in streaming mode. The Vertex output_format+tools parity concern is already tracked in the previous review thread. All other changes are well-tested and correct.

litellm/llms/cloudflare/chat/transformation.py (streaming path not updated); litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/transformation.py (Vertex parity assumption).

Important Files Changed

Filename	Overview
litellm/llms/cloudflare/chat/transformation.py	Adds response_text fallback for non-streaming responses but leaves the streaming chunk_parser unchanged, producing empty content for Nemotron models in streaming mode.
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py	Adds ANTHROPIC_ONLY_REQUEST_KEYS constant and gates the post-translation extra-kwargs re-merge with it, correctly preventing output_config leakage to non-Anthropic backends.
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py	Extends _translate_output_format_to_openai to read output_config.format as a fallback when top-level output_format is absent; clean precedence logic.
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/output_params_utils.py	New leaf-module helper sanitize_vertex_anthropic_output_params that strips effort from output_config while preserving format, avoiding the CodeQL cyclic-import flagged by the original design.
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/transformation.py	Replaces unconditional output_format/output_config drops with sanitize_vertex_anthropic_output_params; behavior depends on Vertex AI accepting output_format alongside tools/tool_choice, which is unconfirmed per PR description.
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py	Same sanitization applied to the Messages pass-through path; same unconfirmed Vertex parity assumption applies.
litellm/integrations/arize/_utils.py	Introduces _safe_get to handle both dict-like and Pydantic-model usage objects; also adds completion_tokens_details branch for Chat Completions API reasoning tokens.
litellm/constants.py	Removes the duplicate MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB definition (512-default); the surviving definition at line 85 (1024-default) is the authoritative one imported by in_memory_cache.py.
litellm/proxy/proxy_server.py	Adds verbose_logger.setLevel(logging.INFO) so LITELLM_LOG=INFO now propagates to the package-level logger, not just the proxy/router loggers.
ui/litellm-dashboard/src/components/provider_info_helpers.tsx	Adds ZAI provider entry to the Providers enum, provider_map, and getPlaceholder; straightforward addition with test coverage.
tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py	Existing regression test inverted to assert output_format IS forwarded; new tests added for sanitizer helper and mixed format+effort cases. Correctness depends on unconfirmed Vertex AI parity.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_handler_output_config_passthrough.py	New test file covering output_config strip, ANTHROPIC_ONLY_REQUEST_KEYS export, precedence rules, and empty extra_kwargs edge case; all mock-based, no real network calls.
tests/test_litellm/integrations/arize/test_arize_utils.py	Adds two regression tests for Pydantic usage objects (CompletionUsage and a plain Responses API shape); mock-only, good coverage for the _safe_get fix.

Comments Outside Diff (1)

1. litellm/llms/cloudflare/chat/transformation.py, line 202 (link)

   Streaming path doesn't handle response_text fallback

   The non-streaming transform_response now falls back to result.get("response_text") for newer Nemotron models, but CloudflareChatResponseIterator.chunk_parser (line 202, used in the streaming path) still only reads chunk["response"]. For streaming requests to Nemotron models, this will silently produce empty content since the chunk never enters the if "response" in chunk branch.

_{Reviews (3): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile}

[greptile-apps]

output_format re-introduced alongside tools/tool_choice on Vertex

The original test_vertex_ai_claude_sonnet_4_5_structured_output_fix was a regression guard for issue #18625, where output_format caused 400 "Extra inputs are not permitted" on Vertex AI Claude. The fix stripped output_format and used tool-based structured output exclusively.

This PR reverses that: output_format is now asserted to be present (assert "output_format" in final_data) while tools and tool_choice are also still present in the same payload. Sending all three simultaneously to Vertex Cloud Claude may reproduce the original 400 error unless Vertex has explicitly documented that output_format is now accepted alongside tool-based structured outputs. The PR comment says "Vertex parity has shifted" but there is no linked changelog or API documentation confirming that Vertex now tolerates output_format when tools are also present.

[greptile-apps]

ImportError: MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB removed but still imported downstream

This PR deletes MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB from constants.py, but litellm/caching/in_memory_cache.py still does from litellm.constants import MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB and uses it in both the constructor fallback and the check_value_size fast-path:

# in_memory_cache.py line 22
from litellm.constants import MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB   # ImportError at startup
...
self.max_size_per_item = max_size_per_item or MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB
...
< self.max_size_per_item * MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB

Any process that imports litellm.caching (which includes the entire proxy stack) will fail with ImportError: cannot import name 'MAX_SIZE_PER_ITEM_IN_MEMORY_CACHE_IN_KB' from 'litellm.constants'. This constant appears unrelated to the output_config fix and its removal looks accidental.

[mateo-berri]

this is a duplicate definition, safe to remove

[codecov]

Codecov Report

❌ Patch coverage is 90.00000% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/arize/_utils.py	85.71%	2 Missing ⚠️
litellm/llms/cloudflare/chat/transformation.py	0.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Sameerlite]

.

[Sameerlite]

@greptile-apps

[Sameerlite]

P1 Streaming path doesn't handle response_text fallback
The non-streaming transform_response now falls back to result.get("response_text") for newer Nemotron models, but CloudflareChatResponseIterator.chunk_parser (line 202, used in the streaming path) still only reads chunk["response"]. For streaming requests to Nemotron models, this will silently produce empty content since the chunk never enters the if "response" in chunk branch.

@mateo-berri is this comment valid?

[mateo-berri]

@mateo-berri is this comment valid?

good call out; fixed. Please re-review

[Sameerlite]

LGTM, thanks!