Skip to content

[litellm-agent] Staging → litellm_internal_staging (5/20/2026) #28310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
oss-pr-review-agent-shin wants to merge 8 commits into from
Closed

[litellm-agent] Staging → litellm_internal_staging (5/20/2026) #28310

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
43bc7d6
feat: add Xiaomi MiMo-V2.5-Pro and MiMo-V2.5 OpenRouter model entries…
TorvaldUtne May 19, 2026
91927eb
fix(ui): trim whitespace from MCP inspector tool call inputs (#28203)
oss-agent-shin May 19, 2026
6f83cb2
gemini-3.1-flash-lite pricing (#27933)
mubashir1osmani May 19, 2026
249ec01
fix: incorrect /v1/agents request example (#28131)
IshaMeera May 19, 2026
d9af172
fix(anthropic): accept dict-shape reasoning_effort from Responses bri…
cwang-otto May 19, 2026
87a55f5
feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite (…
ro31337 May 19, 2026
5039e63
fix(router): wrap aresponses streaming iterator for mid-stream fallba…
cwang-otto May 20, 2026
6ea1f57
fix(router): unblock staging — mypy + coverage for aresponses streami…
cwang-otto May 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions litellm/llms/anthropic/chat/transformation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1506,9 +1506,21 @@ def map_openai_params( # noqa: PLR0915
optional_params["metadata"] = {"user_id": value}
elif param == "thinking":
optional_params["thinking"] = value
elif param == "reasoning_effort" and isinstance(value, str):
elif param == "reasoning_effort":
# Accept both string ("low") and dict ({"effort": "low",
# "summary": "concise"}). The Responses->Chat parser keeps the
# full dict when `summary` is set (see #25359), so a dict here
# is the standard shape Otto/OpenAI-Responses-Bridge callers
# send. Coerce to the effort string before mapping — same
# shape-tolerance the GPT-5 path already implements in
# `_normalize_reasoning_effort_for_chat_completion`.
effort_value = value
if isinstance(effort_value, dict):
effort_value = effort_value.get("effort")
if not isinstance(effort_value, str):
continue
mapped_thinking = AnthropicConfig._map_reasoning_effort(
reasoning_effort=value,
reasoning_effort=effort_value,
model=model,
llm_provider=self.custom_llm_provider or "anthropic",
)
Expand All @@ -1519,12 +1531,12 @@ def map_openai_params( # noqa: PLR0915
optional_params["thinking"] = mapped_thinking
if AnthropicConfig._is_adaptive_thinking_model(model):
mapped_effort = REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get(
value
effort_value
)
if mapped_effort is None:
AnthropicConfig._raise_invalid_reasoning_effort(
model=model,
value=value,
value=effort_value,
llm_provider=self.custom_llm_provider or "anthropic",
)
optional_params["output_config"] = {"effort": mapped_effort}
Comment on lines 1506 to 1542

@greptile-apps greptile-apps Bot May 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Hardcoded model-specific flag check — should use model_prices_and_context_window.json

_is_effort_supported_model and _is_adaptive_thinking_model (called later in this block) are hardcoded model-name lists. The team rule is that model capability flags belong in model_prices_and_context_window.json and are read via get_model_info / supports_* helpers so they work automatically when new models are added. Putting them here means every new model that supports effort-based thinking requires a LiteLLM code change rather than a config-file entry.

Rule Used: What: Do not hardcode model-specific flags in the ... (source)

Expand Down
52 changes: 52 additions & 0 deletions litellm/model_prices_and_context_window_backup.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26951,6 +26951,58 @@
"supports_web_search": true,
"tpm": 800000
},
"openrouter/google/gemini-3.1-flash-lite": {
"cache_read_input_token_cost": 2.5e-08,
"cache_read_input_token_cost_per_audio_token": 5e-08,
"input_cost_per_audio_token": 5e-07,
"input_cost_per_token": 2.5e-07,
"litellm_provider": "openrouter",
"max_audio_length_hours": 8.4,
"max_audio_per_prompt": 1,
"max_images_per_prompt": 3000,
"max_input_tokens": 1048576,
"max_output_tokens": 65536,
"max_pdf_size_mb": 30,
"max_tokens": 65536,
"max_video_length": 1,
"max_videos_per_prompt": 10,
"mode": "chat",
"output_cost_per_reasoning_token": 1.5e-06,
"output_cost_per_token": 1.5e-06,
"rpm": 2000,
"source": "https://ai.google.dev/gemini-api/docs/pricing#gemini-3.1-flash-lite",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions",
"/v1/batch"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_audio_input": true,
"supports_audio_output": false,
"supports_code_execution": true,
"supports_file_search": true,
"supports_function_calling": true,
"supports_parallel_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_url_context": true,
"supports_video_input": true,
"supports_vision": true,
"supports_web_search": true,
"tpm": 800000
},
"openrouter/google/gemini-3.1-pro-preview": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
Expand Down
2 changes: 1 addition & 1 deletion litellm/proxy/_lazy_openapi_snapshot.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3171,7 +3171,7 @@
]
},
"post": {
"description": "Create a new agent\n\nExample Request:\n```bash\ncurl -X POST \"http://localhost:4000/agents\" \\\n -H \"Authorization: Bearer <your_api_key>\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"agent\": {\n \"agent_name\": \"my-custom-agent\",\n \"agent_card_params\": {\n \"protocolVersion\": \"1.0\",\n \"name\": \"Hello World Agent\",\n \"description\": \"Just a hello world agent\",\n \"url\": \"http://localhost:9999/\",\n \"version\": \"1.0.0\",\n \"defaultInputModes\": [\"text\"],\n \"defaultOutputModes\": [\"text\"],\n \"capabilities\": {\n \"streaming\": true\n },\n \"skills\": [\n {\n \"id\": \"hello_world\",\n \"name\": \"Returns hello world\",\n \"description\": \"just returns hello world\",\n \"tags\": [\"hello world\"],\n \"examples\": [\"hi\", \"hello world\"]\n }\n ]\n },\n \"litellm_params\": {\n \"make_public\": true\n }\n }\n }'\n```",
"description": "Create a new agent\n\nExample Request:\n```bash\ncurl -X POST \"http://localhost:4000/v1/agents\" \\\n -H \"Authorization: Bearer <your_api_key>\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"agent_name\": \"my-custom-agent\",\n \"agent_card_params\": {\n \"protocolVersion\": \"1.0\",\n \"name\": \"Hello World Agent\",\n \"description\": \"Just a hello world agent\",\n \"url\": \"http://localhost:9999/\",\n \"version\": \"1.0.0\",\n \"defaultInputModes\": [\"text\"],\n \"defaultOutputModes\": [\"text\"],\n \"capabilities\": {\n \"streaming\": true\n },\n \"skills\": [\n {\n \"id\": \"hello_world\",\n \"name\": \"Returns hello world\",\n \"description\": \"just returns hello world\",\n \"tags\": [\"hello world\"],\n \"examples\": [\"hi\", \"hello world\"]\n }\n ]\n },\n \"litellm_params\": {\n \"make_public\": true\n }\n }'\n```",
"operationId": "create_agent_v1_agents_post",
"requestBody": {
"content": {
Expand Down
Loading
Loading