Skip to content

fix(hosted_vllm): map output_config effort and adaptive thinking to reasoning_effort #44563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
iosayin wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix(hosted_vllm): map output_config effort and adaptive thinking to reasoning_effort #44563

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5176440
fix(hosted_vllm): map output_config effort and adaptive thinking to r…
iosayin Oct 5, 2026
68f0a52
fix(hosted_vllm): remove dead elif in reasoning_effort mapping
iosayin Oct 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions litellm/llms/hosted_vllm/chat/transformation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,13 +95,20 @@ def map_openai_params(
if _tools is not None:
non_default_params["tools"] = _tools

output_config: Final = non_default_params.pop("output_config", None)

@greptile-apps greptile-apps Bot Oct 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Output effort never reaches mapper Direct completion() calls discard output_config during preprocessing, so this mapping cannot send the requested effort to hosted vLLM. The new tests bypass preprocessing

Knowledge Base Used: Model invocation runtime

thinking: Final = non_default_params.pop("thinking", None)
if thinking is not None and isinstance(thinking, dict):
if thinking.get("type") == "enabled":
if "reasoning_effort" not in non_default_params:
if "reasoning_effort" not in non_default_params:
if isinstance(output_config, dict) and output_config.get("effort"):
non_default_params["reasoning_effort"] = output_config["effort"]
elif thinking is not None and isinstance(thinking, dict):
if thinking.get("type") == "enabled":
non_default_params["reasoning_effort"] = reasoning_effort_from_thinking_budget(
thinking.get("budget_tokens", 0)
)
elif thinking.get("type") == "adaptive" and thinking.get("effort"):
non_default_params["reasoning_effort"] = thinking["effort"]
elif thinking is not None and isinstance(thinking, dict) and thinking.get("type") == "enabled":
pass

@greptile-apps greptile-apps Bot Oct 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Empty branch obscures precedence This branch only executes pass. Removing it would preserve behavior and make the effort precedence easier to follow

Suggested change
elif thinking is not None and isinstance(thinking, dict) and thinking.get("type") == "enabled":
pass

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


return super().map_openai_params(non_default_params, optional_params, model, drop_params)

Expand Down
25 changes: 25 additions & 0 deletions tests/unit/llms/hosted_vllm/chat/test_hosted_vllm_chat_transformation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,31 @@ def test_hosted_vllm_supports_thinking():
)
assert optional_params["reasoning_effort"] == "low"

# Test output_config.effort is mapped to reasoning_effort

@greptile-apps greptile-apps Bot Oct 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Test narration violates comment rule The new comments here and below restate the assertions. The repository prohibits such comments; remove both before merging

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

optional_params = config.map_openai_params(
non_default_params={
"thinking": {"type": "adaptive"},
"output_config": {"effort": "low"},
},
optional_params={},
model="hosted_vllm/GLM-4.6-FP8",
drop_params=False,
)
assert "thinking" not in optional_params
assert "output_config" not in optional_params
assert optional_params["reasoning_effort"] == "low"

# Test adaptive thinking with effort inside thinking dict
optional_params = config.map_openai_params(
non_default_params={
"thinking": {"type": "adaptive", "effort": "max"},
},
optional_params={},
model="hosted_vllm/GLM-4.6-FP8",
drop_params=False,
)
assert optional_params["reasoning_effort"] == "max"


def test_hosted_vllm_reasoning_content_kept_and_thinking_blocks_removed():
"""
Expand Down
Loading