fix: clear response_format when tool_choice is auto to allow tool calls

[he-yufeng]

Summary

When a request includes both tools and response_format (e.g. json_object) with tool_choice: "auto" (the default), constrained JSON decoding from response_format forces the model to produce JSON content and prevents it from generating tool call tokens. The model returns tool_calls: [] and answers directly as JSON.

This was already fixed for tool_choice: "required" in #32006, but the tool_choice: "auto" case was missed.

Fix

In adjust_request(), when get_json_schema_from_tools returns None (the "auto" case) but tools are present, clear response_format so the model can freely choose between tool calls and text output. The tool parser handles extraction regardless of output format.

Fixes #39929

Changes

1 file changed — vllm/tool_parsers/abstract_tool_parser.py

# Before: response_format only cleared for "required"/"forced function"
if json_schema_from_tool is not None:
    request.response_format = None

# After: also cleared for "auto" when tools are present
if json_schema_from_tool is not None:
    request.response_format = None
elif isinstance(request, ChatCompletionRequest):
    # tool_choice: "auto" -- clear response_format so constrained
    # decoding doesn't prevent the model from generating tool calls.
    request.response_format = None

Test plan

• Code review: the elif only triggers when json_schema_from_tool is None (i.e. tool_choice="auto") and request.tools is non-empty (checked at line 80)
• With tool_choice="auto" + response_format=json_object + tools → model can now return tool_calls
• Without tools, adjust_request returns early (line 80-81), so response_format is preserved
• tool_choice="required" path unchanged (still goes through the if branch)

[gemini-code-assist]

Code Review

This pull request updates the adjust_request method in abstract_tool_parser.py to clear the response_format for ChatCompletionRequest instances, ensuring that constrained decoding does not interfere with tool call generation. A review comment identifies a potential regression where this change would incorrectly clear the response format even when tool_choice is set to 'none', and suggests refining the condition to specifically target 'auto' tool selection.

[gemini-code-assist]

The current implementation clears response_format for any ChatCompletionRequest where json_schema_from_tool is None. However, json_schema_from_tool is None for both tool_choice="auto" and tool_choice="none" (as defined in get_json_schema_from_tools in vllm/tool_parsers/utils.py).

If a user provides tools but explicitly sets tool_choice="none" while also requesting a specific response_format (e.g., json_object), this change will incorrectly clear their response_format. This results in a regression where the model's output is no longer constrained to JSON even though tool calling is disabled. The condition should explicitly check for tool_choice == "auto".

Suggested change

	elif isinstance(request, ChatCompletionRequest):
	elif isinstance(request, ChatCompletionRequest) and request.tool_choice == "auto":

[sfeng33]

nit - can be more explicit here

elif isinstance(request, ChatCompletionRequest) and request.tool_choice in ("auto", None):

[sfeng33]

Can you please fix DCO?

[chaunceyjiang]

I think an E2E test is needed.

[he-yufeng]

Thanks for the reviews! Addressed:

• @sfeng33 — tightened the condition to tool_choice in ("auto", None) so tool_choice="none" (caller explicitly opted out of tools) keeps response_format intact. DCO fixed via sign-off.
• @chaunceyjiang — added unit tests covering all four cases in tests/tool_parsers/test_openai_tool_parser.py:
  • tool_choice="auto" + response_format → cleared
  • tool_choice unset + tools → cleared (auto is the default)
  • tool_choice="none" + response_format → preserved
  • no tools + response_format → preserved

Since adjust_request lives in the base ToolParser class, the OpenAIToolParser exercises it directly. Let me know if you'd like the same check added as a protocol-level test in tests/tool_use/ — happy to move it.

[he-yufeng]

Hi, checking in on this after the latest update. I addressed the DCO issue and the review feedback by preserving response_format for tool_choice="none", then added the focused parser tests for the covered cases. GitHub currently shows DCO, pre-commit, RTD, and the summary check passing. Happy to adjust further if you prefer a protocol-level test instead.

[sfeng33]

Thanks for the work! The change LGTM, deferring to @chaunceyjiang for feedback on the test coverage.