Repository navigation
[Bugfix][Tool Calling] Fix zero-argument and named tool calling in Qwen3 tool parser - #63277
sieuchuoicb wants to merge 4 commits into
Conversation
…en3 tool parser Fixes tool call argument handling in Qwen3 tool call parser: - Set supports_required_and_named = False and implement adjust_request() in Qwen3EngineToolParser to bypass structured JSON output enforcement for tool_choice="required" and named tool choices when structural tags are not used, allowing native XML tool calls to be emitted and parsed. - Strip leading and trailing whitespace/newlines from function names in ParserEngine._emit_name_delta() and _handle_tool_end(). - Update AnthropicServingMessages so zero-argument tools are recognized as complete with tool_use stop reason. - Add zero-parameter function invocation instructions in Qwen3-Coder chat template. - Add test coverage for zero-argument tool calls and named tool choice. Co-authored-by: Antigravity <antigravity@google.com> Signed-off-by: sieuchuoicb <nguyenphuclancb@gmail.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment Once the PR is approved or has the If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
|
Documentation preview: https://vllm--63277.org.readthedocs.build/en/63277/ |
mergify
Bot
added
documentation
|
For background: We are running a production Coding Agent served on 8x H200 GPUs. When using Qwen models (Qwen3.8-27B, Qwen-Flash-Next, etc.), zero-argument tool calls and forced tool choices consistently fail in client validation with __unparsedToolInput, whereas GPT and Gemma 4 models complete without errors. This PR resolves that parity issue by skipping JSON guided decoding enforcement for Qwen's native XML tool format and handling function name whitespace in streaming. |
|
@jperezdealgaba can you take a look of my PR, please? |
Overview
Fixes tool call parser behavior for Qwen3 (
qwen3_coderandqwen3_xml) when handling zero-argument tools and forced/named tool choices (tool_choice="required"ortool_choice={"type": "function", "function": {"name": ...}}), preventing Pydantic / MCP client validation failures (e.g.,unexpected_keyword_argument: __unparsedToolInput).Claims
structured_outputs.json) fortool_choice="required"and named tool choices inQwen3EngineToolParserwhen structural tags are not used, allowing Qwen3 to emit its native XML format (<tool_call><function=...) instead of triggering structured output mismatches (mirrorsGemma4EngineToolParserandGlm47MoeToolParser).ParserEngine(_emit_name_deltaand_handle_tool_end), preventing downstream client tool lookup failures when models emit<function=name \n>.AnthropicServingMessagesstreaming completion check so zero-argument tool calls without argument chunks are recognized as complete (tool_complete = True) withstop_reason="tool_use"rather than falling back tostop_sequence.examples/tool_chat_template_qwen3coder.jinjaprompt instructions (<tool_call>\n<function=name>\n</function>\n</tool_call>).Validation
Unit and Regression Tests
tests/parser/engine/test_qwen3.py:.venv/bin/python -m pytest tests/parser/engine/test_qwen3.py -vtest_streaming_empty_argsand addedtest_streaming_function_name_with_trailing_whitespace.tests/tool_parsers/test_qwen3coder_tool_parser.py:.venv/bin/python -c "import torch.accelerator; torch.accelerator.empty_host_cache = lambda: None; import pytest, sys; sys.exit(pytest.main(['tests/tool_parsers/test_qwen3coder_tool_parser.py', '-v']))"single_zero_arg_toolacross non-streaming and streaming tests for Chat Completions and Responses API, andtest_adjust_request_skips_json_guided_decoding_for_named_tool_choice..venv/bin/pre-commit run --files examples/tool_chat_template_qwen3coder.jinja tests/parser/engine/test_qwen3.py tests/tool_parsers/test_qwen3coder_tool_parser.py vllm/entrypoints/anthropic/serving.py vllm/parser/engine/parser_engine.py vllm/tool_parsers/qwen3_engine_tool_parser.pyruff check,ruff format,typos,check-test-tethering,mypy,Check SPDX headers, etc.).Model Evaluation
Not applicable: this change touches API serving and tool call parsing logic only; model weights, token generation kernels, and forward pass outputs are unaffected.
Details
Root Cause
ToolParserdefaultssupports_required_and_named = True. When client requests usedtool_choice="required"or a named tool choice (common in MCP tool orchestration agents),ToolParser.adjust_requestsetrequest.structured_outputsto JSON guided decoding. Because Qwen models generate XML (<tool_call><function=...), JSON guided decoding either restricted output to raw JSON (which Qwen chat template did not prompt for) or caused fallback wrappers (__unparsedToolInput: {"raw": "{}", "summary": {}}) in client frameworks (e.g., Pydantic AI / MCP). Overridingadjust_requestinQwen3EngineToolParser(matching Gemma4 and GLM4 implementations) skips JSON guided decoding when structural tags are not active, preserving native XML tool output generation and parsing.<function=describe_cluster \n>) were emitted directly intoDeltaFunctionCall.name, causing client agent lookup failures.AnthropicServingMessages, zero-argument tool calls emit no argument delta chunks (last_tool_args == [])._is_json("".join(last_tool_args))evaluated_is_json("") -> False, settingtool_complete = Falseand returningstop_reason="stop_sequence"instead ofstop_reason="tool_use". Defaulting empty argument buffers to"{}"ensures zero-argument tools are marked complete.Duplicate-Work Checks
Checked open issues and pull requests via GitHub CLI:
gh pr list --repo vllm-project/vllm --state open --search "qwen3 tool"gh pr list --repo vllm-project/vllm --state open --search "qwen3_coder"gh pr list --repo vllm-project/vllm --state open --search "supports_required_and_named"gh pr list --repo vllm-project/vllm --state open --search "AnthropicServingMessages"Found no overlapping open PRs addressing zero-argument or named tool choice handling in Qwen3 tool parsing.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.