[Refactor] Consolidate required/named tool_choice streaming into DelegatingParser#41876
[Refactor] Consolidate required/named tool_choice streaming into DelegatingParser#41876vllm-bot merged 1 commit intovllm-project:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the tool call streaming logic by consolidating the handling of 'required' and 'named' tool choices into the abstract parser. It removes redundant extraction logic from the OpenAIServingChat class and ensures that the parser state is correctly initialized with tool call metadata. A potential issue was identified in how the tool call counter is incremented, as the current implementation only checks the first tool call in a delta message, which could lead to inaccuracies if multiple tool calls are initiated simultaneously.
chaunceyjiang
left a comment
There was a problem hiding this comment.
LGTM.
One small question: why does your test command explicitly specify an additional chat template?
Oh it works without specifying the chat template, the agent picked it up this time randomly |
…gatingParser (vllm-project#41876) Signed-off-by: sfeng33 <4florafeng@gmail.com> Signed-off-by: alexagriffith <agriffith96@gmail.com>
…gatingParser (vllm-project#41876) Signed-off-by: sfeng33 <4florafeng@gmail.com> Signed-off-by: Libin Tang <libin.tang@intel.com>
Purpose
For chat completion api, migrate required/named tool_choice streaming to use parse_delta in the unified parser.
Test Plan
"required"{"type":"function","function":{"name":"get_weather"}}"auto"