Repository navigation
[Bugfix][Tool Parser] Use LongCat tags in the LongCat structural tag - #60747
Conversation
LongcatFlashToolParser inherited structural_tag_model = "hermes" from Hermes2ProToolParser. With strict tool calling enabled (the default), tool_choice="required" and named tool choice were constrained to <tool_call>...</tool_call>, which the LongCat parser does not recognize, so every forced call came back as plain content. Register a "longcat" structural tag that reuses the Hermes builder with <longcat_tool_call> delimiters, matching the LongCat-Flash chat template. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment Once the PR is approved or has the If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
|
✅ @hpdkhoa, CI is now available for this PR.
|
|
/ci run |
|
✅ Triggered Buildkite CI #93738 for commit |
3 participants
Overview
With strict tool calling, forced tool calls on LongCat come back as plain text. The LongCat parser inherits the Hermes structural tag, so the grammar forces
<tool_call>tags that the LongCat parser does not recognize. This registers alongcatstructural tag that uses LongCat's own<longcat_tool_call>delimiters. The bug dates from #45003, which gave Hermes a structural tag.Claims
tool_choice="required"and named tool choice with--tool-call-parser longcatnow return parsedtool_callsinstead of raw tag text incontent.Validation
Unit test
test_longcat_forced_tool_choice_round_trips_native_tagsintests/tool_parsers/test_structural_tag_registry.pybuilds the forced grammar for the LongCat parser and checks that output in LongCat's native tags is accepted. It fails on main (2 failed: the grammar rejects the native tags) and passes with this change.pytest tests/tool_parsers/test_structural_tag_registry.py tests/tool_parsers/test_longcat_tool_parser.py tests/tool_parsers/test_hermes_tool_parser.py # 313 passed, 1 skipped, 1 xfailedEnd to end,
vllm serve Qwen/Qwen2.5-0.5B-Instruct --enable-auto-tool-choice --tool-call-parser longcatwithtool_choice="required":tool_callscontentA small model stands in for LongCat-Flash, because the structural tag decides the output format whatever the model is.
Details
_hermes_tool_tagsnow takes the start and end tag, and the shared body moved into_hermes_style_structural_tag.hermesand the newlongcattag both call it, so no grammar logic is duplicated.structural_tag_model = "hermes", so it keeps this bug. If [Parser] Migrate Hermes tool parser to the new streaming Parser Engine #51937 lands first, its LongCat adapter needs the same one-line setting.Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.