[WoosukKwon]

Overview

Second slice of #61978 (Reflection Beam), after #62042: the beam reasoning and tool parsers, in Python and in the Rust frontend. They take effect only with --reasoning-parser beam or --tool-call-parser beam. This does not enable the model end to end.

Claims

• New parsers: --reasoning-parser beam and --tool-call-parser beam parse output with the checkpoint's response_template metadata, as hf does. They also derive a structural-tag grammar from the template, which constrains:
  • strict tools;
  • required and named tool choice;
  • structured outputs;
  • by default, every other request, so output follows the template's format. VLLM_BEAM_ALWAYS_CONSTRAIN_OUTPUT=0 turns this off.
• Docs: docs/features/tool_calling.md documents the parser.

Validation

Results on this branch next to upstream main (fa75d34), on the same machine:

Command	This PR	main
pytest tests/parser tests/reasoning tests/tool_parsers --ignore=tests/parser/cohere	8267 passed	8064 passed
cargo test -p vllm-parser	578 passed	561 passed
cargo test -p vllm-chat --lib	357 passed	356 passed

• Same failures on both trees, all from this machine's setup:
  • Python: 10 failed and 117 errors in tests/parser/mistral and the Step-3.5 parser tests, because their tokenizers can't be downloaded into the read-only shared HF cache.
  • Rust: the vllm-chat roundtrip and grammar_replay tests, because they need write access to a lock file or that same HF cache.
• Not run: tests/parser/cohere needs the optional cohere_melody package, which isn't installed here.

Details

• Shared code touched:
  • ParserManager now picks any response-template parser, not just hf. A parser opts in through unified_parser_cls(). The hf behavior is unchanged except for error messages, which now name the selected flags.
  • OnlineRenderer passes the prompt token IDs to the parser before adjust_request. Beam needs this to see whether the prompt already opened reasoning or an answer. Only the response-template parsers (hf, beam) override set_prompt_token_ids; for all other parsers the base class method does nothing.
  • ResponsesRequest gets two private attributes, so a parser-built grammar can replace text.format without changing the request's public fields.
• Response-template parser fixes, which also apply to hf:
  • When output stops inside a closer whose last part is the next message's start anchor, the parser completes the closer instead of reporting it as malformed.
  • Reasoning tokens are counted incrementally from the template.
• Size: about 2.3k non-test lines. Most of it is the grammar builder (vllm/parser/response_template_grammar.py) and the Rust beam parser.
• Not a duplicate: no open PR adds a Beam parser.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[mergify]

Documentation preview: https://vllm--62052.org.readthedocs.build/en/62052/

[WoosukKwon]

/ci run

[github-actions]

✅ Triggered Buildkite CI #95456 for commit 43a4cf47789d.