Repository navigation
[Rust Frontend] Integrate xgrammar-structural-tag for strict and required tool calling - #46057
Merged
Merged
Conversation
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
BugenZhao
force-pushed
the
bz/xgrammar-overwhelming-jackal
branch
from
June 22, 2026 11:47
12ab6b2 to
fd79067
Compare
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
BugenZhao
marked this pull request as ready for review
June 22, 2026 12:16
BugenZhao
commented
Jun 22, 2026
| vllm-tokenizer = { path = "src/tokenizer" } | ||
| vllm-tool-parser = { path = "src/tool-parser" } | ||
| winnow = "1.0.2" | ||
| xgrammar-structural-tag = { git = "https://github.com/Inferact/xgrammar-structural-tag.git", rev = "8c74b8c415834170364f075bd04457955639744d" } |
Member
Author
There was a problem hiding this comment.
We can release it to crates.io once it's stabilized.
Member
Author
There was a problem hiding this comment.
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
njhill
approved these changes
Jun 24, 2026
nkzhenhua
pushed a commit
to nkzhenhua/vllm
that referenced
this pull request
Jun 24, 2026
…`required` tool calling (vllm-project#46057) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
wincent8
pushed a commit
to wincent8/vllm
that referenced
this pull request
Jun 29, 2026
…`required` tool calling (vllm-project#46057) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
efschu
pushed a commit
to efschu/shvllm
that referenced
this pull request
Jul 18, 2026
…`required` tool calling (vllm-project#46057) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
6 tasks done
michael-gpu
pushed a commit
to practical-inference/vllm-ampere-extended
that referenced
this pull request
Oct 20, 2026
…`required` tool calling (vllm-project#46057) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Integrates
xgrammar-structural-taginto the Rust frontend tool-calling path:xgrammar-structural-tagdependency and exposes structural-tag model mappings from supported Rust tool parsers.tool_choice=required, named function choices, andtool_choice=autowith strict tools.tool_choice=requiredand named function choices through the OpenAI chat request conversion path.response_format/structured_outputslowering at the server boundary while the chat parser adjustment layer owns structural-tag overrides.AI assistance was used to help prepare this change.
Test Plan
cargo nextest run -p vllm-server validate_request_compat prepare_chat_requestcargo nextest run -p vllm-chat structural_tagTest Result
cargo nextest run -p vllm-server validate_request_compat prepare_chat_request: 49 passed, 206 skipped.cargo nextest run -p vllm-chat structural_tag: 8 passed, 187 skipped.Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.