Repository navigation
Conversation
Registers a gemma4 xgrammar structural tag that forces the native call syntax, attaches it on both the composed parser path and the unified engine path, and lexes the call markers from text. Source: klanggames/iac-gcloud images/vllm-gemma4 at ce8e18d76. Upstream: vllm-project#50477, vllm-project#53363.
…a style Needs the Klang xgrammar fork (EvansThomas/xgrammar, tag klang-gemma-2), which adds style="gemma" to JSONSchemaFormat. Schemas the style cannot drive fall back to a syntax-only tag. Source: klanggames/iac-gcloud images/vllm-gemma4 at eea31c1ec.
Upstream xgrammar (mlc-ai/xgrammar#923) adds the gemma JSON-schema style but keeps declaration order. Sort properties and object literals like the chat template's dictsort here, so the Klang xgrammar fork is no longer necessary.
vLLM 0.32.0 treats a tool without an explicit strict as non-strict, so its arguments went unconstrained. Restore the 0.28.0 rule for gemma4: unless tool_choice is auto with no strict tool, a tool without strict is constrained, and strict=false stays free.
Upstream xgrammar excludes only the string delimiter from gemma strings, so pass both call markers as excludes. It also compiles patternProperties and non-identifier keys, which the fork rejected, so those schemas now stay constrained.
oe374kt
marked this pull request as ready for review
October 29, 2026 14:46
EvansThomas
approved these changes
Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
klang/v0.32.0).4221346(feat: support Gemma 4 tool calling with the <|"|> string delimiter mlc-ai/xgrammar#923):dictsort.strictschema-constrained (vLLM 0.32.0 made them non-strict).excludes.