[alec-flowers]

Overview

The Rust frontend's gRPC server lowers every structured-output constraint to the guidance backend (#44729). Guidance only parses the legacy triggers/structures structural tag (#55152), so gRPC clients that send xgrammar-format structural tags for tool calling get INTERNAL on every request. This PR makes the gRPC path use xgrammar for every constraint, as Python's auto does, teaches the xgrammar backend to compile choice itself, and applies Python's xgrammar schema check in the Rust frontend's shared request validation.

Claims

• gRPC structural_tag constraints in the current {"type": "structural_tag", "format": ...} shape compile and constrain output instead of failing in EngineCore with KeyError: 'triggers'.
• All gRPC structured-output requests select one backend, so a gRPC-served engine no longer depends on which request arrives first to pick its backend.
• The xgrammar backend compiles choice requests with the same EBNF grammar Python's choice_as_grammar produces, so frontends that skip Python request validation can send choice as is. This also fixes Rust HTTP choice requests failing with grammar is not of valid supported types once xgrammar is the engine's backend.
• JSON schemas that use keywords xgrammar ignores are rejected when the request's backend is xgrammar, with Python's message, instead of being silently under-enforced. gRPC returns INVALID_ARGUMENT; Rust HTTP would return 400, though no HTTP constraint takes this path yet.
• Regexes containing NUL are rejected in StructuredOutputsParams::validate, as Python does for every backend. This also applies to the Rust HTTP routes.

Validation

Qwen3.6-35B-A3B on GB200 through the Dynamo vLLM sidecar, which talks to vllm-rs over gRPC and sends xgrammar-format structural tags for tool calls (ai-dynamo/dynamo#16494). Same vLLM nightly in both columns; the right column adds an earlier revision of this commit (xgrammar for every gRPC constraint plus the choice rewrite) to vllm-rs.

Check	vLLM main	This PR
tool_choice: auto + one tool	HTTP 500 (gRPC INTERNAL, KeyError: 'triggers')	HTTP 200
Dynamo tool-calling suite (48 cases)	not run, every tool request 500s	47/48 (the miss is tool markup inside reasoning, also intermittent on other runtimes)
BFCL sample (50 cases)	not run	39/50, all HTTP 200 (other Dynamo runtimes score 38 to 43 on this model)

Tests on the current head:

• Rust: cargo nextest run -p vllm-engine-core-client -p vllm-text -p vllm-server: 713 passed. unary_generate_invalid_structured_outputs_returns_invalid_argument sends an empty grammar, an empty choice, a NUL regex and an xgrammar-unsupported schema through the real gRPC server against the mock engine and expects INVALID_ARGUMENT for each. With the text-lowering validation removed, the request reaches the engine and the test fails.
• Python: test_xgrammar_compiles_choice_without_rewrite compiles an unrewritten choice request through get_structured_output_key. It fails without the backend change (grammar is not of valid supported types. (StructuredOutputOptions.CHOICE)) and passes with it. test_backend_xgrammar.py and test_backend_xgrammar_lark.py have the same results with and without this PR.

Details

Root cause. StructuredOutputManager creates one backend from the first structured-output request it sees and compiles every later request with it (vllm/v1/structured_output/__init__.py). The Rust frontend skips Python's SamplingParams validation, which is where _backend is normally chosen, so #44729 set the backend to guidance, which handles choice natively. The guidance backend reads structural tags in the legacy shape only. The format shape that tool-calling clients and the Rust chat path (#46057) produce raises KeyError: 'triggers' in GuidanceBackend.compile_grammar, the request finishes with an error, and the gRPC server returns INTERNAL.

Why every constraint, not only structural tags. With xgrammar for structural tags and guidance for the rest, whichever kind of request reached the engine first would decide the backend, and the other kind would fail. One backend for the whole gRPC path avoids that.

Where the xgrammar checks live. Python's validate_xgrammar_grammar rewrites choice into a grammar (the xgrammar backend has no choice path), and auto keeps schemas that has_xgrammar_unsupported_json_features flags away from xgrammar. The Rust frontend bypasses that validation. Both rules depend on the backend, not on the protocol, so neither goes into the gRPC converter:

• choice: XgrammarBackend.compile_grammar compiles CHOICE with choice_as_grammar, as [Structured Outputs] Parse Lark grammars natively in the xgrammar backend #58321 did for Lark. Every frontend sends choice unchanged, and the grammar has one implementation.
• Unsupported schema keywords: has_unsupported_json_features is ported into engine-core-client, and StructuredOutputsParams::validate rejects such schemas when the backend is xgrammar. Text lowering (lower_sampling_params) validates the final constraint for every route, after chat may have replaced the user's constraint with a parser-built output grammar. The error maps to 400 / INVALID_ARGUMENT. The gRPC converter only selects xgrammar and no longer validates on its own.

Python's NUL-in-regex rejection is backend-independent, so it goes into StructuredOutputsParams::validate next to the other early checks.

Tradeoff. Python's auto sends schemas with keywords xgrammar ignores (multipleOf, uniqueItems/contains, unsupported string formats, pattern with length bounds, some propertyNames/patternProperties combinations, multi-branch allOf) to guidance. EngineCore holds one backend, so the gRPC path cannot do that per request. It now rejects them with INVALID_ARGUMENT, as Python does when the backend is xgrammar, rather than accepting them and enforcing less than the schema says. gRPC clients that relied on guidance for those keywords will see the error.

Not changed. Python also parses grammars, regexes and schemas with xgrammar up front; on the gRPC path an invalid constraint still fails at EngineCore compile time, as it did under guidance. The Rust frontend has no JSON nesting cap (#60036) on any route, before or after this PR. Python's auto sends Lark grammars for Tekken Mistral tokenizers to guidance; on the gRPC path they now use xgrammar.

Not a duplicate.

• [Rust Frontend] Only accept XGrammar backend for structured outputs #55258 (closed) moved all Rust-frontend structured outputs to xgrammar and added a choice path to the Python xgrammar backend. Its author noted that the engine supporting several backends at once may be the right long-term direction. This PR is narrower: it changes the gRPC backend choice, adds the same choice path to the Python xgrammar backend, and adds the backend-keyed schema check and the NUL check to shared Rust validation, with no CLI changes. It fixes a hard failure now and does not get in the way of engine-side multi-backend support.
• [Bugfix][Structured Outputs] Support new-format structural tags in the guidance backend (strict/required/named tool calling) #55153 (open) teaches the guidance backend some format shapes (triggered_tags, tags_with_separator, any_text) and rejects the rest. xgrammar compiles the full structural-tag format, and the Rust chat path already sends these tags with backend = xgrammar ([Rust Frontend] Integrate xgrammar-structural-tag for strict and required tool calling #46057), so the gRPC path now matches it. The two PRs do not conflict.
• [Bugfix] Reject mixed structured output backends during validation #49625 (open) rejects mismatched backends at validation time. It addresses the same single-backend limit from the engine side.

Limitation / follow-up. The Rust HTTP routes still lower native constraints to guidance while parser-built output grammars use xgrammar (#46057, #60781). A server that takes both kinds of traffic over HTTP is still order-dependent. With the xgrammar choice path, HTTP choice no longer fails when xgrammar is chosen first, but parser-built structural tags still fail when guidance is chosen first. This PR leaves those routes' backend choice unchanged; the follow-up is a single operator-selected backend for every Rust route, defaulting to xgrammar.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

AI assistance (Claude) was used to investigate, write and test this change. I reviewed every changed line and ran the Rust unit tests, clippy and pre-commit on it.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[BugenZhao]

Thanks @alec-flowers I'll add some commits to address my comments

[BugenZhao]

/ci run

[github-actions]

✅ Triggered Buildkite CI #95727 for commit 485a752af90a.

ℹ️ This PR is 136 commits behind upstream main. Merged upstream main into this branch as 485a752af90a before starting CI. Pull the branch before pushing more changes.

[BugenZhao]

/ci run

[github-actions]

✅ Triggered Buildkite CI #95738 for commit ffc91f253112.

ℹ️ This PR is 4 commits behind upstream main. Merged upstream main into this branch as ffc91f253112 before starting CI. Pull the branch before pushing more changes.