[BugenZhao]

Overview

EngineCore compiles every structured-output request with the backend of the first one it sees. The Rust frontend sends gRPC constraints and parser-built output grammars as xgrammar, but every other HTTP constraint as guidance (#44729), so one server that takes both kinds of traffic is order-dependent: after a native constraint reaches the engine first, every later strict, required, or named tool call and every response_format composed with a reasoning parser fails with KeyError: 'triggers' (HTTP 500). This PR makes every Rust route use one backend, chosen by the operator with --structured-outputs-config.backend and defaulting to xgrammar, as Python's auto does for most requests. It is the follow-up named in #61892.

Claims

• Every Rust route (chat, completions, gRPC) sends one backend: --structured-outputs-config.backend, default xgrammar. auto selects xgrammar; guidance, outlines, and lm-format-enforcer are accepted. A server no longer depends on which request arrives first.
• With xgrammar, Rust HTTP rejects JSON schemas whose keywords xgrammar ignores with 400, as gRPC does since [Bugfix][Rust Frontend] Use xgrammar for gRPC structured outputs #61892 and as Python does with backend=xgrammar.
• With another backend, chat does not build parser output grammars (they are xgrammar structural tags): the request keeps its own constraint, which the engine applies after reasoning, and tool calls are parsed without a grammar. --tool-strict-level function|parameter fails startup unless the backend is xgrammar.
• Constraints the selected backend cannot compile return 400 instead of failing in EngineCore, following Python's per-backend validation: format-shape structural tags on guidance, and the constraint kinds outlines and LM Format Enforcer reject.
• vllm-rs serve forwards --structured-outputs-config to the managed engine, so engine-side options such as disable_any_whitespace take effect.

Validation

GPU smoke on GB200, Qwen3-8B with --reasoning-parser qwen3 --tool-call-parser hermes via VLLM_USE_RUST_FRONTEND=1 vllm serve. Same request sequence per server, starting with a native /v1/completions constraint; the completions prompts close the think block, since the engine's reasoning gate otherwise waits for </think> before applying a raw completions constraint.

Request (in order)	main	This PR, default (xgrammar)	This PR, backend=guidance
completions choice	200, valid	200, valid	200, valid
chat tool_choice=required	500 (KeyError: 'triggers')	200, get_weather call	200, get_weather call
chat response_format json_schema	500	200, valid JSON	200, valid JSON
chat strict tool, tool_choice=auto	500	200, get_weather call	200, get_weather call
completions regex	200, valid	200, valid	200, valid
completions json_schema with multipleOf	200, enforced (guidance)	400 (xgrammar ignores multipleOf)	200, enforced

Rust tests on the branch tip: cargo nextest run -p vllm-engine-core-client -p vllm-text -p vllm-chat -p vllm-server -p vllm-cmd -p vllm-managed-engine -p vllm-parser (1806 passed), cargo clippy --workspace --all-targets, cargo fmt --check. New tests:

• chat_structured_outputs_use_configured_backend (routes): a chat request with a strict tool and response_format through a mock engine; the engine request carries the configured backend, a structural tag with xgrammar and the request's own JSON schema with guidance. With the chat-side backend check removed, the guidance case fails (lowering rejects the format-shape tag with 400).
• lower_sampling_params_validates_against_configured_backend, backend_rejects_constraint_kinds_it_cannot_compile, and engine_structured_outputs_serializes_through_raw_shape (msgpack round trip).
• CLI: config parsing on vllm-rs serve and the --args-json bootstrap path, auto, forwarding to the managed engine, and the --tool-strict-level startup check.

Details

Root cause. StructuredOutputManager.grammar_init creates one backend from the first structured request and keeps it until shutdown. Python chooses _backend in SamplingParams validation, which the Rust frontend skips, so #44729 defaulted Rust to guidance; guidance handled choice, which xgrammar could not compile then. Parser-built output grammars (#46057) and gRPC (#61892) later moved to xgrammar, and guidance reads only the legacy triggers/structures structural tag.

Where the backend lives. The backend is now a field of the EngineCore request type only (EngineCoreStructuredOutputsParams). StructuredOutputsParams, the type the HTTP/gRPC entry points and chat build, carries no backend, so nothing picks one implicitly. TextRequestProcessor holds the configured backend, and text lowering, which HTTP and gRPC share, attaches it and validates the final constraint against it. gRPC no longer selects a backend itself.

Why one backend per server, not per request. The engine still holds one backend, so a per-request choice would bring the order dependence back. Python's auto fallback has the same limit in a mixed workload. Engine-side multi-backend support (#62246, draft) would allow request-level selection later; a single configured backend maps onto a one-element list then.

Tradeoffs.

• HTTP constraints that used guidance now use xgrammar by default. Schemas with keywords xgrammar ignores (multipleOf, uniqueItems/contains, unsupported string formats, pattern with length bounds, ...) return 400 instead of being enforced by guidance; --structured-outputs-config '{"backend": "guidance"}' restores that. llguidance does not implement uniqueItems either: Python auto rejects it in guidance validation, while the Rust frontend on main returns an empty 200 when the matcher errors in EngineCore.
• Lark grammars now compile with xgrammar's Lark support ([Structured Outputs] Parse Lark grammars natively in the xgrammar backend #58321); llguidance-only extensions need backend=guidance.
• Schemas composed into a parser grammar are not checked for xgrammar-unsupported keywords, as in Python.
• vllm-rs serve reads only the JSON form of --structured-outputs-config; the dotted form (--structured-outputs-config.backend guidance) goes to the Python engine only, like other dotted flags on that path. The vllm serve path reads both.

Not a duplicate.

• [Bugfix][Rust Frontend] Use xgrammar for gRPC structured outputs #61892 (merged) moved gRPC to xgrammar and named this change as its follow-up.
• [Rust Frontend] Make unhandled vllm serve frontend flags explicit #62431 (open) no longer touches --structured-outputs-config; the two PRs are independent.
• [Bugfix] Reject mixed structured output backends during validation #49625 (merged) makes EngineCore reject a request whose backend differs from the one it already uses, with RequestError(code="validation_error"). On main the Rust frontend's mixed traffic now hits that rejection instead of KeyError: 'triggers' (still a 500 in Rust, which does not map that code yet); with this PR every Rust request names the same backend, so the rejection is not reached. [Bugfix][Structured Output] Honor resolved backends in mixed requests #62246 (draft) would let EngineCore hold several backends. This PR changes only what the Rust frontend sends.
• [Bugfix][Structured Outputs] Support new-format structural tags in the guidance backend (strict/required/named tool calling) #55153 (open) teaches guidance some format-shape structural tags; with it, more guidance requests would pass the new check, which can be relaxed then.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

AI assistance (Claude) was used to investigate, write, and test this change. I reviewed every changed line and ran the tests and GPU smoke above.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @BugenZhao.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @BugenZhao.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[JohnZhao01]

We hit this on DeepSeek-V4.1-Flash (4×H200, TP4, Rust frontend, --reasoning-parser deepseek_v41 --tool-call-parser deepseek_v41, thinking on by default) and tested this PR there, so here is a second model next to the Qwen3-8B check in the description.

This PR (nightly-e68efc68 + 817cdef, which applies without conflicts; no other patches). A fresh server per row, 10 requests per step:

Server	Order	first constraint	tool_choice named	tool_choice: required	json_schema
e68efc6 as is	json_schema first	json_schema: 200, 10/10 valid	200, 10/10 valid calls	200, 10/10 valid calls	200, 10/10 valid
e68efc6 as is	/v1/completions regex first	regex: 200	HTTP 500, 10/10	HTTP 500, 10/10	HTTP 500, 10/10
this PR, default (xgrammar)	json_schema first	json_schema: 200, 10/10 valid	200, 10/10 valid calls	200, 10/10 valid calls	200, 10/10 valid
this PR, default (xgrammar)	/v1/completions regex first	regex: 200	200, 10/10 valid calls	200, 10/10 valid calls	200, 10/10 valid
this PR, backend=guidance	json_schema first	json_schema: 200, 10/10 valid	200, 10/10 valid calls	200, 10/10 valid calls	200, 10/10 valid
this PR, backend=guidance	/v1/completions regex first	regex: 200	200, 10/10 valid calls	200, 10/10 valid calls	200, 10/10 valid

• On e68efc6 as is, the regex request pins the engine backend to guidance, and the 30 tool-call and json_schema requests after it fail in serialize_guidance_grammar with KeyError: 'triggers'. With this PR both orders return 200 on both backends, and the server logs have no grammar compile errors.
• On this build a json_schema request with a reasoning parser is composed into the parser's output grammar and sent as xgrammar, as the description says, so it does not pin the engine to guidance; that is why the json_schema-first order passes here and the repro starts with a native /v1/completions constraint, as in the description.
• With backend=guidance, chat builds no parser output grammar, so the named and required tool calls above were parsed without a grammar. DeepSeek-V4.1 still produced 10/10 valid calls in both orders, which supports the trade-off described for non-xgrammar backends on this model.
• The /v1/completions regex requests return text that does not match the regex on all three servers: our prompts do not close the think block, so as the description notes, the reasoning gate never applies the constraint. They still reach the engine first and pin its backend, which is all they are used for here.

An older build, for context (nightly-92044241 with #55153 applied, plus our own fixes #62189 / #62290, unrelated to structured outputs). A fresh server per row:

Order of structured requests after startup	response_format json_schema	tool_choice named	tool_choice: required
json_schema first, then tool calls	200, 40/40 valid	HTTP 500, 10/10	HTTP 500, 10/10
tool calls first, then json_schema, then tool calls again	200, 40/40 valid	200, 20/20 valid calls	200, 20/20 valid calls

On that build the json_schema request reaches the engine as a guidance constraint and pins the engine backend to guidance. The engine error in the first row is then structural tag format type 'sequence' is not supported by the guidance backend: with thinking on, the DeepSeek-V4.1 parser builds a sequence structural tag (the reasoning block up to </think>, then the DSML call). #55153 only changes the error from KeyError: 'triggers' to this one, so relaxing the new guidance check after #55153 would not help this model: its tags use sequence, which guidance still rejects.

[BugenZhao]

/ci run

[github-actions]

✅ Triggered Buildkite CI #97196 for commit 76522ab6b44e.