[BugenZhao]

Overview

With the default structured_outputs_config.backend="auto", each request picks its backend from its own content: xgrammar first, then guidance or outlines when xgrammar rejects the constraint. EngineCore keeps a single backend, the one the first structured request chose, so a request's outcome depends on the traffic before it. Before #49625 a request that resolved to the other backend was compiled by the wrong one, silently dropping constraints or failing with HTTP 500; since #49625 it is a 400. This PR makes auto choose one backend per server from the tokenizer alone, so every request on a server names the same backend and the result no longer depends on request order.

Claims

• auto resolves to guidance for Tekken-based Mistral tokenizers and to xgrammar for every other tokenizer, for every request. It no longer falls back per request.
• A constraint the selected backend cannot enforce (for example a JSON schema with multipleOf under xgrammar) is rejected with the backend's validation error (400), on a fresh server and after any other traffic alike. Selecting guidance explicitly restores it.
• On Tekken Mistral servers, the Lark grammars the Mistral tool parser generates and the server's JSON schema and regex requests all use guidance, instead of mixing guidance (Lark) and xgrammar (everything else).

Validation

• tests/v1/structured_output/ and tests/test_sampling_params.py: 325 passed, 1 xfailed. New tests: test_auto_backend_rejects_schema_without_fallback (a supported schema resolves to xgrammar; multipleOf and patternProperties schemas are rejected instead of falling back) and test_auto_selects_guidance_for_tekken_mistral. All three fail on main.
• tests/entrypoints/llm/test_struct_output_generate.py::test_structured_output_auto_mode (GB200, Qwen2.5-1.5B-Instruct and Ministral-8B-Instruct-2410): 2 passed. The test now expects auto to reject UNSUPPORTED_JSON_SCHEMA (multipleOf) unless the tokenizer is Tekken Mistral, where it generates valid JSON with guidance. With tokenizer_mode=mistral, Ministral-8B loads a Tekken MistralTokenizer and auto resolves to guidance; I did not confirm which tokenizer the test's tokenizer_mode=auto loads.
• pre-commit on the changed files, including mypy.

Details

Why per request is the wrong granularity. StructuredOutputManager builds one backend from the first structured request and #49625 now rejects requests naming another one. Under the old auto, a server receiving both an xgrammar-compatible schema and a multipleOf schema answered the second one differently depending on which came first:

First request	Then a multipleOf schema (resolves to guidance)	Then a plain schema (resolves to xgrammar)
plain schema (xgrammar)	400 (backend mismatch)	200
multipleOf schema (guidance)	200	400 (backend mismatch)

The tokenizer is fixed for the server's lifetime, so resolving from it gives one backend per server, consistent with what EngineCore can serve.

Why Tekken Mistral selects guidance. The Mistral tool parser emits llguidance-dialect Lark for grammar-capable Mistral tokenizers, and the xgrammar backend declares no special tokens for Mistral tokenizers, so a regex like . can match [TOOL_CALLS] as text (#58321). Before this PR only those Lark grammars went to guidance; JSON schema and regex requests on the same server went to xgrammar, so Mistral servers mixed backends too. Guidance handles both.

Behavior change. Requests that relied on the fallback are now rejected under auto:

• JSON schemas with keywords xgrammar does not enforce (multipleOf, uniqueItems/contains, unsupported string formats, pattern with length bounds, ...), and regexes or grammars xgrammar cannot compile. They previously worked only while the engine had not latched xgrammar.
• Non-Tekken Mistral requests that fell back to outlines.

--structured-outputs-config.backend guidance (or outlines) selects those backends for the whole server. The removed fallback also leaves has_guidance_unsupported_json_features unused, so it is deleted.

Migration. This changes the default behavior, so it could instead first ship one release in which auto still falls back but logs a deprecation warning naming the backend to configure. The rejection message is the selected backend's own; an auto-specific hint pointing to --structured-outputs-config.backend could be added.

Rust frontend. #62456 gives the Rust frontend the same model: one configured backend, auto meaning xgrammar. The Rust frontend does not load Mistral tokenizers, so the Tekken rule does not apply there.

Not a duplicate. No open PR changes how auto resolves. #49625 (merged) rejects mismatched backends in EngineCore; #62246 (draft) would let EngineCore hold several backends, which would allow request-level selection later.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

AI assistance (Claude) was used to investigate, write, and test this change. I reviewed every changed line and ran the tests above.

🤖 Generated with Claude Code

[mergify]

Documentation preview: https://vllm--62877.org.readthedocs.build/en/62877/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @BugenZhao.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork