[Bugfix][Structured Output][Spec Decode] Advance grammar same-step for all structured types at the reasoning boundary

[vvsotnikov]

Purpose

Fixes #48228 (structured output under speculative decoding starts with a doubled
opening token, e.g. {{). Fixes #34650 (structured output under speculative
decoding runs entirely unconstrained with delta-sensitive reasoning parsers). Both
are reasoning-boundary bugs addressed by the two changes below.

1. Same-step advance is gated to STRUCTURAL_TAG; other structured types fall one token behind

When reasoning ends mid-step under speculative decoding, the same step can also
accept post-marker content — a { drafted right after </think> and verified in
the same step (grammar_bitmask() already constrains those positions since #44297).
But should_advance() returns True same-step only for structural tags; for
JSON/regex/choice/grammar the advance is deferred, so the accepted { never reaches
accept_tokens(). The persisted grammar stays one token behind the sampled stream,
and the next step's bitmask — computed from the un-advanced grammar — forces a
duplicate of the token the model already emitted: {{. (This is the persisted-state
counterpart of the transient window handling hardened in #44297: its in-window
advances are rolled back by design, so the persisted grammar must catch up here.)

Fix: extend the same-step trimmed advance to all structured output types under
speculative decoding. trim_reasoning_for_advance() already drops everything up to
and including the marker, so the grammar never sees reasoning content and #44006
stays fixed.

2. should_advance() reconstructs the step window from pre-incremented counters

should_advance() derives "this step's tokens" as
all_token_ids[num_computed_tokens - num_output_placeholders:], but with
speculative decoding num_computed_tokens is already advanced past the accepted
draft tokens when update_from_output() runs, so the window can miss the
reasoning-end marker entirely for delta-sensitive streaming detectors (#34650 — the
BaseThinkingReasoningParser family, e.g. deepseek_r1), and it mislocates the
marker index needed for trimming with full-scan detectors (the current qwen3
adapter).

Fix: pass the call site's actual new_token_ids into should_advance(), used both
as the detection delta and to locate the marker within the true step window. The
counter arithmetic remains as a fallback for call sites without token context (the
two draft-validation sites, which run after update_from_output() and inherit the
persisted reasoning_ended).

Test Plan

• 3 new unit tests in tests/v1/structured_output/test_reasoning_structured_output.py:
  same-step advance for JSON under spec decode; detection via explicit
  new_token_ids where the counter-derived window is empty; a straddle step
  [</think>, "{"] advancing the grammar with exactly the post-marker suffix.
• pytest tests/v1/structured_output/test_reasoning_structured_output.py
• E2E ([Bug]: response_format emits a doubled leading brace "{{" with reasoning enabled (Qwen3.6, --reasoning-parser qwen3); guided_json and enable_thinking=false are unaffected #48228 config): Qwen3.6-27B (MTP weights preserved), TP=2,
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder --speculative-config '{"method":"mtp","num_speculative_tokens":2}' --structured-outputs-config '{"backend": "xgrammar", "disable_any_whitespace": true}'; 10 concurrent requests
  per cell over 5 JSON schemas (3 from the structured-outputs docs + 2 production
  ones) × strict on/off, plus forced / required / auto tool calls.
• E2E repro of Bug: Speculative Decoding (MTP) Causes </think> Detection Failure in Structured Output + Reasoning Mode #34650: same model and flags but --reasoning-parser deepseek_r1
  (delta-sensitive; its </think> string resolves in the Qwen3.6 vocab), stock
  v0.25.0 vs this patch, 5 runs per schema × strict cell.

Test Result

• Unit: 12/12 in the file (9 pre-existing, including the structural-tag same-step
  test, + 3 new).
• E2E ([Bug]: response_format emits a doubled leading brace "{{" with reasoning enabled (Qwen3.6, --reasoning-parser qwen3); guided_json and enable_thinking=false are unaffected #48228 config) before: ~50% of response_format json_schema responses
  invalid (doubled {{), including the simple schemas from the structured-outputs
  docs. After: 100/100 valid across all cells; tool calls unaffected (10/10).
• Bug: Speculative Decoding (MTP) Causes </think> Detection Failure in Structured Output + Reasoning Mode #34650 A/B with the delta-sensitive parser: stock v0.25.0 free-generates under MTP
  (e.g. MathResponse 1/5 valid, a 6-field production schema 0/5; failures are
  invented keys or markdown-fenced JSON with finish_reason=stop), while this patch
  yields 50/50 valid across all schema × strict cells.

AI disclosure: developed with AI assistance (Claude Code); the submitter
reviewed all changed lines and validated behavior end-to-end on the deployments
described above. Commits carry Co-authored-by attribution.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[vvsotnikov]

Already fixed