Fix spec v2 stop output boundary

[gq112]

Motivation

This PR fixes a stop boundary issue under spec v2. When spec v2 accepts multiple tokens in one step, a stop string may appear before the end of the accepted tokens. In that case, tokens after the matched stop should not be emitted.

This is an enhancement to #23802: after the stop match is detected, this PR ensures the final emitted output is cut at the actual stop boundary.

Modifications

• Track the actual stop boundary with finished_len for string/regex stop matches.
• Emit token ids only through output_ids_through_stop.
• For no_stop_trim=true, keep the matched stop string but still trim text after it.

Accuracy Tests

Server:

CUDA_VISIBLE_DEVICES=2,3 python3 -m sglang.launch_server \
  --port 9000 \
  --tp 2 \
  --model /models/Qwen3-14B \
  --context-length 24000 \
  --mem-fraction-static 0.7 \
  --log-requests-level 3 \
  --max-running-requests 10 \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path /models/EAGLE3-Qwen3-14B

Request uses a MATH500 sample :

curl -X POST http://127.0.0.1:9000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "/models/Qwen3-14B",
    "messages": [
      {
        "role": "user",
        "content": "Let \\mathbf{A} be a matrix such that \n[\\mathbf{A} \\begin{pmatrix} 3 \\\\ 1 \\\\ 0 \\end{pmatrix} = \\begin{pmatrix} 2 \\\\ 4 \\\\ -3 \\end{pmatrix} \\quad \\text{and} \\quad \\mathbf{A} \\begin{pmatrix} -5 \\\\ 2 \\\\ 2 \\end{pmatrix} = \\begin{pmatrix} 0 \\\\ -5 \\\\ -5 \\end{pmatrix}].\nFind \\mathbf{A} \\begin{pmatrix} -13 \\\\ 3 \\\\ 4 \\end{pmatrix}. Please reason step by step, and put your final answer within \\\\boxed{}."
      }
    ],
    "temperature": 0,
    "stop": ["given"],
    "no_stop_trim": true,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

Before this PR:

{
  "content": "We are given that",
  "finish_reason": "stop",
  "matched_stop": "given",
  "completion_tokens": 4
}

With this PR:

{
  "content": "We are given",
  "finish_reason": "stop",
  "matched_stop": "given",
  "completion_tokens": 3
}

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ⏳ Run #27190443615
Latest PR Test (Extra): ❌ Run #27190443069

[gemini-code-assist]

Code Review

This pull request enhances stop sequence detection and handling, specifically for speculative decoding scenarios where multiple tokens are accepted simultaneously. It introduces logic to accurately determine the finish length when a stop string or regex is matched and updates the detokenizer to support non-trimming of stop sequences. A performance optimization was suggested for the stop sequence location logic to minimize expensive decoding operations by narrowing the search window based on the number of newly accepted tokens.

[hnyls2002]

/tag-and-rerun-ci