[Frontend] Improve the performance of is_reasoning_end

[chaunceyjiang]

Purpose

is_reasoning_end is executed once for every token generated during the reasoning phase, which results in poor performance.
Optimize it to perform incremental checks instead.

Currently, is_reasoning_end cannot support incremental checking for the following reason:

When stream=true, is_reasoning_end checks output.token_ids and res.prompt_token_ids separately. This leads to non-deterministic inputs to is_reasoning_end within the same request, making incremental checking impossible.

Therefore, the only feasible optimization at present is to modify is_reasoning_end to search backward for the end_token from the end of the token sequence.

Test Plan

vllm serve /home/jovyan/qwen3-8b  --reasoning-parser qwen3 --guided-decoding-backend xgrammar --enable-auto-tool-choice --tool-call-parser hermes --no-enable-prefix-caching

vllm bench serve \   
  --backend vllm \
  --model /home/jovyan/qwen3-8b \
  --served-model-name /home/jovyan/qwen3-8b \
  --endpoint /v1/completions \
  --dataset-name random \
  --random-input 2048 \
  --random-output 1024 \
  --max-concurrency 10 \
  --num-prompt 100

main

============ Serving Benchmark Result ============
Successful requests:                     100       
Maximum request concurrency:             10        
Benchmark duration (s):                  95.70     
Total input tokens:                      204800    
Total generated tokens:                  96051     
Request throughput (req/s):              1.04      
Output token throughput (tok/s):         1003.69   
Peak output token throughput (tok/s):    1150.00   
Peak concurrent requests:                20.00     
Total Token throughput (tok/s):          3143.74   
---------------Time to First Token----------------
Mean TTFT (ms):                          311.72    
Median TTFT (ms):                        282.02    
P99 TTFT (ms):                           525.12    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          9.56      
Median TPOT (ms):                        9.17      
P99 TPOT (ms):                           11.46     
---------------Inter-token Latency----------------
Mean ITL (ms):                           9.13      
Median ITL (ms):                         8.88      
P99 ITL (ms):                            10.55     
==================================================

this pr

============ Serving Benchmark Result ============
Successful requests:                     100       
Maximum request concurrency:             10        
Benchmark duration (s):                  95.65     
Total input tokens:                      204800    
Total generated tokens:                  96316     
Request throughput (req/s):              1.05      
Output token throughput (tok/s):         1006.96   
Peak output token throughput (tok/s):    1150.00   
Peak concurrent requests:                20.00     
Total Token throughput (tok/s):          3148.09   
---------------Time to First Token----------------
Mean TTFT (ms):                          316.62    
Median TTFT (ms):                        284.75    
P99 TTFT (ms):                           529.61    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          9.57      
Median TPOT (ms):                        9.12      
P99 TPOT (ms):                           11.55     
---------------Inter-token Latency----------------
Mean ITL (ms):                           9.11      
Median ITL (ms):                         8.87      
P99 ITL (ms):                            10.53     
==================================================

Test Result

The performance improvement is very marginal.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @chaunceyjiang.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[chatgpt-codex-connector]

💡 Codex Review

https://github.com/vllm-project/vllm/blob/0f3384862593027cd7ea8433c9c5314536011ecc/vllm/reasoning/basic_parsers.py#L60-L66
Referencing undefined token attribute in incremental end check

The new incremental is_reasoning_end logic now compares tokens against self.think_end_token_id, but BaseThinkingReasoningParser only defines start_token_id/end_token_id. Subclasses like DeepSeekR1ReasoningParser and SeedOSSReasoningParser never create a think_end_token_id, so the first call to is_reasoning_end will raise AttributeError and break reasoning parsing for those models. The loop should use the existing self.end_token_id instead.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

[chaunceyjiang]

/cc @njhill Due to the various uses of is_reasoning_end, it is not possible to perform incremental checks for the end_token. Currently, the only optimization is to search from the end backwards.

[DarkLight1337]

Can you edit the PR description accordingly?

[chaunceyjiang]

@DarkLight1337 I've updated the PR description.

this optimization provides only a very small performance improvement. 😂