[DSv4 Perf] Optimize workspace reuse for eager break

[yewentao256]

Purpose

Part of #45861

Optimize workspace reuse for eager break

Test

vllm serve deepseek-ai/DeepSeek-V4-Flash --tensor-parallel-size 4 --enable-expert-parallel --attention-backend FLASHMLA_SPARSE_DSV4 --attention-config '{"use_fp4_indexer_cache":true}' --kv-cache-dtype fp8 --tokenizer-mode deepseek_v4 --all2all-backend allgather_reducescatter --port 8003

Acc

lm_eval --model  local-completions --model_args "base_url=http://127.0.0.1:8003/v1/completions,model=deepseek-ai/DeepSeek-V4-Flash,num_concurrent=1024" --tasks gsm8k
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9507|±  |0.0060|
|     |       |strict-match    |     5|exact_match|↑  |0.9515|±  |0.0059|

Perf

vllm bench serve   --model deepseek-ai/DeepSeek-V4-Flash   --dataset-name random   --host 127.0.0.1   --port 8003   --random-input-len 2   --random-output-len 2048   --request-rate inf   --max-concurrency 1   --num-prompts 4   --num-warmups 2   --ignore-eos   --temperature 0   --seed 0

# now

============ Serving Benchmark Result ============
Successful requests:                     4         
Failed requests:                         0         
Maximum request concurrency:             1         
Benchmark duration (s):                  46.96     
Total input tokens:                      8         
Total generated tokens:                  8192      
Request throughput (req/s):              0.09      
Output token throughput (tok/s):         174.45    
Peak output token throughput (tok/s):    176.00    
Peak concurrent requests:                2.00      
Total token throughput (tok/s):          174.62    
---------------Time to First Token----------------
Mean TTFT (ms):                          44.65     
Median TTFT (ms):                        44.44     
P99 TTFT (ms):                           46.31     
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          5.71      
Median TPOT (ms):                        5.71      
P99 TPOT (ms):                           5.72      
---------------Inter-token Latency----------------
Mean ITL (ms):                           5.71      
Median ITL (ms):                         5.69      
P99 ITL (ms):                            5.90      
==================================================

# main
============ Serving Benchmark Result ============
Successful requests:                     4         
Failed requests:                         0         
Maximum request concurrency:             1         
Benchmark duration (s):                  47.20     
Total input tokens:                      8         
Total generated tokens:                  8192      
Request throughput (req/s):              0.08      
Output token throughput (tok/s):         173.58    
Peak output token throughput (tok/s):    176.00    
Peak concurrent requests:                2.00      
Total token throughput (tok/s):          173.74    
---------------Time to First Token----------------
Mean TTFT (ms):                          46.41     
Median TTFT (ms):                        46.79     
P99 TTFT (ms):                           47.81     
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          5.74      
Median TPOT (ms):                        5.74      
P99 TPOT (ms):                           5.74      
---------------Inter-token Latency----------------
Mean ITL (ms):                           5.74      
Median ITL (ms):                         5.71      
P99 ITL (ms):                            5.95      
==================================================

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[tlrmchlsmth]

nit: Here we have a "output_buffers implies use_fp4", which could make for a "bumpy" user interface for this function - the function would have a nicer interface without this restriction. Either document this in the docstring or extend support

[yewentao256]

Fixed, we extend the support

[yewentao256]

In previous commit of this PR, the test passes https://buildkite.com/vllm/ci/builds/80503#019fa437-a168-432a-b60e-05bc623b2090

So the failed unit test is not related