[Feature][Spec] Support disabling trailing prefix-cache block dropping

[ZeldaHuang]

Purpose

Add an opt-in disable_eagle_block_drop speculative-decoding option for
EAGLE-family methods, including dSpark. When enabled, vLLM keeps the trailing
prefix-cache block instead of conservatively dropping it after speculative-model
prefill.

This does not bypass target-model verification. The option can change which
draft tokens are proposed and therefore may affect speculative-token acceptance
rates, but accepted output tokens are still verified by the target model.

Test Plan

Run an A/B prefix-cache benchmark with the following configuration:

• 2 nodes x 4 NVIDIA GB300 GPUs, tensor parallel size 8
• moonshotai/Kimi-K3 MXFP4 weights
• Inferact/Kimi-K3-DSpark
• 7 speculative tokens, greedy draft sampling, and FLASHINFER_MLA
• 12,288-token shared prefix, 256-token suffix, 160 requests, and concurrency 16

Run the following command on both nodes. Set NODE_RANK=0 on the API node and
NODE_RANK=1 on the worker node; only the worker uses --headless.

NODE_RANK=${NODE_RANK:?set NODE_RANK to 0 or 1}
MASTER_ADDR=${MASTER_ADDR:?set MASTER_ADDR to the rank-0 host}
HEADLESS=()
(( NODE_RANK > 0 )) && HEADLESS=(--headless)

vllm serve moonshotai/Kimi-K3 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --nnodes 2 \
  --node-rank "$NODE_RANK" \
  --master-addr "$MASTER_ADDR" \
  --served-model-name moonshotai/Kimi-K3 \
  --gpu-memory-utilization 0.92 \
  --max-model-len 32768 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 16384 \
  --moe-backend auto \
  --quantization-config.moe.activation mxfp8 \
  --kv-cache-dtype fp8 \
  --attention-config '{"use_prefill_query_quantization":true,"mla_prefill_backend":"FLASHINFER"}' \
  --enable-prefix-caching \
  --speculative-config '{"method":"dspark","model":"Inferact/Kimi-K3-DSpark","num_speculative_tokens":7,"attention_backend":"FLASHINFER_MLA","draft_sample_method":"greedy","disable_eagle_block_drop":true}' \
  --enforce-eager \
  "${HEADLESS[@]}"

For the default baseline, use the same command but omit
"disable_eagle_block_drop": true.

After resetting the prefix cache, send one untimed request to populate the
12,288-token shared prefix, capture the Prometheus counters, and then run the
timed benchmark:

vllm bench serve \
  --backend openai \
  --base-url http://${MASTER_ADDR}:8000 \
  --model moonshotai/Kimi-K3 \
  --tokenizer moonshotai/Kimi-K3 \
  --trust-remote-code \
  --dataset-name prefix_repetition \
  --num-prompts 160 \
  --prefix-repetition-prefix-len 12288 \
  --prefix-repetition-suffix-len 256 \
  --prefix-repetition-num-prefixes 1 \
  --prefix-repetition-output-len 1 \
  --disable-shuffle \
  --request-rate inf \
  --max-concurrency 16 \
  --temperature 0 \
  --ignore-eos \
  --percentile-metrics ttft,e2el \
  --metric-percentiles 50,90,99

The shared prefix is aligned to Kimi-K3's 1,536-token Mamba state interval.
With the default behavior, the final aligned block is dropped, so each timed
request re-computes 1,536 additional prompt tokens.

For acceptance rate, run a second A/B on the complete GSM8K test split. The
official openai/gsm8k test parquet was exported one-to-one to ShareGPT JSON
because the runtime image does not include the optional Hugging Face datasets
dependency. Both variants use all 1,319 questions in the same order, a maximum
output length of 256, concurrency 16, temperature 0, and EOS handling enabled.
Run one untimed request before capturing the Prometheus counters, then run:

vllm bench serve \
  --backend openai \
  --base-url http://${MASTER_ADDR}:8000 \
  --model moonshotai/Kimi-K3 \
  --tokenizer moonshotai/Kimi-K3 \
  --trust-remote-code \
  --dataset-name sharegpt \
  --dataset-path gsm8k-test-sharegpt.json \
  --sharegpt-output-len 256 \
  --num-prompts 1319 \
  --no-oversample \
  --disable-shuffle \
  --request-rate inf \
  --max-concurrency 16 \
  --temperature 0 \
  --percentile-metrics ttft,tpot,e2el \
  --metric-percentiles 50,90,99

Test Result

All timed prefix-cache requests succeeded (160/160) in both variants.

Metric	Default drop	disable_eagle_block_drop=true	Delta
Cached tokens per request	10,752	12,288	+1,536
Locally computed tokens per request	1,792	256	-1,536
Total cached tokens	1,720,320	1,966,080	+245,760
Total locally computed tokens	286,720	40,960	-245,760
Request throughput	12.720 req/s	25.556 req/s	+100.91%
Mean TTFT	1,222.00 ms	604.88 ms	-50.50%
P50 TTFT	1,225.40 ms	566.73 ms	-53.75%
P90 TTFT	1,227.04 ms	746.72 ms	-39.14%

All GSM8K requests also succeeded (1,319/1,319) in both variants. Timed-pass
Prometheus counter deltas matched the benchmark summary:

GSM8K metric	Default drop	disable_eagle_block_drop=true	Delta
Draft steps	84,278	84,436	+158
Draft tokens	589,946	591,052	+1,106
Accepted tokens	239,307	240,658	+1,351
Acceptance rate	40.564%	40.717%	+0.153 pp (+0.376% relative)
Mean acceptance length	3.84	3.85	+0.01

No acceptance-rate regression was observed on the complete GSM8K test split;
the no-drop variant increased acceptance by 0.153 percentage points. GSM8K's
independent prompts produced a 0% prefix-cache hit rate, so this result checks
for a general acceptance regression rather than the long-prefix cache-hit path.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ZeldaHuang.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[ivanium]

LGTM. cc @WoosukKwon @njhill

[ZJY0516]

Let's merge this first to unblock perf

[ZJY0516]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86514 for commit 39b9a8596d5a.

[ZeldaHuang]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86534 for commit 2f5aefeb0b5c.

[ZeldaHuang]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86546 for commit 412e47943c5d.