[Performance]: Cache group-invariant DSA-CP drafting metadata across kv-cache groups

[frankie-ys]

What this PR does / why we need it?

Only format_dsa_slot_mapping and build_dspark_swa_indices (which depend on each group's block_table / slot_mapping) genuinely differ per group, so the two calls cannot be merged into one. Instead, this PR makes the second call nearly free by reusing the existing common_ratio_to_sas_metadata cross-group sharing convention and computing every group-invariant sub-result once per step.

Key changes:

• LLMBaseProposer.build_draft_attn_metadata: creates one shared dict before the group loop and passes it unconditionally to every group's build_for_drafting (DSpark branch) and build() (non-DSpark branch). Previously the build() branch was gated on use_compress and created a fresh dict per group, which defeated build()'s built-in cross-group cache and did not satisfy its assert common_ratio_to_sas_metadata is not None.
• AscendDSACPMetadataBuilder.build_for_drafting / build_req_metadata_for_drafting: consume the shared dict as a cross-group cache isolated by ("draft_for_drafting", draft_index), reusing split statistics, input_positions/cos/sin, device/CPU local token metadata, max_local_*, local_cos/local_sin, start_pos, the seq_lens_cpu sync fallback, A5 cu_seqlens_ori_kv, and the sas_metadata op result (guarded by num_heads/head_dim); a debug log is emitted on cache hit.

Correctness: cached items depend only on group-invariant fields of common_attn_metadata (positions / seq_lens / query_start_loc), never on per-group block_table / slot_mapping; shared tensors are read-only consumers of the returned metadata. Paths without a shared dict (e.g. attn_update_stack_num_spec_norm, which receives per-group-updated metadata, and dummy_run, which creates a fresh dict per draft step) keep an empty cache and behave exactly as before.

dataset gsm8k
07:30:13 PM
run_single_eval-d47
prefect.task_runs
07:30:13 PM
run_single_eval-d47
prefect.task_runs
Overall report table:
+---------+-----------+----------+----------+-------+---------+---------+
| Model | Dataset | Metric | Subset | Num | Score | Cat.0 |
+=========+===========+==========+==========+=======+=========+=========+
| illm | gsm8k | mean_acc | main | 1319 | 0.9682 | default |
+---------+-----------+----------+----------+-------+---------+---------+

Does this PR introduce any user-facing change?

No. This is a pure performance optimization on the speculative-decoding metadata-building path; no API/interface/behavior change.

Gain

The second build_for_drafting time was reduced from 3.5ms to 1.2ms.

Before

After

• vLLM main: vllm-project/vllm@ba07e4a

[gemini-code-assist]

Warning

Gemini encountered an error creating the summary. You can try again by commenting /gemini summary.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[pisceskkk]

LGTM.
btw, do you have a report about acceptance ratio compared between w/ and w/o this pr?

[frankie-ys]

LGTM. btw, do you have a report about acceptance ratio compared between w/ and w/o this pr?

Thank you for your review. I have re-added the benchmark test results for DSpark's acceptance rate.
tip: install termplotlib and gnuplot to plot the metrics
============ Serving Benchmark Result ============
Successful requests: 100
Failed requests: 0
Maximum request concurrency: 3
Benchmark duration (s): 261.59
Total input tokens: 5000400
Total generated tokens: 35000
Request throughput (req/s): 0.38
Output token throughput (tok/s): 133.79
Peak output token throughput (tok/s): 64.00
Peak concurrent requests: 5.00
Total token throughput (tok/s): 19248.84
---------------Time to First Token----------------
Mean TTFT (ms): 1360.99
Median TTFT (ms): 1043.93
P99 TTFT (ms): 9252.78
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 18.53
Median TPOT (ms): 17.70
P99 TPOT (ms): 33.10
---------------Inter-token Latency----------------
Mean ITL (ms): 58.65
Median ITL (ms): 49.16
P99 ITL (ms): 622.03
----------------End-to-end Latency----------------
Mean E2EL (ms): 7828.58
Median E2EL (ms): 7259.45
P99 E2EL (ms): 17126.53
---------------Speculative Decoding---------------
Acceptance rate (%): 30.86
Acceptance length: 3.16
Drafts: 11128
Draft tokens: 77896
Accepted tokens: 24038
Per-position acceptance (%):
Position 0: 62.28
Position 1: 43.82
Position 2: 34.08
Position 3: 27.52
Position 4: 22.28
Position 5: 16.20
Position 6: 9.85

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[pisceskkk]

Acceptance rate (%): 30.86
Acceptance length: 3.16
Drafts: 11128
Draft tokens: 77896
Accepted tokens: 24038
Per-position acceptance (%):
Position 0: 62.28
Position 1: 43.82
Position 2: 34.08
Position 3: 27.52
Position 4: 22.28
Position 5: 16.20
Position 6: 9.85

I’m not sure what the baseline acceptance rate is for this model with DSpark, but this number looks a bit low to me. It might be better to include a baseline for comparison.

[frankie-ys]

Acceptance rate (%): 30.86
Acceptance length: 3.16
Drafts: 11128
Draft tokens: 77896
Accepted tokens: 24038
Per-position acceptance (%):
Position 0: 62.28
Position 1: 43.82
Position 2: 34.08
Position 3: 27.52
Position 4: 22.28
Position 5: 16.20
Position 6: 9.85

I’m not sure what the baseline acceptance rate is for this model with DSpark, but this number looks a bit low to me. It might be better to include a baseline for comparison.

Yes, I used a random dataset, so the acceptance rate will be lower. The acceptance rate results are similar for /w and /wo.

============ Serving Benchmark Result ============
Successful requests: 100

Failed requests: 0

Maximum request concurrency: 3

Benchmark duration (s): 276.57

Total input tokens: 5000400

Total generated tokens: 35000

Request throughput (req/s): 0.36

Output token throughput (tok/s): 126.55

Peak output token throughput (tok/s): 64.00

Peak concurrent requests: 5.00

Total token throughput (tok/s): 18206.57

---------------Time to First Token----------------
Mean TTFT (ms): 1332.33
Median TTFT (ms): 1059.39
P99 TTFT (ms): 7873.03
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 19.50
Median TPOT (ms): 17.83
P99 TPOT (ms): 38.47
------------------Inter-token Latency----------------
Mean ITL (ms): 58.71
Median ITL (ms): 49.84
P99 ITL (ms): 617.62
----------------End-to-end Latency----------------
Mean E2EL (ms): 8138.72
Median E2EL (ms): 7392.91
P99 E2EL (ms): 15641.78

---------------Speculative Decoding---------------

Acceptance rate (%): 28.22

Acceptance length: 2.98

Drafts: 11807

Draft tokens: 82649

Accepted tokens: 23321

Per-position acceptance (%):

Position 0: 58.32

Position 1: 39.78

Position 2: 30.63

Position 3: 24.38

Position 4: 20.23

Position 5: 14.52

Position 6: 9.66

====================================================

Our performance in real-world data testing was similar.

[frankie-ys]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[frankie-ys]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[pisceskkk]

Our performance in real-world data testing was similar.

LGTM, I have no other question, thanks for contribution!

[frankie-ys]

Our performance in real-world data testing was similar.

LGTM, I have no other question, thanks for contribution!

Thank you for your review. The current PR is failing to run successfully in CI testing on an A3 8-card CPU. Could you please take a look?

[frankie-ys]

/rerun

[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E