[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill by TheEpicDolphin · Pull Request #53694 · vllm-project/vllm

TheEpicDolphin · 2026-08-25T07:16:53Z

Summary

Currently, the V2 model runner speculator for EAGLE/MTP performs a CPU all-reduce (in sync_cudagraph_and_dp_padding) to synchronize the num_tokens, uniform_token_count, cg_mode, etc, across DP ranks before each of the draft prefill (drafts first token), and the draft multi-step decode (drafts N-1 tokens). Similarly, the multi-module MTP speculator does the DP-sync before the mutl-step drafting loop (drafts all N tokens). The CPU all-reduce adds significant CPU overhead during serving.

This PR

Passes the DP-synced values (number of tokens, uniform token count, eager) from the model runner's call to dispatch_cg_and_sync_dp before the target forward, and reuses it for draft prefill during EAGLE/MTP and before the draft loop during multi-module MTP. This allows skipping a CPU all-reduce, and directly fetching the correct batch descriptor using the DP-synced values.

Evals/Benchmarks

Setup. GSM8K (1319 questions), dp=2, tp=2, --enable-expert-parallel on 4×GB200.
Perf via vllm bench serve --dataset-name custom --dataset-path gsm8k.jsonl --backend openai-chat --request-rate inf --max-concurrency 64 --num-warmups 64 --num-prompts 1319 --custom-output-len 2048 --temperature 0 --top-p 1.0.
Perf figures are the median of 3 runs against one server; accuracy is a single
full-dataset pass. VLLM_USE_BREAKABLE_CUDAGRAPH=0 on both sides (its capture asserts in
PyTorch's CachingHostAllocator under dp>1, independent of this change).

DeepSeek-V4-Flash + MTP=3 (`AutoRegressiveSpeculator`)

metric	before	after	Δ
GSM8K accuracy	0.948	0.950	+0.002
Invalid responses	0.000	0.000	—
Output token throughput (tok/s)	2946.74	3024.36	+2.6%
Mean TPOT (ms)	20.68	20.14	−2.6%
Median TPOT (ms)	21.02	20.57	−2.1%
P99 TPOT (ms)	25.84	24.95	−3.4%
Mean TTFT (ms)	365.04	345.02	−5.5%
Acceptance rate	60.19%	60.17%	−0.02pp
Acceptance length	2.81	2.80	−0.01
Per-position acceptance	89.00 / 63.06 / 28.50	88.96 / 63.04 / 28.52	flat

individual runs (output tok/s)

before: 2946.74, 2935.29, 2950.43
after:  3003.74, 3024.36, 3059.22

Inkling-NVFP4 + MTP=8 (`MultiModuleMTPSpeculator`)

metric	before	after	Δ
GSM8K accuracy	0.899	0.894	−0.005
Invalid responses	0.000	0.000	—
Output token throughput (tok/s)	1763.53	1809.68	+2.6%
Mean TPOT (ms)	31.50	30.39	−3.5%
Median TPOT (ms)	31.43	30.32	−3.5%
P99 TPOT (ms)	44.63	43.56	−2.4%
Mean TTFT (ms)	1042.77	998.50	−4.2%
Acceptance rate	42.98%	42.81%	−0.17pp
Acceptance length	4.44	4.42	−0.02

Per-position acceptance (median of 3):

pos	0	1	2	3	4	5	6	7
before	75.91	60.32	50.29	41.73	35.91	30.29	26.49	22.81
after	75.84	60.11	50.10	41.46	35.61	30.14	26.41	22.76

individual runs (output tok/s)

before: 1759.79, 1776.35, 1763.53
after:  1809.68, 1876.91, 1789.58

In summary, accuracy and acceptance are unchanged. Throughput gains are consistent but modest (~2.6% tok/s improvment on both models).

Profile

Deepseek-V4-Flash + MTP 3

Before

After

Inkling-NVFP4 Multi-layer MTP 8

Before

After

TheEpicDolphin · 2026-08-25T20:53:06Z

+    # If ranks disagree on the uniform token count, or its 0 (means None) set to None
+    synced_uniform_token_count: int | None = int(uniform_token_counts_across_dp[0])
+    if synced_uniform_token_count == 0 or not torch.all(
+        uniform_token_counts_across_dp == synced_uniform_token_count
+    ):
+        synced_uniform_token_count = None
+


Moved up from the bottom of this method because synced_uniform_token_count can now be returned in the eager branch.

…efill Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

GirasoleY · 2026-08-25T23:51:33Z

It looks like we currently skip draft collectives for uniform batches. I feel we should extend this for general case. i.e. have target model carry a reusable DP state, from which the draft model’s token shape can be derived.

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

TheEpicDolphin · 2026-08-26T02:26:57Z

Thanks @GirasoleY, I updated the PR so that it skips the draft prefill DP-sync collective for all batches types (eager, mixed, uniform decode) for MTP/EAGLE speculators. I know pass a DPSync object to the speculator containing all the sync state it needs from the target model's. Hope this addresses your comment!

GirasoleY

LGTM!

GirasoleY · 2026-08-26T20:06:09Z

+    if dp_sync is not None:
+        assert dp_sync.num_tokens[dp_rank] == num_tokens, (
+            "reusing a DP sync taken over a different batch"
+        )
+        assert (
+            dp_sync.uniform_token_count is None
+            or uniform_token_count == dp_sync.uniform_token_count
+        ), "reusing a DP sync taken over a different batch"
+        if not dp_sync.eager and batch_desc.num_tokens != num_tokens:
+            # Capture sizes can differ between managers, so this one may
+            # pad further. Every rank pads alike, so report what will run.
+            dp_sync = replace(
+                dp_sync,
+                num_tokens=torch.full_like(dp_sync.num_tokens, batch_desc.num_tokens),
+            )
+        return batch_desc, dp_sync
+


Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

TheEpicDolphin · 2026-08-26T21:37:39Z

/ci run

github-actions · 2026-08-26T21:37:58Z

✅ Triggered Buildkite CI #85707 for commit 3ff7aa901548.

WoosukKwon · 2026-08-26T23:10:42Z

@TheEpicDolphin The CI failure on buildkite/ci/pr/nvidia-h200-v1-core-plus-kv-plus-metrics

FAILED v1/worker/test_gpu_model_runner_v2_eplb.py::test_v2_sample_tokens_runs_eplb_on_non_last_pp_rank - AttributeError: 'types.SimpleNamespace' object has no attribute 'dp_sync'

Seems to be because of this PR. Can you please fix it?

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

TheEpicDolphin · 2026-08-27T00:26:21Z

/ci run

github-actions · 2026-08-27T00:26:38Z

✅ Triggered Buildkite CI #85719 for commit 55a4cc9b0045.

…efill (vllm-project#53694) Signed-off-by: khushali9 <khushali.desai9@gmail.com>

…efill (vllm-project#53694)

…efill (vllm-project#53694) Signed-off-by: mikeshawcode <michaelwshaw2@gmail.com>

…efill (vllm-project#53694)

### What this PR does / why we need it This PR upgrades the verified vLLM main anchor from [`ba07e4a48fc951300d97eb506217dd530583dea3`](vllm-project/vllm@ba07e4a) to [`e6bfe03ad73a3330cb427885aa90d97a12e1c704`](vllm-project/vllm@e6bfe03). The exact upstream range is [ba07e4a48...e6bfe03ad](vllm-project/vllm@ba07e4a...e6bfe03). The branch is rebased onto the latest vllm-ascend `origin/main`. The current revision adds two follow-ups on top of the reviewed source mapping: - **`e67948b9c` - drop the local `pr_test.yaml` tweak.** The earlier "raise e2e timeout for `ready-all` partitions" change is reverted, so this PR no longer modifies `.github/workflows/pr_test.yaml`. - **`fe7f550b7` - drop the vLLM `v0.27.1` release lane from the tests.** Every `vllm_version_is("0.27.1")` gate in the unit/e2e suite is removed and each site resolves to the vLLM-main behavior: dual-lane if/else/ternary branches collapse to the main path, the v0.27.1-only skip and the `test_kimi_k3_gqa_mixed_groups_use_expected_physical_layout` test are deleted, the profiling-time and `prepare_inputs` AST contract tests are reshaped to the single main-lane implementation, dead `vllm_version_is` mocks/imports are dropped, the e2e `hunyuan-vl` case always skips, the obsolete `VLLM_VERSION=0.27.1` hack in `test_num_nans` is removed, and the orphaned legacy `_get_kv_cache_config_deepseek_v4` planner is deleted. `vllm_version_is()` stays in `vllm_ascend/utils.py` with its unit test. #### Review conclusion - **Latest revision (`fe7f550b7`):** clean rebase onto current `origin/main`; the two follow-ups above are committed and pushed. Local-only, non-PR working-tree sources are not part of this branch. - **Source review (earlier revisions):** the main2main adaptations for the pinned upstream range were reviewed and are documented below; no PR-introduced source-level blocker was found. - **CI:** the run triggered on the rebased head (`fe7f550b7`) supersedes the earlier run and is the authoritative gate for this revision. #### Upstream changes covered | Upstream PR | Exact commit | Contract adopted here | |---|---|---| | [#50465](vllm-project/vllm#50465) | [`d154d90d6c`](vllm-project/vllm@d154d90) | Batch-sharded sampling and `skip_gather` | | [#51718](vllm-project/vllm#51718) | [`8bdc70ec7b`](vllm-project/vllm@8bdc70e) | Standardized KV-cache layout | | [#52209](vllm-project/vllm#52209) | [`b26039b09f`](vllm-project/vllm@b26039b) | Custom routed-expert weight loading | | [#52560](vllm-project/vllm#52560) | [`2f55ef254c`](vllm-project/vllm@2f55ef2) | Qwen3-Omni DSpark support | | [#52816](vllm-project/vllm#52816) | [`b389ac2946`](vllm-project/vllm@b389ac2) | DFlash2 and DFlash class factories | | [#53183](vllm-project/vllm#53183) | [`4aab2b0ebe`](vllm-project/vllm@4aab2b0) | MRV2 becomes the default runner | | [#53435](vllm-project/vllm#53435) | [`a9a17e7095`](vllm-project/vllm@a9a17e7) | DFlash2 subclass loading fix | | [#53508](vllm-project/vllm#53508) | [`479eeb32d2`](vllm-project/vllm@479eeb3) | Isolated sleep-mode KV allocations | | [#53515](vllm-project/vllm#53515) | [`b1fbbc2ade`](vllm-project/vllm@b1fbbc2) | Persistent PCP graph input buffers | | [#53694](vllm-project/vllm#53694) | [`5acc1c4e4b`](vllm-project/vllm@5acc1c4) | Spec-decode `dp_sync` contract | | [#53869](vllm-project/vllm#53869) | [`b3af042abd`](vllm-project/vllm@b3af042) | PCP slot mappings for PIECEWISE capture | ### Changes by file > Note: the per-file notes below document the reviewed source mapping for the pinned upstream range. Where they describe code as keeping a v0.27.1 lane, the latest revision (`fe7f550b7`) removes the `vllm_version_is("0.27.1")` gates from the unit/e2e tests listed below and deletes the 0.27.1-only coverage; see "What this PR does". #### Repository metadata and CI ##### `.github/vllm-main-verified.commit` 1. Updates the verified vLLM main SHA to `e6bfe03ad73a3330cb427885aa90d97a12e1c704`. - Upstream: [exact compare range](vllm-project/vllm@ba07e4a...e6bfe03). - Review: correct; this is the exact new anchor used by the source and CI review. ##### `.github/workflows/pr_test.yaml` This revision reverts the earlier local e2e-timeout tweak; this PR no longer modifies `.github/workflows/pr_test.yaml`. #### Runtime source ##### `vllm_ascend/_310p/model_runner_310p.py` 1. Adds a version-aware `KVCacheTensor` layer-name accessor and keeps v0.27.1 aliasing while allocating main-lane attention/Mamba buffers per layer. 2. Marks the 310P runner as not supporting the standardized shared backing and derives cache sizes from each layer spec. - Upstream: [#51718](vllm-project/vllm#51718) / [`8bdc70ec7b`](vllm-project/vllm@8bdc70e). - Review: correct; it avoids treating an all-layer descriptor size as one layer's allocation. ##### `vllm_ascend/_310p/worker/v2/model_runner.py` 1. Reads `shared_by` on v0.27.1 and `layers` on main when binding 310P V2 KV tensors. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the active descriptor field is selected without changing the release-lane behavior. ##### `vllm_ascend/_310p/worker_310p.py` 1. Applies the multi-group KV-memory scaling helper when the runner cannot consume standardized shared backing. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; this prevents per-layer materialization from exceeding the planner's shared-allocation budget. ##### `vllm_ascend/attention/context_parallel/dsa_cp.py` 1. Reads the DeepSeek V4 compression ratio from `compress_ratio` on v0.27.1 or `tokens_per_state` on main. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; both fields encode the same logical ratio in their respective lanes. ##### `vllm_ascend/attention/dsa_v1.py` 1. Applies the same `compress_ratio` / `tokens_per_state` compatibility when building DSA metadata. 2. Retains `AscendDSABackend.get_kv_cache_shape` intentionally: main removed the generic base declaration, but Ascend allocation code still calls the concrete backend helper. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; deleting the concrete helper would break Ascend's own allocator. ##### `vllm_ascend/core/kv_cache_interface.py` 1. Makes `AscendMLAAttentionSpec.storage_block_size` and `merge()` lane-aware for `compress_ratio` versus `tokens_per_state`. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; layout compatibility is compared using the field that exists in each lane. ##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py` 1. Replaces direct `shared_by` reads with the version-aware helper. 2. Registers each real per-layer storage when one standardized descriptor represents multiple private Ascend buffers. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; registration uses actual aligned storage addresses instead of assuming descriptor-level aliasing. ##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py` 1. Uses the version-aware tensor-layer accessor for hybrid Mooncake transfers. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py` 1. Uses the version-aware tensor-layer accessor for layerwise Mooncake transfers. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/layerwise_cache_layout.py` 1. Reads layer names through the compatibility helper. 2. Constructs v0.27.1 tensors with `shared_by` and main tensors with `layers`, `layer_stride`, `block_stride`, and `offset`. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the produced descriptor is valid in both dataclass versions. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/offloading_connector.py` 1. Removes the deleted `is_kv_cache_tensor_packed` import/call and uses `bool(block_stride)` on main. 2. Replaces `shared_by` with the version-aware layer accessor. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; this resolves both introduced P1 import/call findings while preserving the old packed-layout meaning. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/manager.py` 1. Uses the version-aware layer accessor when building recompute offload metadata. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/worker.py` 1. Preserves new descriptor geometry (`layers`, strides, offset) on main and old `shared_by` construction on v0.27.1. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; reconstructed tensors retain the layout information required by main. ##### `vllm_ascend/models/deepseek_v4/indexer.py` 1. Constructs `AscendMLAAttentionSpec` with `compress_ratio` on v0.27.1 and `tokens_per_state` on main. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/models/layer/attention/layer.py` 1. Applies the same lane-specific MLA spec field when attention layers publish their KV-cache specs. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/models/qwen3_dflash2.py` 1. Declares `decoder_layer_cls` and `model_cls` for the new upstream factory-based construction path. 2. Retains the module-global swap only for v0.27.1, where the factories do not exist. - Upstream: [#52816](vllm-project/vllm#52816) / [`b389ac2946`](vllm-project/vllm@b389ac2), finalized by [#53435](vllm-project/vllm#53435) / [`a9a17e7095`](vllm-project/vllm@a9a17e7). - Review: correct; each lane instantiates `DFlash2Qwen3DecoderLayer` and `DFlash2Qwen3Model` through its native mechanism. ##### `vllm_ascend/ops/vocab_parallel_embedding.py` 1. Adds the new `skip_gather` argument and mirrors the upstream early return before tensor-parallel gather. - Upstream: [#50465](vllm-project/vllm#50465) / [`d154d90d6c`](vllm-project/vllm@d154d90). - Review: correct; the trailing default keeps the old call contract valid. ##### `vllm_ascend/patch/platform/patch_fused_moe.py` 1. Composes an upstream custom `RoutedExperts` subclass with `AscendRoutedExperts` instead of replacing the class by name. 2. Preserves the upstream subclass's custom loader while retaining Ascend routing/EPLB behavior. - Upstream: [#52209](vllm-project/vllm#52209) / [`b26039b09f`](vllm-project/vllm@b26039b). - Review: correct; it adapts the actual factory return type and avoids bypassing new upstream loading behavior. ##### `vllm_ascend/patch/platform/patch_kv_cache_utils.py` 1. Constructs lane-correct `KVCacheTensor` descriptors and inlines page-size calculation removed from the old patch target. 2. Replaces the removed `_get_kv_cache_config_packed` hook on main with patches for `get_kv_cache_config_from_groups`, `_max_memory_usage_bytes_from_groups`, and `_pool_bytes_per_block`. 3. Preserves DeepSeek V4 shared tuples and rank-consistent KV block planning. - Upstream: [#51718](vllm-project/vllm#51718) / [`8bdc70ec7b`](vllm-project/vllm@8bdc70e). - Review: correct; this resolves the introduced P0 removed-target finding against the live main entry points. ##### `vllm_ascend/patch/platform/patch_use_v2_model_runner.py` 1. Removes DSpark and DFlash2 from Ascend's V1-only unsupported-feature result when the upstream helper exists. - Upstream: MRV2 default switch [#53183](vllm-project/vllm#53183), with DSpark from [#52560](vllm-project/vllm#52560) and DFlash2 from [#52816](vllm-project/vllm#52816). - Review: correct; the filter is narrow and does not change other unsupported features. ##### `vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py` 1. Keeps the legacy `_allocate_kv_cache` / `_reshape_kv_cache` patches only on v0.27.1. 2. Patches main's live `allocate_kv_cache` entry point with `allocate_kv_cache_main` and retains Ascend reshape binding. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; this resolves the removed-import and removed-monkey-patch-target P0/P1 findings. ##### `vllm_ascend/utils.py` 1. Adds `get_kv_cache_tensor_layers()` to normalize `shared_by` and `layers` reads. - Upstream: [#51718](vllm-project/vllm#51718). 2. Strips a PEP 440 local suffix (for example `+empty`) before `vllm_version_is()` comparison. - Upstream: no direct upstream patch; downstream compatibility hardening needed for release-lane version strings. - Review: correct; the comparison changes only local build metadata handling. ##### `vllm_ascend/worker/model_runner_v1.py` 1. Implements lane-correct KV descriptor reads and advertises support for standardized shared backing. 2. On main, overlays compatible attention/Mamba groups in one backing store and exposes descriptor-offset views; otherwise materializes correctly sized private per-layer buffers. 3. Preserves v0.27.1 aliasing, SFA/indexer layouts, sparse/offload paths, cache-only caches, and page-padding geometry. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the allocation follows the new descriptor geometry without changing the release-lane memory model. ##### `vllm_ascend/worker/v2/aclgraph_utils.py` 1. Keeps old-lane dummy-batch repartitioning but consumes already-local persistent buffers on main. 2. Uses PCP dummy block tables and slot mappings and forwards `pcp_manager` through graph capture. - Upstream: persistent buffers [#53515](vllm-project/vllm#53515) and capture slot mappings [#53869](vllm-project/vllm#53869). - Review: correct; main no longer repartitions an already rank-local capture batch. ##### `vllm_ascend/worker/v2/attn_utils.py` 1. Removes main-lane dependence on the deleted `indexes_kv_by_block_stride` marker and uses standardized page geometry. 2. Allocates one hybrid backing on main, then creates per-layer views from `offset`, `layer_stride`, and `block_stride`; private SFA/attention allocations are retained where sharing is invalid. 3. Adds `allocate_kv_cache_main`, reconstructs Ascend attention groups, and binds the live upstream allocation entry point. 4. Retains calls to concrete Ascend `get_kv_cache_shape` helpers because Ascend still needs backend-specific views after the generic base method was removed. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the three machine-reported calls are intentional concrete-backend calls, not calls to the removed base implementation. ##### `vllm_ascend/worker/v2/model_runner.py` 1. Advertises standardized shared KV backing and keeps separate `prepare_inputs` implementations for the two upstream signatures. 2. Preserves the larger of real PCP tokens and graph-descriptor padding; main forwards `padded_num_tokens` to the PCP manager. - Upstream: KV layout [#51718](vllm-project/vllm#51718), persistent PCP buffers [#53515](vllm-project/vllm#53515), and capture mappings [#53869](vllm-project/vllm#53869). - Review: correct; runtime PCP tokens are not truncated to the graph descriptor. ##### `vllm_ascend/worker/v2/pcp_manager.py` 1. Matches the optional constructor/partition keywords exposed by each lane. 2. Uses persistent `AscendInputBuffers`, including the `max_num_reqs + 1` query-offset view required by prefix sums. 3. Preserves explicit graph padding in the main-lane local batch. - Upstream: [#53515](vllm-project/vllm#53515) and [#53869](vllm-project/vllm#53869). - Review: correct; buffer lifetime, shape, and padding match the new PCP capture contract. ##### `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py` 1. Accepts `dp_sync`, forwards `num_tokens_across_dp` on v0.27.1, and forwards `dp_sync` on main. - Upstream: [#53694](vllm-project/vllm#53694) / [`5acc1c4e4b`](vllm-project/vllm@5acc1c4). - Review: correct. ##### `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py` 1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync` forwarding in DFlash. - Upstream: [#53694](vllm-project/vllm#53694). - Review: correct. ##### `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py` 1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync` forwarding in DSpark. - Upstream: [#53694](vllm-project/vllm#53694). - Review: correct. ##### `vllm_ascend/worker/worker.py` 1. Guards the removed `post_kv_cache_wake_up` hook with `hasattr`. - Upstream: [#53508](vllm-project/vllm#53508) / [`479eeb32d2`](vllm-project/vllm@479eeb3). 2. Scales multi-group KV memory only when Ascend must materialize private buffers; skips DeepSeek V4 custom planning and compatible standardized hybrid sharing. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; it prevents OOM without reducing capacity for runners that can consume the shared layout. #### Unit tests ##### `tests/ut/_310p/test_model_runner_310p.py` 1. Verifies that the 310P runner does not advertise standardized shared KV backing. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/_310p/test_model_runner_v2_310p.py` 1. Makes the descriptor fixture valid with `shared_by` on v0.27.1 and `layers` on main. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/attention/test_dsa_v1.py` 1. Covers both `compress_ratio` and `tokens_per_state` metadata inputs. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/core/test_recompute_scheduler.py` 1. Constructs base MLA specs with the ratio field available in the active lane. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/distributed/ascend_store/test_layerwise_cache_layout.py` 1. Adds a lane-aware `KVCacheTensor` fixture and validates layout reads through the compatibility helper. 2. Covers main descriptor strides/offsets and packed-descriptor rejection. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/kv_offload/test_mooncake_connector.py` 1. Adapts fixtures/assertions to standardized descriptors and main group-allocation sizes. 2. Adds coverage for registering multiple private per-layer storages represented by one descriptor. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/patch/platform/test_patch_fused_moe.py` 1. Verifies composition of a custom upstream routed-expert subclass with the Ascend contract and loader preservation. - Upstream coverage: [#52209](vllm-project/vllm#52209). ##### `tests/ut/patch/platform/test_patch_use_v2_model_runner.py` 1. Verifies that only DSpark and DFlash2 are removed from the V1-only unsupported list. - Upstream coverage: [#53183](vllm-project/vllm#53183), [#52560](vllm-project/vllm#52560), and [#52816](vllm-project/vllm#52816). ##### `tests/ut/patch/platform/test_prefix_cache_cp_patches.py` 1. Adapts standardized KV descriptor fixtures. 2. Covers DeepSeek V4 shared-tuple capacity and rank-consistent replanning. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/spec_decode/test_dflash2_proposer.py` 1. Verifies DFlash2's `decoder_layer_cls` and `model_cls` declarations. - Upstream coverage: [#52816](vllm-project/vllm#52816) and [#53435](vllm-project/vllm#53435). ##### `tests/ut/test_compressed_prefix_cache.py` 1. Constructs compressed-prefix MLA specs with the lane-specific ratio field. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/tools/bisect/test_version_compat.py` 1. Isolates `VLLM_VERSION` environment state so one compatibility test cannot leak its lane into another. - Upstream coverage: no direct source patch; downstream test isolation for the two-lane compatibility logic. ##### `tests/ut/worker/a2/test_model_runner_v1.py` 1. Covers standardized descriptor allocation, per-layer views, shared capacity, and cache-only behavior. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/a2/test_model_runner_v1_with_device.py` 1. Adapts device-backed V1 fixtures to `shared_by` / `layers` and validates main-lane geometry. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/a2/test_worker_v1.py` 1. Covers shared-layout capacity versus private-buffer budget scaling. 2. Covers the optional wake hook after sleep-mode KV allocation changes. - Upstream coverage: [#51718](vllm-project/vllm#51718) and [#53508](vllm-project/vllm#53508). ##### `tests/ut/worker/test_attn_utils_v2.py` 1. Covers the main `allocate_kv_cache` entry point, new descriptor geometry, MLA ratio field, and flat attention-group reshape contract. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/test_model_runner_v2.py` 1. Verifies both prepare-input implementations preserve real PCP tokens and only main forwards graph padding. - Upstream coverage: [#53515](vllm-project/vllm#53515) and [#53869](vllm-project/vllm#53869). ##### `tests/ut/worker/test_model_runner_v2_finegrained_tp.py` 1. Adds `batch_sharder` and request-count fields to the bare fixture to match the new sampling contract. - Upstream coverage: [#50465](vllm-project/vllm#50465). ##### `tests/ut/worker/test_model_runner_v2_mamba.py` 1. Adds lane-aware descriptors and validates one main-lane hybrid backing with per-layer offsets. 2. Covers the removal of `indexes_kv_by_block_stride` through observable page-padding geometry. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/test_pcp_manager_v2.py` 1. Covers persistent Ascend input buffers, prefix-sum offset shape, padded local batches, main capture slot mappings, and old/new keyword signatures. 2. Verifies `dp_sync` on all three Ascend speculators and `pcp_manager` on graph capture. - Upstream coverage: [#53515](vllm-project/vllm#53515), [#53694](vllm-project/vllm#53694), and [#53869](vllm-project/vllm#53869). ### Does this PR introduce any user-facing change? No. This is an internal compatibility update; it does not add an Ascend-specific public API. ### How was this patch tested? - Exact-contract main2main validation and range prediction for vLLM [`ba07e4a48...e6bfe03ad`](vllm-project/vllm@ba07e4a...e6bfe03). - GitHub Actions: [run 33229439657](https://github.com/vllm-project/vllm-ascend/actions/runs/33229439657). - Successful in that run: pre-commit, both 310P jobs, all A3 jobs, and the passing A2 shards on both `e6bfe03ad...` and `v0.27.1`. - Pending rerun: ModelScope HTTP 500 failures on A2; two unchanged EPLB CPU tests fail identically on both lanes. - vLLM main: vllm-project/vllm@ba07e4a --------- Signed-off-by: liaoqidan <1107297340@qq.com> Signed-off-by: shenzhao <shenzhao9@huawei.com> Signed-off-by: LQDLove <LQDLove@users.noreply.github.com> Co-authored-by: shenzhao <shenzhao9@huawei.com> Co-authored-by: LQDLove <LQDLove@users.noreply.github.com>

…efill (vllm-project#53694)

### What this PR does / why we need it This PR upgrades the verified vLLM main anchor from [`ba07e4a48fc951300d97eb506217dd530583dea3`](vllm-project/vllm@ba07e4a) to [`e6bfe03ad73a3330cb427885aa90d97a12e1c704`](vllm-project/vllm@e6bfe03). The exact upstream range is [ba07e4a48...e6bfe03ad](vllm-project/vllm@ba07e4a...e6bfe03). The branch is rebased onto the latest vllm-ascend `origin/main`. The current revision adds two follow-ups on top of the reviewed source mapping: - **`e67948b9c` - drop the local `pr_test.yaml` tweak.** The earlier "raise e2e timeout for `ready-all` partitions" change is reverted, so this PR no longer modifies `.github/workflows/pr_test.yaml`. - **`fe7f550b7` - drop the vLLM `v0.27.1` release lane from the tests.** Every `vllm_version_is("0.27.1")` gate in the unit/e2e suite is removed and each site resolves to the vLLM-main behavior: dual-lane if/else/ternary branches collapse to the main path, the v0.27.1-only skip and the `test_kimi_k3_gqa_mixed_groups_use_expected_physical_layout` test are deleted, the profiling-time and `prepare_inputs` AST contract tests are reshaped to the single main-lane implementation, dead `vllm_version_is` mocks/imports are dropped, the e2e `hunyuan-vl` case always skips, the obsolete `VLLM_VERSION=0.27.1` hack in `test_num_nans` is removed, and the orphaned legacy `_get_kv_cache_config_deepseek_v4` planner is deleted. `vllm_version_is()` stays in `vllm_ascend/utils.py` with its unit test. #### Review conclusion - **Latest revision (`fe7f550b7`):** clean rebase onto current `origin/main`; the two follow-ups above are committed and pushed. Local-only, non-PR working-tree sources are not part of this branch. - **Source review (earlier revisions):** the main2main adaptations for the pinned upstream range were reviewed and are documented below; no PR-introduced source-level blocker was found. - **CI:** the run triggered on the rebased head (`fe7f550b7`) supersedes the earlier run and is the authoritative gate for this revision. #### Upstream changes covered | Upstream PR | Exact commit | Contract adopted here | |---|---|---| | [#50465](vllm-project/vllm#50465) | [`d154d90d6c`](vllm-project/vllm@d154d90) | Batch-sharded sampling and `skip_gather` | | [#51718](vllm-project/vllm#51718) | [`8bdc70ec7b`](vllm-project/vllm@8bdc70e) | Standardized KV-cache layout | | [#52209](vllm-project/vllm#52209) | [`b26039b09f`](vllm-project/vllm@b26039b) | Custom routed-expert weight loading | | [#52560](vllm-project/vllm#52560) | [`2f55ef254c`](vllm-project/vllm@2f55ef2) | Qwen3-Omni DSpark support | | [#52816](vllm-project/vllm#52816) | [`b389ac2946`](vllm-project/vllm@b389ac2) | DFlash2 and DFlash class factories | | [#53183](vllm-project/vllm#53183) | [`4aab2b0ebe`](vllm-project/vllm@4aab2b0) | MRV2 becomes the default runner | | [#53435](vllm-project/vllm#53435) | [`a9a17e7095`](vllm-project/vllm@a9a17e7) | DFlash2 subclass loading fix | | [#53508](vllm-project/vllm#53508) | [`479eeb32d2`](vllm-project/vllm@479eeb3) | Isolated sleep-mode KV allocations | | [#53515](vllm-project/vllm#53515) | [`b1fbbc2ade`](vllm-project/vllm@b1fbbc2) | Persistent PCP graph input buffers | | [#53694](vllm-project/vllm#53694) | [`5acc1c4e4b`](vllm-project/vllm@5acc1c4) | Spec-decode `dp_sync` contract | | [#53869](vllm-project/vllm#53869) | [`b3af042abd`](vllm-project/vllm@b3af042) | PCP slot mappings for PIECEWISE capture | ### Changes by file > Note: the per-file notes below document the reviewed source mapping for the pinned upstream range. Where they describe code as keeping a v0.27.1 lane, the latest revision (`fe7f550b7`) removes the `vllm_version_is("0.27.1")` gates from the unit/e2e tests listed below and deletes the 0.27.1-only coverage; see "What this PR does". #### Repository metadata and CI ##### `.github/vllm-main-verified.commit` 1. Updates the verified vLLM main SHA to `e6bfe03ad73a3330cb427885aa90d97a12e1c704`. - Upstream: [exact compare range](vllm-project/vllm@ba07e4a...e6bfe03). - Review: correct; this is the exact new anchor used by the source and CI review. ##### `.github/workflows/pr_test.yaml` This revision reverts the earlier local e2e-timeout tweak; this PR no longer modifies `.github/workflows/pr_test.yaml`. #### Runtime source ##### `vllm_ascend/_310p/model_runner_310p.py` 1. Adds a version-aware `KVCacheTensor` layer-name accessor and keeps v0.27.1 aliasing while allocating main-lane attention/Mamba buffers per layer. 2. Marks the 310P runner as not supporting the standardized shared backing and derives cache sizes from each layer spec. - Upstream: [#51718](vllm-project/vllm#51718) / [`8bdc70ec7b`](vllm-project/vllm@8bdc70e). - Review: correct; it avoids treating an all-layer descriptor size as one layer's allocation. ##### `vllm_ascend/_310p/worker/v2/model_runner.py` 1. Reads `shared_by` on v0.27.1 and `layers` on main when binding 310P V2 KV tensors. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the active descriptor field is selected without changing the release-lane behavior. ##### `vllm_ascend/_310p/worker_310p.py` 1. Applies the multi-group KV-memory scaling helper when the runner cannot consume standardized shared backing. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; this prevents per-layer materialization from exceeding the planner's shared-allocation budget. ##### `vllm_ascend/attention/context_parallel/dsa_cp.py` 1. Reads the DeepSeek V4 compression ratio from `compress_ratio` on v0.27.1 or `tokens_per_state` on main. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; both fields encode the same logical ratio in their respective lanes. ##### `vllm_ascend/attention/dsa_v1.py` 1. Applies the same `compress_ratio` / `tokens_per_state` compatibility when building DSA metadata. 2. Retains `AscendDSABackend.get_kv_cache_shape` intentionally: main removed the generic base declaration, but Ascend allocation code still calls the concrete backend helper. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; deleting the concrete helper would break Ascend's own allocator. ##### `vllm_ascend/core/kv_cache_interface.py` 1. Makes `AscendMLAAttentionSpec.storage_block_size` and `merge()` lane-aware for `compress_ratio` versus `tokens_per_state`. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; layout compatibility is compared using the field that exists in each lane. ##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py` 1. Replaces direct `shared_by` reads with the version-aware helper. 2. Registers each real per-layer storage when one standardized descriptor represents multiple private Ascend buffers. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; registration uses actual aligned storage addresses instead of assuming descriptor-level aliasing. ##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py` 1. Uses the version-aware tensor-layer accessor for hybrid Mooncake transfers. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py` 1. Uses the version-aware tensor-layer accessor for layerwise Mooncake transfers. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/layerwise_cache_layout.py` 1. Reads layer names through the compatibility helper. 2. Constructs v0.27.1 tensors with `shared_by` and main tensors with `layers`, `layer_stride`, `block_stride`, and `offset`. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the produced descriptor is valid in both dataclass versions. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/offloading_connector.py` 1. Removes the deleted `is_kv_cache_tensor_packed` import/call and uses `bool(block_stride)` on main. 2. Replaces `shared_by` with the version-aware layer accessor. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; this resolves both introduced P1 import/call findings while preserving the old packed-layout meaning. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/manager.py` 1. Uses the version-aware layer accessor when building recompute offload metadata. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/worker.py` 1. Preserves new descriptor geometry (`layers`, strides, offset) on main and old `shared_by` construction on v0.27.1. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; reconstructed tensors retain the layout information required by main. ##### `vllm_ascend/models/deepseek_v4/indexer.py` 1. Constructs `AscendMLAAttentionSpec` with `compress_ratio` on v0.27.1 and `tokens_per_state` on main. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/models/layer/attention/layer.py` 1. Applies the same lane-specific MLA spec field when attention layers publish their KV-cache specs. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct. ##### `vllm_ascend/models/qwen3_dflash2.py` 1. Declares `decoder_layer_cls` and `model_cls` for the new upstream factory-based construction path. 2. Retains the module-global swap only for v0.27.1, where the factories do not exist. - Upstream: [#52816](vllm-project/vllm#52816) / [`b389ac2946`](vllm-project/vllm@b389ac2), finalized by [#53435](vllm-project/vllm#53435) / [`a9a17e7095`](vllm-project/vllm@a9a17e7). - Review: correct; each lane instantiates `DFlash2Qwen3DecoderLayer` and `DFlash2Qwen3Model` through its native mechanism. ##### `vllm_ascend/ops/vocab_parallel_embedding.py` 1. Adds the new `skip_gather` argument and mirrors the upstream early return before tensor-parallel gather. - Upstream: [#50465](vllm-project/vllm#50465) / [`d154d90d6c`](vllm-project/vllm@d154d90). - Review: correct; the trailing default keeps the old call contract valid. ##### `vllm_ascend/patch/platform/patch_fused_moe.py` 1. Composes an upstream custom `RoutedExperts` subclass with `AscendRoutedExperts` instead of replacing the class by name. 2. Preserves the upstream subclass's custom loader while retaining Ascend routing/EPLB behavior. - Upstream: [#52209](vllm-project/vllm#52209) / [`b26039b09f`](vllm-project/vllm@b26039b). - Review: correct; it adapts the actual factory return type and avoids bypassing new upstream loading behavior. ##### `vllm_ascend/patch/platform/patch_kv_cache_utils.py` 1. Constructs lane-correct `KVCacheTensor` descriptors and inlines page-size calculation removed from the old patch target. 2. Replaces the removed `_get_kv_cache_config_packed` hook on main with patches for `get_kv_cache_config_from_groups`, `_max_memory_usage_bytes_from_groups`, and `_pool_bytes_per_block`. 3. Preserves DeepSeek V4 shared tuples and rank-consistent KV block planning. - Upstream: [#51718](vllm-project/vllm#51718) / [`8bdc70ec7b`](vllm-project/vllm@8bdc70e). - Review: correct; this resolves the introduced P0 removed-target finding against the live main entry points. ##### `vllm_ascend/patch/platform/patch_use_v2_model_runner.py` 1. Removes DSpark and DFlash2 from Ascend's V1-only unsupported-feature result when the upstream helper exists. - Upstream: MRV2 default switch [#53183](vllm-project/vllm#53183), with DSpark from [#52560](vllm-project/vllm#52560) and DFlash2 from [#52816](vllm-project/vllm#52816). - Review: correct; the filter is narrow and does not change other unsupported features. ##### `vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py` 1. Keeps the legacy `_allocate_kv_cache` / `_reshape_kv_cache` patches only on v0.27.1. 2. Patches main's live `allocate_kv_cache` entry point with `allocate_kv_cache_main` and retains Ascend reshape binding. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; this resolves the removed-import and removed-monkey-patch-target P0/P1 findings. ##### `vllm_ascend/utils.py` 1. Adds `get_kv_cache_tensor_layers()` to normalize `shared_by` and `layers` reads. - Upstream: [#51718](vllm-project/vllm#51718). 2. Strips a PEP 440 local suffix (for example `+empty`) before `vllm_version_is()` comparison. - Upstream: no direct upstream patch; downstream compatibility hardening needed for release-lane version strings. - Review: correct; the comparison changes only local build metadata handling. ##### `vllm_ascend/worker/model_runner_v1.py` 1. Implements lane-correct KV descriptor reads and advertises support for standardized shared backing. 2. On main, overlays compatible attention/Mamba groups in one backing store and exposes descriptor-offset views; otherwise materializes correctly sized private per-layer buffers. 3. Preserves v0.27.1 aliasing, SFA/indexer layouts, sparse/offload paths, cache-only caches, and page-padding geometry. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the allocation follows the new descriptor geometry without changing the release-lane memory model. ##### `vllm_ascend/worker/v2/aclgraph_utils.py` 1. Keeps old-lane dummy-batch repartitioning but consumes already-local persistent buffers on main. 2. Uses PCP dummy block tables and slot mappings and forwards `pcp_manager` through graph capture. - Upstream: persistent buffers [#53515](vllm-project/vllm#53515) and capture slot mappings [#53869](vllm-project/vllm#53869). - Review: correct; main no longer repartitions an already rank-local capture batch. ##### `vllm_ascend/worker/v2/attn_utils.py` 1. Removes main-lane dependence on the deleted `indexes_kv_by_block_stride` marker and uses standardized page geometry. 2. Allocates one hybrid backing on main, then creates per-layer views from `offset`, `layer_stride`, and `block_stride`; private SFA/attention allocations are retained where sharing is invalid. 3. Adds `allocate_kv_cache_main`, reconstructs Ascend attention groups, and binds the live upstream allocation entry point. 4. Retains calls to concrete Ascend `get_kv_cache_shape` helpers because Ascend still needs backend-specific views after the generic base method was removed. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; the three machine-reported calls are intentional concrete-backend calls, not calls to the removed base implementation. ##### `vllm_ascend/worker/v2/model_runner.py` 1. Advertises standardized shared KV backing and keeps separate `prepare_inputs` implementations for the two upstream signatures. 2. Preserves the larger of real PCP tokens and graph-descriptor padding; main forwards `padded_num_tokens` to the PCP manager. - Upstream: KV layout [#51718](vllm-project/vllm#51718), persistent PCP buffers [#53515](vllm-project/vllm#53515), and capture mappings [#53869](vllm-project/vllm#53869). - Review: correct; runtime PCP tokens are not truncated to the graph descriptor. ##### `vllm_ascend/worker/v2/pcp_manager.py` 1. Matches the optional constructor/partition keywords exposed by each lane. 2. Uses persistent `AscendInputBuffers`, including the `max_num_reqs + 1` query-offset view required by prefix sums. 3. Preserves explicit graph padding in the main-lane local batch. - Upstream: [#53515](vllm-project/vllm#53515) and [#53869](vllm-project/vllm#53869). - Review: correct; buffer lifetime, shape, and padding match the new PCP capture contract. ##### `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py` 1. Accepts `dp_sync`, forwards `num_tokens_across_dp` on v0.27.1, and forwards `dp_sync` on main. - Upstream: [#53694](vllm-project/vllm#53694) / [`5acc1c4e4b`](vllm-project/vllm@5acc1c4). - Review: correct. ##### `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py` 1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync` forwarding in DFlash. - Upstream: [#53694](vllm-project/vllm#53694). - Review: correct. ##### `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py` 1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync` forwarding in DSpark. - Upstream: [#53694](vllm-project/vllm#53694). - Review: correct. ##### `vllm_ascend/worker/worker.py` 1. Guards the removed `post_kv_cache_wake_up` hook with `hasattr`. - Upstream: [#53508](vllm-project/vllm#53508) / [`479eeb32d2`](vllm-project/vllm@479eeb3). 2. Scales multi-group KV memory only when Ascend must materialize private buffers; skips DeepSeek V4 custom planning and compatible standardized hybrid sharing. - Upstream: [#51718](vllm-project/vllm#51718). - Review: correct; it prevents OOM without reducing capacity for runners that can consume the shared layout. #### Unit tests ##### `tests/ut/_310p/test_model_runner_310p.py` 1. Verifies that the 310P runner does not advertise standardized shared KV backing. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/_310p/test_model_runner_v2_310p.py` 1. Makes the descriptor fixture valid with `shared_by` on v0.27.1 and `layers` on main. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/attention/test_dsa_v1.py` 1. Covers both `compress_ratio` and `tokens_per_state` metadata inputs. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/core/test_recompute_scheduler.py` 1. Constructs base MLA specs with the ratio field available in the active lane. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/distributed/ascend_store/test_layerwise_cache_layout.py` 1. Adds a lane-aware `KVCacheTensor` fixture and validates layout reads through the compatibility helper. 2. Covers main descriptor strides/offsets and packed-descriptor rejection. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/kv_offload/test_mooncake_connector.py` 1. Adapts fixtures/assertions to standardized descriptors and main group-allocation sizes. 2. Adds coverage for registering multiple private per-layer storages represented by one descriptor. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/patch/platform/test_patch_fused_moe.py` 1. Verifies composition of a custom upstream routed-expert subclass with the Ascend contract and loader preservation. - Upstream coverage: [#52209](vllm-project/vllm#52209). ##### `tests/ut/patch/platform/test_patch_use_v2_model_runner.py` 1. Verifies that only DSpark and DFlash2 are removed from the V1-only unsupported list. - Upstream coverage: [#53183](vllm-project/vllm#53183), [#52560](vllm-project/vllm#52560), and [#52816](vllm-project/vllm#52816). ##### `tests/ut/patch/platform/test_prefix_cache_cp_patches.py` 1. Adapts standardized KV descriptor fixtures. 2. Covers DeepSeek V4 shared-tuple capacity and rank-consistent replanning. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/spec_decode/test_dflash2_proposer.py` 1. Verifies DFlash2's `decoder_layer_cls` and `model_cls` declarations. - Upstream coverage: [#52816](vllm-project/vllm#52816) and [#53435](vllm-project/vllm#53435). ##### `tests/ut/test_compressed_prefix_cache.py` 1. Constructs compressed-prefix MLA specs with the lane-specific ratio field. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/tools/bisect/test_version_compat.py` 1. Isolates `VLLM_VERSION` environment state so one compatibility test cannot leak its lane into another. - Upstream coverage: no direct source patch; downstream test isolation for the two-lane compatibility logic. ##### `tests/ut/worker/a2/test_model_runner_v1.py` 1. Covers standardized descriptor allocation, per-layer views, shared capacity, and cache-only behavior. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/a2/test_model_runner_v1_with_device.py` 1. Adapts device-backed V1 fixtures to `shared_by` / `layers` and validates main-lane geometry. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/a2/test_worker_v1.py` 1. Covers shared-layout capacity versus private-buffer budget scaling. 2. Covers the optional wake hook after sleep-mode KV allocation changes. - Upstream coverage: [#51718](vllm-project/vllm#51718) and [#53508](vllm-project/vllm#53508). ##### `tests/ut/worker/test_attn_utils_v2.py` 1. Covers the main `allocate_kv_cache` entry point, new descriptor geometry, MLA ratio field, and flat attention-group reshape contract. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/test_model_runner_v2.py` 1. Verifies both prepare-input implementations preserve real PCP tokens and only main forwards graph padding. - Upstream coverage: [#53515](vllm-project/vllm#53515) and [#53869](vllm-project/vllm#53869). ##### `tests/ut/worker/test_model_runner_v2_finegrained_tp.py` 1. Adds `batch_sharder` and request-count fields to the bare fixture to match the new sampling contract. - Upstream coverage: [#50465](vllm-project/vllm#50465). ##### `tests/ut/worker/test_model_runner_v2_mamba.py` 1. Adds lane-aware descriptors and validates one main-lane hybrid backing with per-layer offsets. 2. Covers the removal of `indexes_kv_by_block_stride` through observable page-padding geometry. - Upstream coverage: [#51718](vllm-project/vllm#51718). ##### `tests/ut/worker/test_pcp_manager_v2.py` 1. Covers persistent Ascend input buffers, prefix-sum offset shape, padded local batches, main capture slot mappings, and old/new keyword signatures. 2. Verifies `dp_sync` on all three Ascend speculators and `pcp_manager` on graph capture. - Upstream coverage: [#53515](vllm-project/vllm#53515), [#53694](vllm-project/vllm#53694), and [#53869](vllm-project/vllm#53869). ### Does this PR introduce any user-facing change? No. This is an internal compatibility update; it does not add an Ascend-specific public API. ### How was this patch tested? - Exact-contract main2main validation and range prediction for vLLM [`ba07e4a48...e6bfe03ad`](vllm-project/vllm@ba07e4a...e6bfe03). - GitHub Actions: [run 33229439657](https://github.com/vllm-project/vllm-ascend/actions/runs/33229439657). - Successful in that run: pre-commit, both 310P jobs, all A3 jobs, and the passing A2 shards on both `e6bfe03ad...` and `v0.27.1`. - Pending rerun: ModelScope HTTP 500 failures on A2; two unchanged EPLB CPU tests fail identically on both lanes. - vLLM main: vllm-project/vllm@ba07e4a --------- Signed-off-by: liaoqidan <1107297340@qq.com> Signed-off-by: shenzhao <shenzhao9@huawei.com> Signed-off-by: LQDLove <LQDLove@users.noreply.github.com> Co-authored-by: shenzhao <shenzhao9@huawei.com> Co-authored-by: LQDLove <LQDLove@users.noreply.github.com>

mergify Bot added speculative-decoding dflash mrv2 Model Runner V2 specific labels Aug 25, 2026

github-project-automation Bot added this to Sprint - DFlash Aug 25, 2026

github-project-automation Bot moved this to Backlog in Sprint - DFlash Aug 25, 2026

TheEpicDolphin force-pushed the mrv2-skip-draft-prefill-dp-sync branch from 3433960 to b433218 Compare August 25, 2026 20:50

TheEpicDolphin commented Aug 25, 2026

View reviewed changes

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

bf9868a

…efill Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

TheEpicDolphin force-pushed the mrv2-skip-draft-prefill-dp-sync branch from b433218 to bf9868a Compare August 25, 2026 21:16

TheEpicDolphin marked this pull request as ready for review August 25, 2026 21:23

TheEpicDolphin requested review from WoosukKwon, njhill and yewentao256 as code owners August 25, 2026 21:23

claude Bot reviewed Aug 25, 2026

View reviewed changes

skip dp sync for mixed batches as well

956adbb

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

TheEpicDolphin force-pushed the mrv2-skip-draft-prefill-dp-sync branch from ab1b7f1 to 956adbb Compare August 26, 2026 02:10

GirasoleY approved these changes Aug 26, 2026

View reviewed changes

address feedback/cleanup

3ff7aa9

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

TheEpicDolphin added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 26, 2026

WoosukKwon approved these changes Aug 26, 2026

View reviewed changes

fix failing test_v2_sample_tokens_runs_eplb_on_non_last_pp_rank test

55a4cc9

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

WoosukKwon merged commit 5acc1c4 into vllm-project:main Aug 27, 2026
105 of 106 checks passed

github-project-automation Bot moved this from Backlog to Done in Sprint - DFlash Aug 27, 2026

TheEpicDolphin deleted the mrv2-skip-draft-prefill-dp-sync branch August 27, 2026 05:20

puririshi98 mentioned this pull request Aug 27, 2026

[Perf][Spec Decode] dflash/dspark: prefix-cache last-block-drop exemption + decode-path overhead reductions #54092

Closed

GirasoleY mentioned this pull request Aug 27, 2026

[SpecDecode] Preserve reject-only residual verifier padding #54102

Open

LQDLove mentioned this pull request Aug 29, 2026

[CI] main2main vllm 0828 vllm-project/vllm-ascend#14872

Merged

khushali9 pushed a commit to khushali9/vllm that referenced this pull request Aug 29, 2026

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

b9a396f

…efill (vllm-project#53694) Signed-off-by: khushali9 <khushali.desai9@gmail.com>

askliar pushed a commit to askliar/vllm that referenced this pull request Aug 30, 2026

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

89cd0db

…efill (vllm-project#53694)

LQDLove mentioned this pull request Aug 31, 2026

[CI] main2main vllm 0828 vllm-project/vllm-ascend#15410

Closed

kyleliang-nv mentioned this pull request Sep 1, 2026

[Model Runner V2][Spec Decode] Overlap draft continuation DP sync with prefill #54656

Closed

am-cohere pushed a commit to am-cohere/vllm that referenced this pull request Sep 1, 2026

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

95200e3

…efill (vllm-project#53694)

mikeshawcode pushed a commit to mikeshawcode/vllm that referenced this pull request Sep 1, 2026

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

742e3de

…efill (vllm-project#53694) Signed-off-by: mikeshawcode <michaelwshaw2@gmail.com>

mikeshawcode pushed a commit to mikeshawcode/vllm that referenced this pull request Sep 1, 2026

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

0f97f10

…efill (vllm-project#53694) Signed-off-by: mikeshawcode <michaelwshaw2@gmail.com>

TheEpicDolphin mentioned this pull request Sep 2, 2026

[Model Runner V2][Spec Decode] Skip DP sync for all speculator uniform decodes #54856

Merged

mylibrar pushed a commit to tanyuqian/vllm that referenced this pull request Sep 3, 2026

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

baa7142

…efill (vllm-project#53694)

sheralskumar pushed a commit to sheralskumar/vllm that referenced this pull request Sep 8, 2026

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft pr…

cf2bcfb

…efill (vllm-project#53694)

xudonlyu mentioned this pull request Sep 9, 2026

[ROCm][DSv4] Sync dsv4-dev-rocm Fangzhou-Ai/vllm#53

Open

Vaishnavi220506 mentioned this pull request Sep 11, 2026

[Bugfix][Spec Decode] Fix DP metadata for DFlash profiling #56330

Closed

Navigation Menu

Uh oh!

Uh oh!

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill - #53694

[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill#53694
WoosukKwon merged 4 commits into
vllm-project:mainfrom
TheEpicDolphin:mrv2-skip-draft-prefill-dp-sync

TheEpicDolphin commented Aug 25, 2026 •

edited

Loading

Uh oh!

TheEpicDolphin Aug 25, 2026

Uh oh!

claude Bot left a comment

Uh oh!

GirasoleY commented Aug 25, 2026

Uh oh!

TheEpicDolphin commented Aug 26, 2026

Uh oh!

GirasoleY left a comment

Uh oh!

GirasoleY Aug 26, 2026

Uh oh!

Uh oh!

Uh oh!

TheEpicDolphin commented Aug 26, 2026

Uh oh!

github-actions Bot commented Aug 26, 2026

Uh oh!

WoosukKwon commented Aug 26, 2026

Uh oh!

TheEpicDolphin commented Aug 27, 2026

Uh oh!

github-actions Bot commented Aug 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

TheEpicDolphin commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

This PR

Evals/Benchmarks

DeepSeek-V4-Flash + MTP=3 (AutoRegressiveSpeculator)

Inkling-NVFP4 + MTP=8 (MultiModuleMTPSpeculator)

Profile

Deepseek-V4-Flash + MTP 3

Before

After

Inkling-NVFP4 Multi-layer MTP 8

Before

After

Uh oh!

TheEpicDolphin Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

GirasoleY commented Aug 25, 2026

Uh oh!

TheEpicDolphin commented Aug 26, 2026

Uh oh!

GirasoleY left a comment

Choose a reason for hiding this comment

Uh oh!

GirasoleY Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TheEpicDolphin commented Aug 26, 2026

Uh oh!

github-actions Bot commented Aug 26, 2026

Uh oh!

WoosukKwon commented Aug 26, 2026

Uh oh!

TheEpicDolphin commented Aug 27, 2026

Uh oh!

github-actions Bot commented Aug 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

TheEpicDolphin commented Aug 25, 2026 •

edited

Loading

DeepSeek-V4-Flash + MTP=3 (`AutoRegressiveSpeculator`)

Inkling-NVFP4 + MTP=8 (`MultiModuleMTPSpeculator`)