[Bugfix][V1] Null sliding-window block-table entries below the window by stu-cao · Pull Request #60767 · vllm-project/vllm

stu-cao · 2026-10-09T03:40:37Z

Purpose

The sliding-window KV manager frees blocks once they fall wholly outside a request's window, but the worker's persistent block table keeps the freed block ids. Attention kernels load KV in tiles (FA4 uses 128-key tiles on SM100). With 64-token pages, the tile that holds the window start can include one page lying wholly below the window. That page is masked, but masked NaN still propagates (0 × NaN). After the freed block is reused, its bytes can decode as NaN in the layer that reads it, and that request's attention output turns NaN.

#60560 has the full analysis. In short, on B300 with a DFlash2 drafter (all 6 layers sliding window, 2048) under the hybrid KV cache manager and --block-size 64:

draft acceptance falls from about 40% to 27–29% at 32 concurrent sessions;
the first divergence is NaN in the last draft layer's attention;
about 21% of that layer's in-tile stale pages contain fp8 NaN bytes.

This PR makes the worker never hand those entries to attention. When gather_block_tables builds the forward-pass tables, every sliding-window entry wholly below first_scheduled_position - (sliding_window - 1 + extra_retained_tokens) is replaced by the null block:

No query of the step can attend those positions.
The cutoff is never below the manager's free boundary. The manager frees on the processed-token count, which is at most the first scheduled position, so every block it may have freed is covered.
Only the forward-pass copy changes. Slot mappings and the persistent tables are untouched.
GPU only. The first scheduled position is the request's num_computed_tokens, already on the device, so there is no host sync.
Every sliding-window spec is covered. SlidingWindowSpec.num_retained_tokens gives the reach. (KpoolTailSpec, which an earlier revision opted out, was removed on main in [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots #57169; its successor CircularBufferSpec is not a SlidingWindowSpec and is unaffected.)
Context parallelism is excluded. Masking is disabled when cp_size > 1, where a block-table column is not a contiguous token range.

Speculator steps that reuse the gathered tables (e.g. DFlash) are covered too, since their queries sit at or after the target's first scheduled position.

Scope

The change is not specific to any model or drafter. It applies to every KV cache group whose spec is a SlidingWindowSpec (including SlidingWindowMLASpec), target or draft, with any attention backend. Why it is safe:

No KV write is redirected. Every KV write slot is computed from the persistent tables (compute_slot_mappings), or, for DFlash, from this step's positions, which are above the cutoff.
Entries are whole KV blocks. Since [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots #57169 BlockTables holds one entry per KV block, so the cutoff is max(first_pos - reach, 0) // block_size entries.

Not covered:

The V1 model runner. The default V2 runner is covered; V1 is still selected for some speculators, DBO and some ROCm architectures.
Context parallelism (DCP/PCP). Excluded on purpose, since a block-table column is not a contiguous token range there.
Chunked-local attention. Not changed. Its chunk boundaries are multiples of the KV tile, so no wholly-outside page shares a tile.

Assumption: the null block holds no NaN. The KV cache is zero-allocated, and the null entries the tables already contain (sliding-window prefix hits, P/D tail clipping) rely on the same thing.

Validation: end-to-end validation so far covers a DFlash2 drafter under the hybrid KV cache manager. A check on a target model with sliding-window layers (e.g. gpt-oss, Gemma 3) at high concurrency is still to do.

Relation to existing PRs

Rewrite worker block tables after SWA eviction #42276 (for [Bug]: SWA eviction leaves stale block table entries #42273, "SWA eviction leaves stale block table entries") targets the same stale entries from the scheduler side: it resends the full rewritten block table to the worker after an eviction. It has had merge conflicts since May and no review. This PR is a worker-only alternative: no scheduler or SchedulerOutput change, and no table resend. If maintainers prefer Rewrite worker block tables after SWA eviction #42276's direction, I'm happy to close this one in its favour, and the regression test here should carry over.
[BugFix][ROCM] DFlash2 fix sliding-window prefix attention NaNs #53628 fixes the same NaN symptom inside the ROCm prefix-prefill kernel, by masking block-table loads. That is complementary kernel hardening; this PR keeps freed blocks out of every backend's tables.

Test Plan

pytest tests/v1/worker/test_gpu_block_table.py: the new test_gather_block_tables_nulls_blocks_below_sliding_window checks:
- the exact cutoff, including a request with no scheduled tokens;
- that full-attention groups are untouched;
- padded rows;
- that the persistent tables are left unchanged;
- that rows are copied verbatim when no positions are passed.
End-to-end on 2× B300, GLM-5.3 NVFP4 + DFlash2, prefill DEP8 / decode 2× TP4 over NIXL, VLLM_KV_CACHE_LAYOUT=BLHNC, --block-size 64, hybrid KV cache manager on, decode prefix caching off. Agentic multi-turn traffic (AgentX), 32 concurrent sessions, 900 s. Acceptance = accepted / drafted tokens over the profiling window.
pre-commit run ruff-check and pre-commit run mypy on the changed files.

Test Result

Unit tests: 11 passed on B300 (the new test and the 10 existing tests in the file).
Rebased 2026-10-13 onto main 791f92135a ([KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots #57169 removed KpoolTailSpec and kernel block sizes; the two review suggestions from @TheEpicDolphin are in): 16 passed, 0 failed on B300 (2026-10-14; test_gpu_block_table.py incl. the new test, plus the selected test_kv_cache_utils cases).
ruff and mypy: passed.
End-to-end, same build and traffic:

	draft acceptance
without this PR	28.95%
with this PR	41.21% (819,107 / 1,987,756)
reference: hybrid KV cache manager off	39.61%

The end-to-end run used this exact diff on our serving build (vLLM 0.30.1-based), where it applies unchanged, because that stack (NVFP4 KV, NIXL whole-row registration) is not on main yet.

AI assistance (Claude) was used to investigate the bug and to draft this change and its test. I reviewed every changed line and the test results.

🤖 Generated with Claude Code

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Dao007forever · 2026-10-09T05:48:16Z

HiSparse added block_table_updates, which #42276 could use. The problem is it's quite expensive for SWA.

TheEpicDolphin

I like the general approach, but it seems like KpoolTailSpec, which inherits from SlidingWindowSpec, would break from this change. It uses a one block ring buffer, and when first_pos gets large enough it will null that one block table entry.

Maybe we can add a num_retained_tokens method to both, and have it return None for KpoolTailSpec so that it opts out of the optimization? Just an idea.

TheEpicDolphin · 2026-10-09T22:06:38Z

+        query_start_loc: torch.Tensor | None = None,
+        seq_lens: torch.Tensor | None = None,


RequestStates has a num_computed_tokens property that already contains the first token pos for each sequence. You may be able to pass that in instead of query_start_loc and seq_lens together.

Fixed in new revision. num_computed_tokens is used here which got passed via num_computed_tokens=self.req_states.num_computed_tokens.gpu

TheEpicDolphin · 2026-10-09T22:07:14Z

        block_ids = tl.load(src_row_ptr + offset, mask=offset < num_blocks)
+        block_ids = tl.where(offset < num_null, 0, block_ids)


nit: can simplify this down to:

block_ids = tl.load( src_row_ptr + offset, mask=(offset >= num_null) & (offset < num_blocks), other=0, )

Thanks! Addressed in new revision

stu-cao · 2026-10-10T16:26:34Z

@TheEpicDolphin thanks for the review, all three addressed in 7a659e9.

KpoolTailSpec: you're right. The tail ring is one block that every position maps into through pos % block_size, so the nulling would have hit its only entry once a request passed 2 * ring - 1 tokens, and the tail would stop being written. I added SlidingWindowSpec.num_retained_tokens (sliding_window - 1 + extra_retained_tokens) as you suggested, with KpoolTailSpec returning None to opt out; the runner maps None to a reach of 0, which the kernel already treats as disabled. SlidingWindowMLASpec keeps the base value, since it uses the same SlidingWindowManager and free boundary.
num_computed_tokens: seq_lens is itself computed as num_computed_tokens[req] + query_len, so seq_len - query_len was always num_computed_tokens[req]. The kernel now reads that tensor directly by request slot, and the two batch-indexed arguments are gone. The test now places the requests in non-consecutive slots, so slot-vs-batch indexing is covered.
Load mask: applied as written.

mergify · 2026-10-10T16:26:59Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @stu-cao.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

The sliding-window manager frees blocks that fall wholly outside a request's window, but the worker's persistent block table keeps the freed block ids. Attention kernels load KV in tiles (FA4: 128 keys on SM100); with 64-token pages the tile that holds the window start can include a page that lies wholly below the window. Its positions are masked, but masked NaN still propagates (0 * NaN). Once the freed block is reused, its bytes can decode as NaN in the layer that reads it, and that request's attention output becomes NaN. When gathering the forward-pass block tables, replace every sliding-window entry that lies wholly below first_scheduled_position - SlidingWindowSpec.num_retained_tokens (sliding_window - 1 + extra_retained_tokens) with the null block. No query of the step can attend those positions, and the cutoff is never below the manager's free boundary, so every block it may have freed is covered. The first scheduled position is RequestState.num_computed_tokens, which the runner already holds per request slot, so that tensor is passed to gather_block_tables and the nulling is folded into the load mask. Only the forward-pass copy changes; slot mappings and the persistent tables are untouched. Disabled under context parallelism. Fixes vllm-project#60560. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Stu Cao <stu@primeintellect.ai>

TheEpicDolphin

Thanks for addressing the feedback, this looks good to me. I think it's a relatively safe short-term fix.

github-actions · 2026-10-14T06:35:58Z

✅ @stu-cao, CI is now available for this PR.

/ci run starts upstream CI; /amd-ci run starts AMD CI only.
Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
/ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
/amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
/ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

TheEpicDolphin · 2026-10-14T23:37:00Z

hey @stu-cao, please run the CI on this PR when you get the chance

stu-cao · 2026-10-15T02:32:51Z

/ci run

github-actions · 2026-10-15T02:33:18Z

✅ Triggered Buildkite CI #95000 for commit 2dfcb1972cde.

ℹ️ This PR is 98 commits behind upstream main. Merged upstream main into this branch as 2dfcb1972cde before starting CI. Pull the branch before pushing more changes.

github-actions · 2026-10-17T07:07:05Z

✅ Triggered Buildkite CI #95559 for commit 05a83b70babf.

ℹ️ This PR is 63 commits behind upstream main. Merged upstream main into this branch as 05a83b70babf before starting CI. Pull the branch before pushing more changes.

mergify · 2026-10-17T08:54:20Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @stu-cao.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Resolves the overlap with vllm-project#61174 (stale shared block-table rows): the gather keeps main's num_rows / _num_dirty_rows handling and this branch's sliding-window masking arguments; both sides' new tests are kept. Signed-off-by: Stu Cao <stu@primeintellect.ai>

stu-cao · 2026-10-17T16:50:42Z

/ci run

github-actions · 2026-10-17T16:51:08Z

✅ Triggered Buildkite CI #95604 for commit df31ba79b9f1.

ℹ️ This PR is 1 commit behind upstream main. Merged upstream main into this branch as df31ba79b9f1 before starting CI. Pull the branch before pushing more changes.

stu-cao · 2026-10-18T04:14:57Z

/ci run

github-actions · 2026-10-18T04:15:15Z

❌ CI already ran on this commit: https://buildkite.com/vllm/ci/builds/95604. No new CI build was started. Push new changes to run CI again, use /ci retry to rerun failed jobs, or ask a reviewer with write access to rerun it.

Picks up vllm-project#62213 (order-dependent ROCm aiter sampler test), which was the only failure in CI build 95604. Signed-off-by: Stu Cao <stu@primeintellect.ai>

stu-cao · 2026-10-18T04:15:49Z

/ci run

github-actions · 2026-10-18T04:16:09Z

✅ Triggered Buildkite CI #95657 for commit 51e63a1cb15e.

Picks up vllm-project#62221 (Rust frontend version mismatch) and vllm-project#62274 (ROCm inductor batch_invariant guard), the only failures in CI build 95657. Signed-off-by: Stu Cao <stu@primeintellect.ai>

stu-cao · 2026-10-19T02:29:09Z

/ci run

github-actions · 2026-10-19T02:29:27Z

✅ Triggered Buildkite CI #95780 for commit d84163379a07.

stu-cao · 2026-10-19T20:08:43Z

@TheEpicDolphin Could you please help merge? CI is all green

BlockTables.gather_block_tables now nulls entries below a sliding window and tracks dirty rows (upstream vllm-project#60767); the fake tables carry the fields of a table with no sliding-window group.

### What this PR does / why we need it? Port `vllm-ascend` to vLLM main `544f182d` (bumped from `c0b6f74`) while keeping the `v0.31.0` release lane green. The upstream window drops `replace_parameter(..., prefer_copy=...)`, fuses the Kimi-K3 output gate into the QKV-A projection, adds the ROCm Kimi-K3 specialized-decode hooks, widens the EPLB expert-weight and rejection-sample contracts, and adds the DP-drafter `num_reqs_override`, the MRv2 `skip_sample` drafting skip, the HiSparse `topk_buffer_width`, the SimpleCPUOffload CPU-block RPC and the abstract `AsyncModelRunnerOutput.get_kv_connector_output`. Every gate below is either a `vllm_version_is("0.31.0")` split or an attribute/signature probe. | Files | Upstream vLLM change (why it breaks vllm-ascend) | vllm-ascend adaptation | |-------|--------------------------------------------------|------------------------| | `.github/vllm-main-verified.commit` | — (main-lane pin only) | Verified main pin bumped `c0b6f74` -> `544f182d` | | `tests/ut/attention/test_mla_prolog_v3_route.py` | [vllm#60805](vllm-project/vllm#60805) makes `MLAAttention.get_kv_cache_spec` read `topk_buffer_width` (set in `__init__`); [vllm#59634](vllm-project/vllm#59634) makes `_mla_preprocess` return a 3-tuple | Set `owner.topk_buffer_width = None` on the stub; mock `_mla_preprocess` as `(None, None, None)` | | `tests/ut/attention/test_mla_v1.py` | [vllm#59634](vllm-project/vllm#59634) fuses the Kimi-K3 output gate as a third QKV-A shard | Add tests for `qkv_a_gate_fused`/`use_output_gate` and `_split_fused_qkv_lora` gate peeling; unpack the 3-tuple | | `tests/ut/models/minimax_m3/test_minimax_m3_vit.py` | [vllm#60238](vllm-project/vllm#60238) makes `_mark_tower_model` read `use_v2_model_runner` | Add `use_v2_model_runner=False` to the constructed runner stub | | `tests/ut/models/test_kimi_k3_adapter.py` | [vllm#60683](vllm-project/vllm#60683) adds the ROCm specialized-decode hooks (`init_specialized_decode`, `get_model_state_cls`, the `step` forward argument, and `layer.kv_cache` reassignment via `clear_layer_kv_caches`) | Add main-only tests (`skipif vllm_version_is("0.31.0")`) covering the inherited hooks | | `tests/ut/ops/test_routed_experts.py` | [vllm#61786](vllm-project/vllm#61786) registers expert weights via `w.view(torch.uint8) if w.element_size() == 1 else w` | New UT asserting `EplbExpertTensorList.element_size()/view()` keep the container and its aliasing semantics | | `tests/ut/patch/platform/test_patch_structured_output.py` | [vllm#49625](vllm-project/vllm#49625) makes `StructuredOutputManager.grammar_init` read `backend_name` | Set `manager.backend_name = None` on the stub | | `tests/ut/runtime_guard/test_unified_wiring.py` | [vllm#56430](vllm-project/vllm#56430) adds `AsyncModelRunnerOutput.get_kv_connector_output` | Add a stub `get_kv_connector_output` to the fake async output | | `tests/ut/simple_kv_offload/test_simple_kv_offload.py` | [vllm#52921](vllm-project/vllm#52921) adds the `get_simple_cpu_offload_num_cpu_blocks` worker RPC | New main-only UTs for the aligned / no-group / no-connector paths | | `tests/ut/spec_decode/test_dflash2_v2.py` | [vllm#58984](vllm-project/vllm#58984) adds `num_reqs_override` to `prepare_dflash_inputs_triton` | Update the expected kwarg; new test for positional forwarding | | `tests/ut/worker/test_model_runner_v1.py` | [vllm#60805](vllm-project/vllm#60805) `topk_buffer_width` | Set `layer.topk_buffer_width = None` on the layer stub | | `tests/ut/worker/test_model_runner_v2.py` | Same [vllm#60805](vllm-project/vllm#60805) | Same stub attribute | | `tests/ut/worker/v2/test_speculator.py` | [vllm#62784](vllm-project/vllm#62784) adds `skip_sample` to the drafter `_prefill` | Expect/forward `skip_sample` on main (7 -> 8 positional args) | | `tests/ut/worker/v2/test_v2_block_table.py` | [vllm#60767](vllm-project/vllm#60767) makes `init_block_table_layout_tensors` read `_sliding_window_reach` | Set `tables._sliding_window_reach = [0, 0]` | | `vllm_ascend/attention/mla_v1.py` | [vllm#59634](vllm-project/vllm#59634) fuses the Kimi-K3 output gate into the QKV-A projection (three shards, no standalone `g_proj`); [vllm#62326](vllm-project/vllm#62326) drops `replace_parameter(..., prefer_copy=)` | Detect the gate-fused layout from `fused_qkv_a_proj.output_sizes`, disable MLAPO/PROLOG_V3 for it, bound the KV-A slice with `_fused_qkv_a_kv_end`, peel the gate in `_split_fused_qkv_lora`; version-split `replace_parameter` | | `vllm_ascend/attention/sfa_v1.py` | [vllm#62326](vllm-project/vllm#62326) drops `replace_parameter(..., prefer_copy=)` | Version-split `replace_parameter` (keep `prefer_copy=True` on v0.31.0) | | `vllm_ascend/models/deepseek_v4/indexer.py` | [vllm#52670](vllm-project/vllm#52670) keeps `STR_DTYPE_TO_TORCH_DTYPE["fp8"]` as the uint8 storage dtype, so `kv_cache_dtype_str_to_dtype` cannot validate the indexer's fp8 compute dtype | Validate `indexer_kv_dtype` against the platform default string and return the compute dtype directly | | `vllm_ascend/models/kimi_k3.py` | [vllm#60683](vllm-project/vllm#60683) adds the ROCm specialized-decode hooks and reassigns `layer.kv_cache` on teardown | Add a `kv_cache` setter, call `init_specialized_decode` + override `get_model_state_cls` to the Ascend hybrid state on main, forward `step` | | `vllm_ascend/observability/runtime_guard/hooks.py` | [vllm#56430](vllm-project/vllm#56430) makes `AsyncModelRunnerOutput.get_kv_connector_output` required | Implement it on `AscendAsyncOutput` (main only) | | `vllm_ascend/ops/fused_moe/routed_experts.py` | [vllm#61786](vllm-project/vllm#61786) moves 1-byte expert tensors as uint8 views | Add `element_size()`/`view()` to `EplbExpertTensorList`; widen the `get_expert_weights` return type | | `vllm_ascend/ops/gdn.py` | [vllm#62326](vllm-project/vllm#62326) drops `replace_parameter(..., prefer_copy=)` | Version-split `replace_parameter` for the packed conv weight | | `vllm_ascend/ops/kimi_kda.py` | Same [vllm#62326](vllm-project/vllm#62326) | Same version-split for the packed conv weight | | `vllm_ascend/ops/linear.py` | Same [vllm#62326](vllm-project/vllm#62326) | Same version-split for `weight_fp32` | | `vllm_ascend/ops/triton/v2/spec_decode/prepare_dflash_inputs.py` | [vllm#58984](vllm-project/vllm#58984) forwards the DP-synced request count during dummy runs | Accept `num_reqs_override`, cap `num_reqs` and slice `num_scheduled_tokens` | | `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py` | [vllm#62784](vllm-project/vllm#62784) adds `skip_sample` to `_prefill` | Version-split `_prefill`, forward `skip_sample` on main | | `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py` | [vllm#58984](vllm-project/vllm#58984) adds `num_reqs_override` | Forward `num_reqs_override` through both `prepare_dflash_inputs` wrappers | | `vllm_ascend/worker/v2/spec_decode/gemma4/speculator.py` | [vllm#62784](vllm-project/vllm#62784) adds `skip_sample` | Accept and forward `skip_sample` | | `vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py` | [vllm#53987](vllm-project/vllm#53987) adds the FLy entropy-gated verification mode; upstream also threads the watermarking kwargs | Accept both kwarg groups, but raise `NotImplementedError` for FLy and watermarking on NPU | | `vllm_ascend/worker/worker.py` | [vllm#52921](vllm-project/vllm#52921) adds the `get_simple_cpu_offload_num_cpu_blocks` RPC to the worker | Implement it (main only) via `_find_simple_cpu_offload_connector().get_aligned_num_cpu_blocks()` | ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? --- ### Checklist #### Testing - [ ] **Boundary / edge-case coverage**: I reviewed whether this change involves boundary conditions (e.g. empty input, single/zero element, min/max values, off-by-one, sequence-length limits, integer overflow). Where applicable, I added unit tests covering these edge cases in `tests/ut/` or `tests/e2e/`. - [ ] **Regression test**: If this PR fixes a bug, I added a regression test that would fail without the fix. - [ ] **Error / failure path coverage**: I covered abnormal paths (exceptions, timeouts, out-of-range, division by zero) in addition to the happy path. - [ ] **Meaningful & reliable tests**: New tests assert real behavior and are not written merely to satisfy CI. - [ ] **Performance impact**: If this PR touches a performance-sensitive path, I attached benchmark results and added performance regression tests. - vLLM main: vllm-project/vllm@c0b6f74 --------- Signed-off-by: zhangxinyuehfad <starmoon_zhang@163.com> Signed-off-by: hfadzxy <starmoon_zhang@163.com>

Both accuracy bugs in ROCM_ATTN's Triton fallback that this avoided are fixed on main: the sliding-window NaN (vllm-project#62828) no longer occurs since vllm-project#60767 nulls below-window block-table entries, and the gfx942 unwritten output heads were fixed by vllm-project#63268. What remains is a performance change, which belongs in vllm-project#61433. Keep only the gfx950 rule for vllm-project#62606. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Chen, Todd <zhenchen@amd.com>

…vllm-project#60767) Signed-off-by: Stu Cao <stu@primeintellect.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com> Co-authored-by: vllm-agent <claw@inferact.ai>

stu-cao requested review from WoosukKwon, njhill and yewentao256 as code owners October 9, 2026 03:40

claude Bot reviewed Oct 9, 2026

View reviewed changes

mergify Bot added mrv2 Model Runner V2 specific bug Something isn't working labels Oct 9, 2026

Dao007forever mentioned this pull request Oct 9, 2026

[Bugfix][MRV2] Null block-table entries the KV cache manager freed Dao007forever/vllm#15

Draft

5 of 6 tasks

stu-cao mentioned this pull request Oct 9, 2026

[KV Connector][NIXL] Opt-in whole-row regions for replicated BLHNC layers #60564

Closed

TheEpicDolphin requested changes Oct 9, 2026

View reviewed changes

stu-cao requested review from heheda12345 and ivanium as code owners October 10, 2026 16:26

mergify Bot added the needs-rebase label Oct 10, 2026

stu-cao mentioned this pull request Oct 10, 2026

[Bugfix][V1] Retain the EAGLE hit blocks below a sliding window across a KV connector import #61011

Open

positive666 mentioned this pull request Oct 12, 2026

[Bugfix][Spec Decode] Clear stale shared block-table rows #61174

Merged

6 tasks done

stu-cao force-pushed the fix/swa-null-freed-block-table-entries branch from 7a659e9 to 16d9ae0 Compare October 14, 2026 00:35

stu-cao requested a review from ZJY0516 as a code owner October 14, 2026 00:35

mergify Bot removed the needs-rebase label Oct 14, 2026

TheEpicDolphin approved these changes Oct 14, 2026

View reviewed changes

TheEpicDolphin added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 14, 2026

Merge branch 'main' into fix/swa-null-freed-block-table-entries

2dfcb19

mergify Bot added the needs-rebase label Oct 17, 2026

mergify Bot removed the needs-rebase label Oct 17, 2026

Merge branch 'main' into fix/swa-null-freed-block-table-entries

df31ba7

Merge branch 'main' into fix/swa-null-freed-block-table-entries

51e63a1

Picks up vllm-project#62213 (order-dependent ROCm aiter sampler test), which was the only failure in CI build 95604. Signed-off-by: Stu Cao <stu@primeintellect.ai>

Merge branch 'main' into fix/swa-null-freed-block-table-entries

d841633

Picks up vllm-project#62221 (Rust frontend version mismatch) and vllm-project#62274 (ROCm inductor batch_invariant guard), the only failures in CI build 95657. Signed-off-by: Stu Cao <stu@primeintellect.ai>

ZJY0516 merged commit e1f3141 into vllm-project:main Oct 19, 2026
260 checks passed

zhangxinyuehfad mentioned this pull request Oct 25, 2026

[Misc] main2main 1023 vllm-project/vllm-ascend#19694

Merged

5 tasks

pepmach mentioned this pull request Oct 26, 2026

[Bug]: OffloadingConnector loads never-written KV for sliding-window layers when offloaded chunks span several GPU blocks #63653

Closed

1 task done

zhentaocc mentioned this pull request Oct 27, 2026

[ROCm] Rank TRITON_ATTN above unified attention on gfx950 until the aiter pin has the #62606 fix #62813

Closed

6 tasks done

zhentaocc mentioned this pull request Oct 29, 2026

[RFC]: DFlash2 speculative decoding performance on ROCm (MI355X) #61761

Open

		query_start_loc: torch.Tensor \| None = None,
		seq_lens: torch.Tensor \| None = None,

		block_ids = tl.load(src_row_ptr + offset, mask=offset < num_blocks)
		block_ids = tl.where(offset < num_null, 0, block_ids)

Conversation

stu-cao commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Scope

Relation to existing PRs

Test Plan

Test Result

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

Dao007forever commented Oct 9, 2026

Uh oh!

TheEpicDolphin left a comment

Choose a reason for hiding this comment

Uh oh!

TheEpicDolphin Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

stu-cao Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

TheEpicDolphin Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stu-cao Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

stu-cao commented Oct 10, 2026

Uh oh!

mergify Bot commented Oct 10, 2026

Uh oh!

TheEpicDolphin left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 14, 2026

Uh oh!

TheEpicDolphin commented Oct 14, 2026

Uh oh!

stu-cao commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 17, 2026

Uh oh!

mergify Bot commented Oct 17, 2026

Uh oh!

stu-cao commented Oct 17, 2026

Uh oh!

github-actions Bot commented Oct 17, 2026

Uh oh!

stu-cao commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

stu-cao commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

stu-cao commented Oct 19, 2026

Uh oh!

github-actions Bot commented Oct 19, 2026

Uh oh!

stu-cao commented Oct 19, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

stu-cao commented Oct 9, 2026 •

edited

Loading

TheEpicDolphin Oct 9, 2026 •

edited

Loading