Add paged-KV block_table bounds check in mha_fwd_kvcache - #2711
Merged
Johnsonms merged 1 commit intoJul 16, 2026
Merged
Conversation
The split-KV kernel (compute_attn_1rowblock_splitkv) indexes block_table[n_block * kBlockN / page_block_size], bounded only by actual_seqlen_k. In the kvcache path actual_seqlen_k is seqlens_k[b] + seqlen_knew, but block_table only has max_num_blocks_per_seq columns per sequence. If a caller passes a cache_seqlens (or appends new keys) exceeding max_num_blocks_per_seq * page_block_size, the kernel reads block_table out of bounds with no in-kernel check (see issue Dao-AILab#2709). Validate the caller contract host-side and raise a clear error instead. The .max().item() sync is only paid on the paged-KV path. Add test_flash_attn_kvcache_paged_block_table_bounds covering both the cache-length overflow and the appended-new-keys overflow, plus a positive control exactly at capacity.
yunweili3
marked this pull request as ready for review
July 16, 2026 07:42
Johnsonms
self-requested a review
July 16, 2026 07:56
Johnsonms
approved these changes
Jul 16, 2026
MatthewBonanni
pushed a commit
to MatthewBonanni/flash-attention
that referenced
this pull request
Jul 22, 2026
) The split-KV kernel (compute_attn_1rowblock_splitkv) indexes block_table[n_block * kBlockN / page_block_size], bounded only by actual_seqlen_k. In the kvcache path actual_seqlen_k is seqlens_k[b] + seqlen_knew, but block_table only has max_num_blocks_per_seq columns per sequence. If a caller passes a cache_seqlens (or appends new keys) exceeding max_num_blocks_per_seq * page_block_size, the kernel reads block_table out of bounds with no in-kernel check (see issue Dao-AILab#2709). Validate the caller contract host-side and raise a clear error instead. The .max().item() sync is only paid on the paged-KV path. Add test_flash_attn_kvcache_paged_block_table_bounds covering both the cache-length overflow and the appended-new-keys overflow, plus a positive control exactly at capacity. Co-authored-by: yunweili3 <yunweili3@users.noreply.github.com>
MatthewBonanni
pushed a commit
to MatthewBonanni/flash-attention
that referenced
this pull request
Jul 22, 2026
) The split-KV kernel (compute_attn_1rowblock_splitkv) indexes block_table[n_block * kBlockN / page_block_size], bounded only by actual_seqlen_k. In the kvcache path actual_seqlen_k is seqlens_k[b] + seqlen_knew, but block_table only has max_num_blocks_per_seq columns per sequence. If a caller passes a cache_seqlens (or appends new keys) exceeding max_num_blocks_per_seq * page_block_size, the kernel reads block_table out of bounds with no in-kernel check (see issue Dao-AILab#2709). Validate the caller contract host-side and raise a clear error instead. The .max().item() sync is only paid on the paged-KV path. Add test_flash_attn_kvcache_paged_block_table_bounds covering both the cache-length overflow and the appended-new-keys overflow, plus a positive control exactly at capacity. Co-authored-by: yunweili3 <yunweili3@users.noreply.github.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2709
Problem
In the paged KV cache path, the split-KV kernel
(
compute_attn_1rowblock_splitkv) indexes the block table as:This index is bounded only by
actual_seqlen_k, which in the kvcache path isseqlens_k[b] + seqlen_knew. However,block_tableonly hasmax_num_blocks_per_seqcolumns per sequence.If a caller passes a
cache_seqlens(or appends new keys) such that a sequencelength exceeds
max_num_blocks_per_seq * page_block_size, the kernel readsblock_tableout of bounds — there is no in-kernel check, so this is silentmemory corruption / undefined behavior rather than a clear error.
Fix
Validate the caller contract host-side in
mha_fwd_kvcache. On the paged-KVpath only, check that
and raise a descriptive
TORCH_CHECKerror otherwise, telling the caller toallocate more columns in
block_table.The check uses
.max().item(), which forces a device→host sync, so it is gatedbehind
if (paged_KV)and only paid on the paged-KV path.Tests
Adds
test_flash_attn_kvcache_paged_block_table_bounds, covering:cache_seqlensbeyond capacity must raise.key must raise.
and returns a valid, non-NaN output.
Changes
csrc/flash_attn/flash_api.cpptests/test_flash_attn.py