Skip to content

Add paged-KV block_table bounds check in mha_fwd_kvcache - #2711

Merged
Johnsonms merged 1 commit into
Dao-AILab:mainfrom
yunweili3:fix/paged-kv-cache-seqlens-bounds-check
Jul 16, 2026
Merged

Add paged-KV block_table bounds check in mha_fwd_kvcache#2711
Johnsonms merged 1 commit into
Dao-AILab:mainfrom
yunweili3:fix/paged-kv-cache-seqlens-bounds-check

Conversation

@yunweili3

Copy link
Copy Markdown
Contributor

Fixes #2709

Problem

In the paged KV cache path, the split-KV kernel
(compute_attn_1rowblock_splitkv) indexes the block table as:

block_table[n_block * kBlockN / page_block_size]

This index is bounded only by actual_seqlen_k, which in the kvcache path is
seqlens_k[b] + seqlen_knew. However, block_table only has
max_num_blocks_per_seq columns per sequence.

If a caller passes a cache_seqlens (or appends new keys) such that a sequence
length exceeds max_num_blocks_per_seq * page_block_size, the kernel reads
block_table out of bounds — there is no in-kernel check, so this is silent
memory corruption / undefined behavior rather than a clear error.

Fix

Validate the caller contract host-side in mha_fwd_kvcache. On the paged-KV
path only, check that

max(seqlens_k) + seqlen_knew  <=  max_num_blocks_per_seq * page_block_size

and raise a descriptive TORCH_CHECK error otherwise, telling the caller to
allocate more columns in block_table.

The check uses .max().item(), which forces a device→host sync, so it is gated
behind if (paged_KV) and only paid on the paged-KV path.

Tests

Adds test_flash_attn_kvcache_paged_block_table_bounds, covering:

  • Cache-length overflowcache_seqlens beyond capacity must raise.
  • Appended-new-keys overflow — a full-at-capacity cache plus one appended
    key must raise.
  • Positive control — exactly at capacity (no appended keys) must not raise
    and returns a valid, non-NaN output.

Changes

File Change
csrc/flash_attn/flash_api.cpp Host-side bounds check on the paged-KV path
tests/test_flash_attn.py New regression test

The split-KV kernel (compute_attn_1rowblock_splitkv) indexes
block_table[n_block * kBlockN / page_block_size], bounded only by
actual_seqlen_k. In the kvcache path actual_seqlen_k is
seqlens_k[b] + seqlen_knew, but block_table only has
max_num_blocks_per_seq columns per sequence. If a caller passes a
cache_seqlens (or appends new keys) exceeding
max_num_blocks_per_seq * page_block_size, the kernel reads block_table
out of bounds with no in-kernel check (see issue Dao-AILab#2709).

Validate the caller contract host-side and raise a clear error instead.
The .max().item() sync is only paid on the paged-KV path.

Add test_flash_attn_kvcache_paged_block_table_bounds covering both the
cache-length overflow and the appended-new-keys overflow, plus a
positive control exactly at capacity.
@yunweili3
yunweili3 marked this pull request as ready for review July 16, 2026 07:42
@Johnsonms
Johnsonms self-requested a review July 16, 2026 07:56
@Johnsonms
Johnsonms merged commit 77aacb6 into Dao-AILab:main Jul 16, 2026
MatthewBonanni pushed a commit to MatthewBonanni/flash-attention that referenced this pull request Jul 22, 2026
)

The split-KV kernel (compute_attn_1rowblock_splitkv) indexes
block_table[n_block * kBlockN / page_block_size], bounded only by
actual_seqlen_k. In the kvcache path actual_seqlen_k is
seqlens_k[b] + seqlen_knew, but block_table only has
max_num_blocks_per_seq columns per sequence. If a caller passes a
cache_seqlens (or appends new keys) exceeding
max_num_blocks_per_seq * page_block_size, the kernel reads block_table
out of bounds with no in-kernel check (see issue Dao-AILab#2709).

Validate the caller contract host-side and raise a clear error instead.
The .max().item() sync is only paid on the paged-KV path.

Add test_flash_attn_kvcache_paged_block_table_bounds covering both the
cache-length overflow and the appended-new-keys overflow, plus a
positive control exactly at capacity.

Co-authored-by: yunweili3 <yunweili3@users.noreply.github.com>
MatthewBonanni pushed a commit to MatthewBonanni/flash-attention that referenced this pull request Jul 22, 2026
)

The split-KV kernel (compute_attn_1rowblock_splitkv) indexes
block_table[n_block * kBlockN / page_block_size], bounded only by
actual_seqlen_k. In the kvcache path actual_seqlen_k is
seqlens_k[b] + seqlen_knew, but block_table only has
max_num_blocks_per_seq columns per sequence. If a caller passes a
cache_seqlens (or appends new keys) exceeding
max_num_blocks_per_seq * page_block_size, the kernel reads block_table
out of bounds with no in-kernel check (see issue Dao-AILab#2709).

Validate the caller contract host-side and raise a clear error instead.
The .max().item() sync is only paid on the paged-KV path.

Add test_flash_attn_kvcache_paged_block_table_bounds covering both the
cache-length overflow and the appended-new-keys overflow, plus a
positive control exactly at capacity.

Co-authored-by: yunweili3 <yunweili3@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Question] Split-KV paged attention block_table out-of-bounds risk in compute_attn_1rowblock_splitkv

2 participants