[MISC] Add strict contiguity check for FlashInfer attention tensors

[vadiklyutiy]

Early check of potential error as in #30842. See also #31617, flashinfer-ai/flashinfer#2232

Updates FlashInfer attention path to use stricter contiguous check, preventing potential CUDA kernel memory access issues.
Introduces is_strictly_contiguous() utility to detect tensors with degenerate strides that PyTorch's is_contiguous() reports as contiguous.

---

Note

Strengthens memory layout validation for FlashInfer TRTLLM attention.

• Adds is_strictly_contiguous(t) in vllm/utils/torch_utils.py to verify canonical contiguous strides and catch degenerate-stride tensors
• Replaces .is_contiguous() assertions with is_strictly_contiguous() in flashinfer.py TRTLLM (HND) prefill and decode paths for query, kv_cache_permute, workspace buffer, block_tables, and seq_lens
• Aims to fail fast before launching CUDA kernels; no algorithmic changes

^{Written by Cursor Bugbot for commit b1e334e. This will update automatically on new commits. Configure here.}

[gemini-code-assist]

Code Review

This pull request introduces a is_strictly_contiguous utility function to perform a stricter check for tensor contiguity, addressing potential memory access issues in FlashInfer CUDA kernels caused by degenerate strides. This new check is correctly applied to the FlashInfer TRTLLM prefill attention path. The implementation of the new utility is robust. However, a similar vulnerability seems to exist in the TRTLLM decode path which has not been addressed in this PR. I've added a critical comment to highlight this omission.

[benchislett]

LGTM!

[pavanimajety]

Should we also check if out is contiguous?

[pavanimajety]

I did some tests, and if torch tensor's is_contiguous() returns True where is_strictly_contiguous returns False, tensor.contiguous() actually doesn't make it a contiguous tensor. Eg:

t_base = torch.randn(16, 8, 32)
t2 = t_base.as_strided(size=(16, 1, 8, 32), stride=(256, 1, 32, 1))
t5 = t2.contiguous()

here, the result is -

t2  ->
  Shape: torch.Size([16, 1, 8, 32]), Stride: (256, 1, 32, 1)
  torch.is_contiguous(): True
  is_strictly_contiguous: False
  Expected canonical stride: (256, 256, 32, 1)

t5 ->
  Shape: torch.Size([16, 1, 8, 32]), Stride: (256, 1, 32, 1)
  torch.is_contiguous(): True
  is_strictly_contiguous: False

t2.contiguous() didn't actually convert to a contiguous tensor. While the assertion works in detecting a noncontiguous tensor, the earlier prefill_query.contiguous() may be insufficient. Wondering if we need to do anything additionally?

[pavanimajety]

we may need to do -

prefill_query_contiguous = torch.empty(t2.shape, dtype=t2.dtype, device=t2.device)
prefill_query_contiguous.copy_(prefill_query)

And similar for the rest where we may do squeeze / unsqueeze / slice.

[mergify]

Hi @vadiklyutiy, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint