Skip to content

[9/N][Attention] Support adaptive verification in FlashMLA Sparse - #56474

Draft
MatthewBonanni wants to merge 1 commit into
feat/hisparse-deepseek-v4from
feat/flashmla-sparse-adaptive-verification
Draft

MatthewBonanni wants to merge 1 commit into
feat/hisparse-deepseek-v4from
feat/flashmla-sparse-adaptive-verification

Conversation

@MatthewBonanni

@MatthewBonanni MatthewBonanni commented Sep 11, 2026

Copy link
Copy Markdown
Member

Summary

  • allow HiSparse-backed FlashMLA Sparse to capture nonuniform adaptive-verification batches
  • preserve packed variable-length decode rows as one MQA batch instead of inventing uniform request boundaries
  • derive request rows from device metadata without a device-to-host synchronization

This is split from the core HiSparse PR because FlashMLA Sparse does not support adaptive verification on main; HiSparse can otherwise use the existing FlashInfer Sparse MLA support.

Duplicate work

I searched open PRs for FlashMLA Sparse adaptive-verification support and found no duplicate. #52988 concerns Kimi-K3 rather than FlashMLA Sparse or HiSparse.

Validation

  • pre-commit run --files tests/v1/attention/test_sparse_mla_backends.py vllm/model_executor/layers/attention/sparse_mla_attention.py vllm/v1/attention/backends/mla/flashmla_sparse.py
  • python -m pytest -q tests/v1/attention/test_sparse_mla_backends.py::test_hisparse_fp8_decode_preserves_packed_varlen_rows tests/v1/attention/test_sparse_mla_backends.py::test_flashmla_hisparse_reports_varlen_cudagraph_support tests/v1/attention/test_sparse_mla_backends.py::test_hisparse_flashmla_reorders_full_speculative_window_as_decode
    • 3 passed on one B300 GPU
  • GLM TP4 end-to-end smoke with FULL_AND_PIECEWISE and forced adaptive rejection:
    • 25 requests completed
    • 7,105 drafted tokens, 0 accepted
    • 0 errors

AI assistance was used. Every changed line and the validation results were reviewed by the human submitter.

@MatthewBonanni
MatthewBonanni added this pull request to stack #55964 September 11, 2026 14:14
@MatthewBonanni MatthewBonanni changed the title feat/flashmla sparse adaptive verification [9/N][Attention] Support adaptive verification in FlashMLA Sparse Sep 11, 2026
@MatthewBonanni
MatthewBonanni force-pushed the feat/flashmla-sparse-adaptive-verification branch from 4d1f72f to 765556b Compare September 11, 2026 15:12
@MatthewBonanni
MatthewBonanni removed this pull request from stack #55964 September 11, 2026 15:13
@MatthewBonanni
MatthewBonanni added this pull request to stack #56481 September 11, 2026 15:13
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
@LucasWilkinson
LucasWilkinson force-pushed the feat/flashmla-sparse-adaptive-verification branch from 765556b to 218a55e Compare September 12, 2026 04:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant