Skip to content

Sync DeepSeek upstream through V4.1 kernels (#221) - #20

Merged
zyongye merged 5 commits into
vllm-project:mainfrom
zyongye:sync/upstream-pr-221
Sep 10, 2026
Merged

zyongye merged 5 commits into
vllm-project:mainfrom
zyongye:sync/upstream-pr-221

Conversation

@zyongye

@zyongye zyongye commented Sep 10, 2026

Copy link
Copy Markdown
Member

Summary

  • Sync deepseek-ai/FlashMLA:main through commit 07a1089857b63e74e3133630c02b083b75e8d4b2, including the DeepSeek V4.1 kernels from Add kernels for DeepSeek v4.1 deepseek-ai/FlashMLA#221 and the reorganized csrc/kernels tree. The upstream history is represented by one signed sync commit for the target repository's DCO policy.
  • Preserve the vLLM fork's stable-ABI registration, SM90 dense-FP8 extension, optional output buffers, thread-safe device properties, and split-workspace optimization.
  • Port the V3.2 NVFP4-NoPE/fp8-RoPE sparse-decode path to the upstream layout without regressing V4/V4.1 dispatch.
  • Keep repository-local virtual environments out of version control.

Validation

  • Full CUDA extension build with CUDA 13.1 for SM90a, SM100a, and SM103a.
  • 6 focused pytest checks for stable operator registration, NVFP4 wire format, and optional output forwarding.
  • GB200 correctness coverage for:
    • V3.2 FP8 split and no-split decode
    • V4 FP8 decode
    • V4.1 FP8 plus V4.1 FP4 extra-cache decode
    • NVFP4 decode with 64/128 heads, top-k lengths, attention sinks, invalid indices, zero-length requests, and an extra cache
    • real preallocated output buffers for decode and sparse prefill
  • Standalone compile/load/registration check for the preserved vLLM SM90 dense-FP8 extension.
  • Built and installed flash_mla-1.0.0+29b0d03-cp310-abi3-linux_aarch64.whl into .venv; out-of-tree import registered all 9 operators.

Test dependency note

The upstream fused end-to-end reference script was not run locally because its optional tile_kernels and deep_gemm dependencies are not installed. All fused V4/V4.1 sources were included in the successful full extension build.

Sync deepseek-ai/FlashMLA main through commit
07a1089, including the DeepSeek V4.1
kernels from deepseek-ai#221 and the reorganized csrc/kernels tree.

The upstream history is represented as one commit because the target repository's
DCO check requires a valid sign-off on every commit in the pull request.

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Port the V3.2 NVFP4 cache path alongside upstream's generalized V4.1 kernels. Restore optional output forwarding and add quantization and API regression coverage.

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
@zyongye
zyongye force-pushed the sync/upstream-pr-221 branch from 52eb6a9 to 29b0d03 Compare September 10, 2026 07:09
Comment thread README.md
Comment thread csrc/api/api.cpp
zyongye and others added 2 commits September 10, 2026 08:25
Address review on vllm-project#20: keep the FLASH_MLA_ENABLE_DENSE_BWD and
PyInit__flashmla_C comments that the upstream sync dropped from
csrc/api/api.cpp, and add a README section summarizing what this fork
carries on top of deepseek-ai/FlashMLA (stable ABI, registered
operators, optional output buffers, SM90 dense FP8 extension, NVFP4 KV
cache, robustness fixes, tests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Extend the "Changes in the vLLM fork" section with a per-area table of
every file that differs from deepseek-ai/FlashMLA@07a1089 and what
changed in it, plus the full diffstat and the command to regenerate it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
@zyongye
zyongye merged commit c112cc1 into vllm-project:main Sep 10, 2026
1 check passed
@zyongye
zyongye deleted the sync/upstream-pr-221 branch September 10, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants