Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
6d2e36f
[SM120] Add CUDA fused Triton sparse-MLA prefill backend for DSA
yunyang1999 Aug 1, 2026
2a625a4
[SM120&90] Shrink the DSA prefill dispatch surface for triton_sparse_mla
yunyang1999 Aug 16, 2026
e33b395
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Aug 31, 2026
b2f0a9f
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 1, 2026
9e1583a
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 1, 2026
56bfaf9
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 2, 2026
74ad693
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 4, 2026
747ccc6
[lint] Reformat multi-context-manager with-statement (ruff format)
yunyang1999 Sep 4, 2026
1aafbe6
Merge branch 'main' into dsa-triton-sparse-mla-prefill
yunyang1999 Sep 4, 2026
dad11cb
Merge branch 'main' into dsa-triton-sparse-mla-prefill
yunyang1999 Sep 7, 2026
ec2ff96
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 7, 2026
af62e4a
[lint] Move triton_sparse_mla test files under the new kind/subsystem…
yunyang1999 Sep 7, 2026
165d639
Merge branch 'main' into dsa-triton-sparse-mla-prefill
yunyang1999 Sep 9, 2026
083eca4
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 10, 2026
fb5a780
Merge branch 'main' into dsa-triton-sparse-mla-prefill
yunyang1999 Sep 14, 2026
2600a0a
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 15, 2026
a67f03a
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 17, 2026
c51b9e2
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 17, 2026
1205fa0
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 18, 2026
8e8d71b
[lint] Move triton_sparse_mla kernel tests back under kernels/{ops,be…
yunyang1999 Sep 18, 2026
6848fe7
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 21, 2026
1b26aef
Merge branch 'main' into dsa-triton-sparse-mla-prefill
yunyang1999 Sep 29, 2026
bcb3aff
[SM120&90] triton_sparse_mla: per-group union dedup, int64 offsets, t…
yunyang1999 Sep 29, 2026
9fe5d72
[SM120&90] triton_sparse_mla: tighten the startup validator and the u…
yunyang1999 Sep 29, 2026
d719df6
[SM120&90] triton_sparse_mla: drop the dead FP8 kernel branch, tidy c…
yunyang1999 Sep 29, 2026
1c505b9
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Sep 30, 2026
072c03e
[SM120&90] triton_sparse_mla: BCG-safe union gate, forced smem step-d…
yunyang1999 Oct 2, 2026
71fbb0c
[SM120&90] triton_sparse_mla: final tidy before re-review
yunyang1999 Oct 2, 2026
e92797e
[SM120&90] triton_sparse_mla: name architectures, not parts, in two c…
yunyang1999 Oct 2, 2026
f734886
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Oct 2, 2026
c62ea22
[SM120&90] triton_sparse_mla: cover the union tail; state the tuned a…
yunyang1999 Oct 5, 2026
3e35818
Merge branch 'main' into dsa-triton-sparse-mla-prefill
nvpohanh Oct 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/docs/advanced_features/attention_backend.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -438,6 +438,12 @@ Internally, the DSA backend dispatches to different sub-backends for prefill and
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Native FP8 (q8×kv8) sparse prefill on Hopper (SM90); requires <code>--kv-cache-dtype fp8_e4m3</code></td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong>triton_sparse_mla</strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>✅</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Fused Triton sparse prefill that works on the post-TP head count instead of padding heads to 64/128; opt-in. CUDA SM90+ (tiles tuned on SM90 and SM120; other architectures run an untuned tile and log a warning at startup), at most 32 query heads per rank, not with <code>index_kpool</code>. <code>--dsa-triton-union 2|4</code> lets adjacent query tokens share one gathered index set (exact; <code>num_q_heads * union</code> must be a power of two in [16, 32])</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong>flashmla_kv</strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>✅</td>
Expand Down
8 changes: 7 additions & 1 deletion docs/docs/advanced_features/server_arguments.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1470,7 +1470,13 @@ Combining `--enable-response-store` with `--disaggregation-mode=prefill` or `dec
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--dsa-prefill-backend`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>DSA backend for the prefill stage (overrides `--attention-backend` when running DeepSeek DSA-style attention).</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Auto (hardware-dependent)</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>flashmla_sparse</code>, <code>flashmla_sparse_q8</code>, <code>flashmla_kv</code>, <code>flashmla_auto</code>, <code>fa3</code>, <code>tilelang</code>, <code>aiter</code>, <code>trtllm</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>flashmla_sparse</code>, <code>flashmla_sparse_q8</code>, <code>triton_sparse_mla</code>, <code>flashmla_kv</code>, <code>flashmla_auto</code>, <code>fa3</code>, <code>tilelang</code>, <code>aiter</code>, <code>trtllm</code></td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--dsa-triton-union`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Union group size for `--dsa-prefill-backend triton_sparse_mla`: adjacent query tokens share one gathered index set (exact; an ownership mask restores each token's own softmax). Requires `num_q_heads * union` to be a power of two in [16, 32]; forced off under `--enable-deterministic-inference`.</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`0`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>0</code>, <code>2</code>, <code>4</code></td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--dsa-decode-backend`</td>
Expand Down
Loading
Loading