Optimize DSA backward SM100 with enhanced topk_length loop and TMA fast path - #5
Merged
Merged
Conversation
…k-copy fast path for consecutive KV tiles
umiswing
approved these changes
Jul 28, 2026
SigureMo
approved these changes
Jul 28, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Perf] DSA backward SM100: per-query topk_length 循环上界 + 连续 KV tile 的 TMA 批量加载
背景 / 动机
DeepSeek Sparse Attention 的 backward kernel(SM100/Blackwell)在加载 KV 时逐行发
cp.async,并对每个 query 固定遍历topk个 tile。两点可优化:topk,尾部整段空洞仍被遍历;本 PR 在 kernel 侧支持按
topk_length收缩循环上界,并对连续 tile 走 TMA 快路径,同时保证稀疏场景完全正确。(调用侧传入topk_length的改动在 PaddleFleet 主库,不在本 PR 范围。)改动内容(仅
dsa_bwd_sm100.py,+205/-33)per-query 循环上界:消费传入的
mTopkLength,kernel 只遍历到该上界为止,跳过尾部全空 tile。连续 KV 的 TMA 快路径:新增
tma_atom_KV+ 带运行时行偏移的 gmem 视图,连续 tile 用单条 TMA 搬 64 行。是否连续由 4 个 load warp 在 sharedkv_vote上投票决定;投票槽按tile_index & 1双缓冲,配合每迭代一次load_KV_sync_barrier,把 warp 间偏移限制在一个迭代内。TMA 事务直接记在管线自身 full mbarrier 上(128 次 arrive +expect_tx),与慢路径producer_commit等价。慢路径无分支化:负索引用位运算 clamp 到合法地址无条件拷贝,
cp_async_wait后按 OR-reduce 符号位判断整块是否含空洞,再对空洞行补零;指令流不分叉,利于 ptxas 调度。compute 路径重排:tmem fence +
consumer_release后移到 STSM+commit 之后,删除主循环冗余的compute_sync_barrier;T2R 由Repetition(8)拆成两次Repetition(4)以重叠计算与访存延迟。reduce 路径提前释放:T2R 取到寄存器 + fence 后即
consumer_release,再发 fire-and-forget 的 atomic_add;not same_hdim_kv分支的 dKV2/3 从store_dKV改为寄存器版reduce_dKV_from_reg。正确性说明
cp.asyncgather,行为与优化前一致。B ≤ KV行数 - block_tile,TMA 坐标恒在界内。load_KV_sync_barrier因kv_fast在全体 producer 线程间取值一致,快/慢分支不会相位错位;reduce_sync_barrier仅存在于same_hdim_kv分支,两分支均为const_expr编译期统一。