Repository navigation

perf(dsa): cuDNN + FlashMLA kernels for GLM-5 sparse attention (model.dsa_backend) - #3829

Merged

samsja merged 11 commits into

mainfrom

perf/sparse-mla-bwd

Oct 5, 2026

Merged

perf(dsa): cuDNN + FlashMLA kernels for GLM-5 sparse attention (model.dsa_backend)#3829
samsja merged 11 commits into
mainfrom
perf/sparse-mla-bwd

Commits on Oct 5, 2026
- perf(kernels): use cuDNN CuTe-DSL DSA kernel for sparse MLA backward on SM90
  
  samsja
  and
  claude
  committed
  91874c3
- chore: drop sparse MLA microbenchmark script
  
  samsja
  and
  claude
  committed
  30cd8c9
- feat(config): model.sparse_mla_backward to choose the DSA backward kernel
  
  samsja
  and
  claude
  committed
  528f731
- feat(trainer): model.dsa_backend with a FlashMLA forward + cuDNN backward route
  
  samsja
  and
  claude
  committed
  f4f1076
- examples(glm-5.3): SFT on cudnn_flashmla DSA backend, add 12-node 16k overlay
  
  samsja
  and
  claude
  committed
  1b935b6
- refactor(trainer): dsa_backend is tilelang (TileLang fwd+bwd) or cudnn_flashmla
  
  samsja
  and
  claude
  committed
  f2372e9
- perf(trainer): cuDNN frontend FP8 indexer scoring + radix top-k under dsa_backend=cudnn_flashmla
  
  samsja
  and
  claude
  committed
  8b5e3b6
- docs(glm-5.3): note the cuDNN indexer on cudnn_flashmla; comment top-k tie behaviour
  
  samsja
  and
  claude
  committed
  513ccfc
- Revert "docs(glm-5.3): note the cuDNN indexer on cudnn_flashmla; comment top-k tie behaviour"
  samsja
  committed
  0388753
- Revert "perf(trainer): cuDNN frontend FP8 indexer scoring + radix top-k under dsa_backend=cudnn_flashmla"
  samsja
  committed
  fb5da5b
- refactor(dsa): DSABackend Literal, required backend arg, if/else dispatch without asserts
  
  samsja
  and
  claude
  committed
  0894aa8