Repository navigation

[Perf] Optimize DeepSeek V4.1 Flash Hopper paths and Blackwell prefill selection - #41251

Merged

BBuf merged 25 commits into

sgl-project:mainfrom

BBuf:perf/dsv41-h200-core-ops

Oct 3, 2026

Commits on Sep 26, 2026
- [Perf] Tune H200 group32 FP8 dense GEMMs for TP8 decode
  BBuf
  committed
  7c9cf23
- Keep the original short-K GEMM configuration above batch 32
  BBuf
  committed
  e75f407
- [Perf] Bound Hopper FP4 indexer work by live context lengths
  BBuf
  committed
  a49faa1
- Prepare the indexer top-k plan before scoring and test large graph batches
  BBuf
  committed
  5496c9e
- [Perf] Cache exact group32 weight expansions for Hopper prefill GEMMs
  BBuf
  committed
  63d795e
- [Fix] Guard live updates of cached Hopper FP8 weight expansions
  BBuf
  committed
  c15ba96
- [Perf] Use compensated BF16 mHC projections for Hopper prefill
  BBuf
  committed
  a0d87fb
- [Fix] Reject inexact Hopper BF16 weight expansions
  BBuf
  committed
  a99be3e
- [Perf] Fuse Hopper Marlin SwiGLU without changing activation rounding
  BBuf
  committed
  f8e12f6
- [Perf] Fuse Hopper indexer top-k masking and slot mapping
  BBuf
  committed
  2b4827d
- [Perf] Extend measured Hopper mHC fusion and compensated projection paths
  BBuf
  committed
  0d3c45d
- [Perf] Specialize Hopper Marlin TP8 single-token MXFP4 gate tile
  BBuf
  committed
  2a7aff9
- [Perf] Overlap Hopper decode mHC statistics without changing arithmetic
  BBuf
  committed
  c810bea
- [Perf] Defer Hopper MXFP4 expert padding until Marlin repacking
  BBuf
  committed
  d7b4b66
- [Test] Supply quantization metadata in MXFP4 shard fixture
  BBuf
  committed
  361f10d
- [Test] Model Marlin GEMM2 rounding before route weighting
  BBuf
  committed
  60a9fa1
- Adapt Hopper paths to the kernel split and simplify test helpers
  BBuf
  committed
  114ab11
- Preserve Hopper paged indexer optimization across the backend split
  BBuf
  committed
  b96859b
- Check candidate mask reuse at the scoring call
  BBuf
  committed
  f3476d4
- Remove the Hopper group32 linear test file
  BBuf
  committed
  50e39bd
Commits on Oct 2, 2026
- Merge latest main and preserve Hopper and gfx950 mHC dispatch
  BBuf
  committed
  2cd22c3
- Clarify Hopper dispatch and indexer cache contracts
  BBuf
  committed
  79ca023
- test: adapt MXFP4 TP loading coverage to current MoE loader API
  BBuf
  committed
  22e352d
- test: align sparse prefill mock with SWA page-size API
  BBuf
  committed
  94d21e3
- perf: fuse Blackwell flattened prefill index selection
  BBuf
  committed
  179b1ab

Navigation Menu

Repository navigation

[Perf] Optimize DeepSeek V4.1 Flash Hopper paths and Blackwell prefill selection - #41251

[Perf] Optimize DeepSeek V4.1 Flash Hopper paths and Blackwell prefill selection#41251
BBuf merged 25 commits into
sgl-project:mainfrom
BBuf:perf/dsv41-h200-core-ops

Commits on Sep 26, 2026

[Perf] Tune H200 group32 FP8 dense GEMMs for TP8 decode

Keep the original short-K GEMM configuration above batch 32

[Perf] Bound Hopper FP4 indexer work by live context lengths

Prepare the indexer top-k plan before scoring and test large graph batches

[Perf] Cache exact group32 weight expansions for Hopper prefill GEMMs

[Fix] Guard live updates of cached Hopper FP8 weight expansions

[Perf] Use compensated BF16 mHC projections for Hopper prefill

[Fix] Reject inexact Hopper BF16 weight expansions

[Perf] Fuse Hopper Marlin SwiGLU without changing activation rounding

[Perf] Fuse Hopper indexer top-k masking and slot mapping

[Perf] Extend measured Hopper mHC fusion and compensated projection paths

[Perf] Specialize Hopper Marlin TP8 single-token MXFP4 gate tile

[Perf] Overlap Hopper decode mHC statistics without changing arithmetic

[Perf] Defer Hopper MXFP4 expert padding until Marlin repacking

[Test] Supply quantization metadata in MXFP4 shard fixture

[Test] Model Marlin GEMM2 rounding before route weighting

Adapt Hopper paths to the kernel split and simplify test helpers

Preserve Hopper paged indexer optimization across the backend split

Check candidate mask reuse at the scoring call

Remove the Hopper group32 linear test file

Commits on Oct 2, 2026

Merge latest main and preserve Hopper and gfx950 mHC dispatch

Clarify Hopper dispatch and indexer cache contracts

test: adapt MXFP4 TP loading coverage to current MoE loader API

test: align sparse prefill mock with SWA page-size API

perf: fuse Blackwell flattened prefill index selection