Skip to content

[Perf] dsv4: Use the fused MXFP8 producers in eager prefill - #45903

Merged
hnyls2002 merged 1 commit into
mainfrom
lsyin/dsv4-fused-mxfp8-prefill
Oct 22, 2026
Merged

hnyls2002 merged 1 commit into
mainfrom
lsyin/dsv4-fused-mxfp8-prefill

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator
  • fused_mxfp8_rows in deepseek_v4_mhc.py decides when the mHC post and RMSNorm steps write MXFP8 activations directly for the next DeepGEMM consumer instead of writing bf16 and quantizing in a separate kernel.
  • It previously returned False for every extend batch, so eager prefill always paid the separate quantize pass.
  • Extend batches now take the fused path, except inside a breakable prefill graph, where the captured buffers stay bf16. Decode and verify keep the existing row cap, and sequence-parallel batches stay excluded.

No change to the computed values: the fused producer emits the same MXFP8 tensor the separate quantize kernel produced.

Verification: DeepSeek-V4.1-Flash DEP4 eager prefill is faster at large batches; the breakable-graph prefill path is unchanged; GSM8K is unchanged.


CI States

Latest PR Test (Base): 🚫 Run #39861312160
Latest PR Test (Extra): ❌ Run #39861312121
Latest PR Test (AMD ROCm 10): 🚫 Run #39861312192

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_deepseek_v41_flash_gb300_dp2ep2.py test_deepseek_v41_flash_gb300_encoder_bcg.py

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

🚀 2-gpu-gb300 (2 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_dp2ep2.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_encoder_bcg.py

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Oct 22, 2026
@hnyls2002
hnyls2002 merged commit f9b7225 into main Oct 22, 2026
203 of 276 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/dsv4-fused-mxfp8-prefill branch October 22, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant