Skip to content

kernels: scalar scale A support for fp8_gemm - #33469

Merged
BBuf merged 1 commit into
sgl-project:mainfrom
DevashishLal-CB:dev/dlal/norm-quant-fusion-kernel
Aug 6, 2026
Merged

kernels: scalar scale A support for fp8_gemm#33469
BBuf merged 1 commit into
sgl-project:mainfrom
DevashishLal-CB:dev/dlal/norm-quant-fusion-kernel

Conversation

@DevashishLal-CB

@DevashishLal-CB DevashishLal-CB commented Aug 4, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

AOT kernel changes for #32994

➜ pytest python/sglang/kernels/aot/tests/test_fp8_gemm.py
=============================================================================== test session starts ===============================================================================
platform linux -- Python 3.12.13, pytest-9.1.1, pluggy-1.6.0
rootdir: /data/users/devcode/gitrepos/sglang/python/sglang/kernels/aot
configfile: pyproject.toml
plugins: anyio-4.14.2, cov-7.1.0
collected 817 items

python/sglang/kernels/aot/tests/test_fp8_gemm.py .......................................................................................................................... [ 14%]
........................................................................................................................................................................... [ 35%]
........................................................................................................................................................................... [ 56%]
........................................................................................................................................................................... [ 77%]
........................................................................................................................................................................... [ 98%]
.......s...                                                                                                                                                                 [100%]

========================================================================= 816 passed, 1 skipped in 1.92s ==========================================================================

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ⏳ Run #30874136883
Latest PR Test (Extra): ✅ Run #31022516137

Devashish Lal
kernel changes for sgl-project#32994
a30bec7 
Signed-off-by: Devashish Lal <devcode@fb.com>
@gemini-code-assist

gemini-code-assist Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@DevashishLal-CB DevashishLal-CB mentioned this pull request Aug 4, 2026
Merged
5 tasks
@DevashishLal-CB DevashishLal-CB changed the title kernels: scalar A support for fp8_gemm kernels: scalar scale A support for fp8_gemm Aug 4, 2026
@BBuf ChatGPT Codex Connector

BBuf commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci extra

@BBuf

BBuf commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

https://github.com/sgl-project/sglang/actions/runs/30874136883/job/92399967561?pr=33469

@BBuf
BBuf merged commit fc74c35 into sgl-project:main Aug 6, 2026
269 of 333 checks passed
kfhfar pushed a commit to kfhfar/sglang that referenced this pull request Aug 7, 2026
@DevashishLal-CB Devashish Lal @kfhfar
kernels: scalar scale A support for fp8_gemm (sgl-project#33469)
a600f58 
Signed-off-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Devashish Lal <devcode@fb.com>
b8zhong added a commit that referenced this pull request Aug 8, 2026
@b8zhong
Merge remote-tracking branch 'origin/main' into jit-fp8-per-tensor-sc…
1891cf4 
…aled-mm

Two upstream changes needed carrying into the JIT port:

- #33469 (scalar scale A support for fp8_gemm) landed in the AOT
  fp8_gemm_kernel.cu that this branch deletes. Ported into the JIT tree:
  JitGemmFp8RowwiseC3x gains a ScalarA parameter selecting
  Sm90ScalarBroadcast over Sm90ColBroadcast (one change here covers SM100
  and SM120, which upstream had to patch separately), the sm100/sm120
  dispatchers branch on scales_a.numel() == 1, SM90 goes back to
  Sm90ColOrScalarBroadcast with its runtime flag, and the entry point
  carries upstream's two validation checks. Its tests come along too.

- #33400 (move JIT kernels into namespace sglang) postdates these files,
  so the seven new fp8_per_tensor headers are now wrapped in
  namespace sglang; without it host:: no longer resolves.

Verified on SM103: 1024 passed, 1 skipped (SM89-only test).
saturn-acc pushed a commit to saturn-acc/sglang that referenced this pull request Aug 16, 2026
@DevashishLal-CB Devashish Lal @saturn-acc
kernels: scalar scale A support for fp8_gemm (sgl-project#33469)
0b4395b 
Signed-off-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Devashish Lal <devcode@fb.com>
jakki-amd pushed a commit to jakki-amd/sglang that referenced this pull request Sep 9, 2026
@DevashishLal-CB Devashish Lal @jakki-amd
kernels: scalar scale A support for fp8_gemm (sgl-project#33469)
1fe04a2 
Signed-off-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Devashish Lal <devcode@fb.com>
Atituiset pushed a commit to Atituiset/sglang that referenced this pull request Sep 10, 2026
@DevashishLal-CB Devashish Lal
kernels: scalar scale A support for fp8_gemm (sgl-project#33469)
8909b35 
Signed-off-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Devashish Lal <devcode@fb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BBuf BBuf BBuf approved these changes
@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner
@yizhang2077 yizhang2077 Awaiting requested review from yizhang2077 yizhang2077 is a code owner
@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner
@FlamingoPg FlamingoPg Awaiting requested review from FlamingoPg FlamingoPg is a code owner
@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw is a code owner

Assignees

No one assigned

Labels

blackwell SM100/SM120 kernel nvidia performance quant LLM Quantization run-ci run-ci-extra sgl-kernel

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@DevashishLal-CB @BBuf