Skip to content

[Hotfix] Fix router gemm on sm103 - #22134

Merged
Kangyan-Zhou merged 1 commit into
mainfrom
fix-router-gemm
Apr 5, 2026
Merged

Kangyan-Zhou merged 1 commit into
mainfrom
fix-router-gemm

Conversation

@Fridge003

Copy link
Copy Markdown
Collaborator

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

@Fridge003

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_deepseek_v3_fp4_4gpu.py

@github-actions

github-actions Bot commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

4-gpu-b200 (1 test): View workflow run

cd test/ && python3 registered/quant/test_deepseek_v3_fp4_4gpu.py

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the DeepSeek-V2 model implementation to restrict a specialized FlashInfer router GEMM kernel optimization specifically to SM 100 devices. The reviewer noted that this change might unnecessarily exclude other Blackwell variants from the optimization and suggested either using a more inclusive check or adding documentation to justify the restriction.

and _device_sm >= 90
):
if _device_sm >= 100 and self.weight.shape[0] == 256:
if _device_sm == 100 and self.weight.shape[0] == 256:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The change from _device_sm >= 100 to _device_sm == 100 effectively disables the specialized FlashInfer router GEMM kernel for Blackwell variants other than SM 100 (such as SM 103 mentioned in the PR title). While this correctly addresses the reported issue on SM 103 by falling back to the dsv3_router_gemm kernel, it also excludes any other potential Blackwell variants (e.g., SM 101) from this optimization. If other variants are known to be compatible, consider using a more inclusive check or adding a comment explaining why this optimization is strictly limited to SM 100.

@Kangyan-Zhou
Kangyan-Zhou merged commit c5fa364 into main Apr 5, 2026
56 of 64 checks passed
@Kangyan-Zhou
Kangyan-Zhou deleted the fix-router-gemm branch April 5, 2026 16:33
Kangyan-Zhou pushed a commit that referenced this pull request Apr 5, 2026
JustinTong0323 pushed a commit to JustinTong0323/sglang that referenced this pull request Apr 7, 2026
caitengwei pushed a commit to caitengwei/sglang that referenced this pull request Jun 1, 2026
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants