[Hotfix] Fix router gemm on sm103 - #22134
Conversation
|
/rerun-test test_deepseek_v3_fp4_4gpu.py |
|
✅ |
There was a problem hiding this comment.
Code Review
This pull request modifies the DeepSeek-V2 model implementation to restrict a specialized FlashInfer router GEMM kernel optimization specifically to SM 100 devices. The reviewer noted that this change might unnecessarily exclude other Blackwell variants from the optimization and suggested either using a more inclusive check or adding documentation to justify the restriction.
| and _device_sm >= 90 | ||
| ): | ||
| if _device_sm >= 100 and self.weight.shape[0] == 256: | ||
| if _device_sm == 100 and self.weight.shape[0] == 256: |
There was a problem hiding this comment.
The change from _device_sm >= 100 to _device_sm == 100 effectively disables the specialized FlashInfer router GEMM kernel for Blackwell variants other than SM 100 (such as SM 103 mentioned in the PR title). While this correctly addresses the reported issue on SM 103 by falling back to the dsv3_router_gemm kernel, it also excludes any other potential Blackwell variants (e.g., SM 101) from this optimization. If other variants are known to be compatible, consider using a more inclusive check or adding a comment explaining why this optimization is strictly limited to SM 100.
Motivation
Modifications
Accuracy Tests
Speed Tests and Profiling
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ci