Conversation
superuct
requested review from
BBuf,
Edwardf0t1,
Fridge003,
HaiShaw,
Ying1123,
ch-wan,
ispobock and
merrymercy
as code owners
September 8, 2026 06:58
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
DeepSeek V4 can reach
tf32_hc_prenorm_gemmthroughSGLANG_OPT_DEEPGEMM_HC_PRENORMeven whenSGLANG_ENABLE_JIT_DEEPGEMM=0. In that configuration, the module-level import is skipped and nonempty inputs fail withNameError: name 'deep_gemm' is not defined, despite DeepGEMM being installed. This can also surface during MHC prewarming at model load time.Fixes #29738. This uses the same local-import approach proposed in #29740 and adds CPU regression coverage against the current main branch.
Modifications
Import
deep_gemminsidetf32_hc_prenorm_gemm, after the existing empty-input return. The independently enabled HC-prenorm path now loads its own dependency instead of relying on the JIT gate's module-level import. Empty inputs still return without requiring DeepGEMM, and a missing dependency raises its actual import error.Add three registered CPU tests that execute the real entrypoint module with the JIT gate disabled and stubbed external dependencies. They check kernel argument forwarding, the empty-input fast path, and missing-dependency error propagation. Before the fix, the two nonempty-input tests reproduce the NameError; after the fix, all three pass.
Accuracy Tests
python3 test/registered/unit/layers/test_deep_gemm_hc_prenorm_import.py: 3 passed.uvx pre-commit run --files python/sglang/srt/layers/deep_gemm_wrapper/entrypoint.py test/registered/unit/layers/test_deep_gemm_hc_prenorm_import.py: all applicable hooks passed, including CI registry validation.Speed Tests and Profiling
Not run. The change adds a function-local Python import (cached by Python after the first import); no measured throughput or latency claim is made.
Checklist
CI States
Latest PR Test (Base): ❌ Run #34197052129
Latest PR Test (Extra): ❌ Run #34197051961
Latest PR Test (AMD ROCm 7.2): ❌ Run #34197051914