Skip to content

[Klaud Cold] Revert #1512 "Fix glm5-fp8-b300 DeepGemm regression" (experimental-only PR) #1536

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
functionstackx merged 1 commit into from
May 20, 2026
Merged

[Klaud Cold] Revert #1512 "Fix glm5-fp8-b300 DeepGemm regression" (experimental-only PR) #1536

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 7 additions & 11 deletions benchmarks/single_node/glm5_fp8_b300.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,13 @@ nvidia-smi

if [[ "$MODEL" != /* ]]; then hf download "$MODEL"; fi

pip install --break-system-packages --no-deps "transformers==5.2.0" "huggingface-hub==1.4.1"

# Testing @trevor-m's suggestion in sgl-project/sglang#25551 (comment 4481466979):
# downgrade sgl-deep-gemm 0.1.0 → 0.0.1 inside the v0.5.12 container to check
# whether the deepgemm version jump is what causes the B300 TMA-descriptor
# CUDA_ERROR_ILLEGAL_ADDRESS regression. Re-enabling JIT DeepGemm so the
# downgraded version actually runs.
# --break-system-packages required: the container's Python is PEP-668 externally-managed,
# so the previous attempt silently failed and left the bundled 0.1.0 in place.
pip install --break-system-packages --no-deps "sgl-deep-gemm==0.0.1"
export SGL_ENABLE_JIT_DEEPGEMM=1
pip install --no-deps "transformers==5.2.0" "huggingface-hub==1.4.1"

# Workaround for sgl-project/sglang#25551: v0.5.12 DeepGemm TMA-descriptor
# regression on B300 (sm_120) crashes CUDA graph capture with
# CUDA_ERROR_ILLEGAL_ADDRESS. Disabling JIT DeepGemm bypasses the affected
# kernel path. Restore to =1 once the upstream regression is fixed.
export SGL_ENABLE_JIT_DEEPGEMM=0

SERVER_LOG=/workspace/server.log
PORT=${PORT:-8888}
Expand Down
18 changes: 7 additions & 11 deletions benchmarks/single_node/glm5_fp8_b300_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,13 @@ nvidia-smi

if [[ "$MODEL" != /* ]]; then hf download "$MODEL"; fi

pip install --break-system-packages --no-deps "transformers==5.2.0" "huggingface-hub==1.4.1"

# Testing @trevor-m's suggestion in sgl-project/sglang#25551 (comment 4481466979):
# downgrade sgl-deep-gemm 0.1.0 → 0.0.1 inside the v0.5.12 container to check
# whether the deepgemm version jump is what causes the B300 TMA-descriptor
# CUDA_ERROR_ILLEGAL_ADDRESS regression. Re-enabling JIT DeepGemm so the
# downgraded version actually runs.
# --break-system-packages required: the container's Python is PEP-668 externally-managed,
# so the previous attempt silently failed and left the bundled 0.1.0 in place.
pip install --break-system-packages --no-deps "sgl-deep-gemm==0.0.1"
export SGL_ENABLE_JIT_DEEPGEMM=1
pip install --no-deps "transformers==5.2.0" "huggingface-hub==1.4.1"

# Workaround for sgl-project/sglang#25551: v0.5.12 DeepGemm TMA-descriptor
# regression on B300 (sm_120) crashes CUDA graph capture with
# CUDA_ERROR_ILLEGAL_ADDRESS. Disabling JIT DeepGemm bypasses the affected
# kernel path. Restore to =1 once the upstream regression is fixed.
export SGL_ENABLE_JIT_DEEPGEMM=0
export SGLANG_ENABLE_SPEC_V2=1

SERVER_LOG=/workspace/server.log
Expand Down
7 changes: 0 additions & 7 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3050,10 +3050,3 @@
description:
- "Update SGLang image from v0.5.11-cu130 (5d old) to v0.5.12-cu130"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1475

- config-keys:
- glm5-fp8-b300-sglang
- glm5-fp8-b300-sglang-mtp
description:
- "Test @trevor-m's suggestion in sgl-project/sglang#25551: pin sgl-deep-gemm==0.0.1 inside v0.5.12 container to isolate whether the deep-gemm 0.0.1→0.1.0 upgrade is the source of the B300 CUDA_ERROR_ILLEGAL_ADDRESS regression."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1512