Repository navigation
[AMD] [GLM5] GLM-5.1 MXFP4 (MI355X) + enable EAGLE for gfx950 in cookbook - #29194
Merged
Merged
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
github-actions
Bot
added
the
documentation
Raiden-Makoto
changed the title
Raiden-Makoto
force-pushed
the
RM/glm51-cookbook-mxfp4
branch
from
June 24, 2026 17:31
89cbd9d to
c3222ba
Compare
Raiden-Makoto
marked this pull request as ready for review
June 24, 2026 17:33
Raiden-Makoto
requested review from
JustinTong0323,
sogalin,
wisclmy0611 and
zijiexia
as code owners
June 24, 2026 17:33
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
- Add the amd/GLM-5.1-MXFP4 checkpoint (tp=4, --kv-cache-dtype fp8_e4m3) as the recommended MI355X (gfx950) path in the deploy generator + ROCm command section, and document the opt-in SGLANG_DSA_TRITON_PREFILL=1 prefill kernel (follow-up to sgl-project#28975). - EAGLE speculative decoding: the old cookbook said it was unsupported on AMD and the generator gated it off for all AMD. We tested EAGLE on gfx950 (MI355X) and it works (lossless, large ITL/throughput win); gfx942 (MI300X/MI325X) is unverified. So the generator now emits the EAGLE flags by default and excludes ONLY MI300X/MI325X (gfx942); the static MXFP4 command and the AMD note reflect this. gfx942 verification can be a follow-up.
Raiden-Makoto
force-pushed
the
RM/glm51-cookbook-mxfp4
branch
from
June 24, 2026 22:32
c3222ba to
e409085
Compare
Raiden-Makoto
changed the title
HaiShaw
approved these changes
Jun 25, 2026
2 tasks done
Chronostasys
pushed a commit
to MindLab-Research/sglang
that referenced
this pull request
Aug 24, 2026
…book (sgl-project#29194) Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The SGLang Cookbook GLM-5.1 page documents AMD deployment only for the BF16/FP8 checkpoints, does not cover the MXFP4 checkpoint (
amd/GLM-5.1-MXFP4) on MI355X (gfx950), and states that EAGLE speculative decoding is unsupported on AMD (with the deploy generator gating it off for all AMD GPUs). This follows up #28975 (opt-in Triton fp8 sparse-MLA prefill kernel for gfx950) and corrects the EAGLE support status for gfx950.Modifications
glm-51-deployment.jsx(interactive deploy generator):MXFP4quantization option, enabled and default on MI355X (gfx950); disabled on other hardware. Addmi355x.mxfp4 = { tp: 4, mem: 0.85 }; generate--model-path amd/GLM-5.1-MXFP4 --tp 4 --kv-cache-dtype fp8_e4m3plus the existing AMD tilelang / chunked-prefill / watchdog flags.GLM-5.1.mdx:MXFP4column to the hardware table (tp=4 on MI355X). Add aMXFP4 (MI355X / gfx950)command block in section 4.2 (prefixed with the optionalSGLANG_DSA_TRITON_PREFILL=1, and including the EAGLE flags).Verification
Built locally with
mint dev/mint validate(build validation passed). EAGLE on MI355X validated separately on hardware (lossless GSM8K, large ITL/throughput improvement). gfx942 (MI300X/MI325X) EAGLE verification can be a follow-up.Scope: GLM-5.1 only; MXFP4 + EAGLE on MI355X (gfx950).
CI States
Latest PR Test (Base): ✅ Run #28133913757
Latest PR Test (Extra): ✅ Run #28151181987