[AMD] GLM-5.2 MI355X MXFP4: bump image to 20260913, enable TOPK_V2 - #39406
Merged
Merged
Conversation
ChangLiu0709
requested review from
JustinTong0323,
sogalin,
wisclmy0611 and
zijiexia
as code owners
September 14, 2026 10:14
github-actions
Bot
added
the
documentation
- Update Docker image from v0.5.19-rocm720-mi35x-20260910 to v0.5.19-rocm720-mi35x-20260913 for the MI355X MXFP4 cells. - Add SGLANG_OPT_USE_TOPK_V2=true env to all four MXFP4 cells (low-latency, balanced, high-throughput, mtp-314): the v2 fused top-k kernel is now available for GLM-5.x on ROCm after upstream sgl-project#36684 and sgl-project#36851. - Update the MTP validation note in the MDX cookbook page to reference the new image and document the TOPK_V2 enablement. Upstream InferenceX PR: SemiAnalysisAI/InferenceX#3081
ChangLiu0709
force-pushed
the
amd/glm52-mi355x-topkv2
branch
from
September 14, 2026 10:33
a1b5b4b to
6e74a21
Compare
Add a documentation note explaining the two-arm InferenceX sweep from SemiAnalysisAI/InferenceX#3081: - TP4/EP4 + HiCache DRAM offload at concurrency >= 4 - TP8/EP1 without KV offloading at low concurrency (c <= 2) The portable cookbook commands intentionally omit HiCache knobs; users can toggle them via the Playground's Hierarchical KV Cache card.
1am9trash
approved these changes
Sep 14, 2026
1am9trash
left a comment
1am9trash
left a comment
Collaborator
There was a problem hiding this comment.
LGTM,
cc @Jacob0226, @EricKing626
This reverts commit 4ea02a0.
sogalin
approved these changes
Sep 14, 2026
Collaborator
|
LGTM, SGLANG_OPT_USE_TOPK_V2=1 can help improve the performance for GLM. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This PR syncs the GLM-5.2 MI355X MXFP4 cookbook recipes with SemiAnalysisAI/InferenceX#3081.
Changes
docs/src/snippets/configs/zai-org/glm-5.2.jsxlmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260910→v0.5.19-rocm720-mi35x-20260913(themi355x|mxfp4entry indockerImages).SGLANG_OPT_USE_TOPK_V2=true: added to all four MXFP4 cells (low-latency, balanced, high-throughput, mtp-314). The v2 fused top-k kernel is now available for GLM-5.x on ROCm after upstream [AMD] Enable deepseek-v4 topk_transform v2 kernel #36684 and [AMD] Enable topk v2 GLM ROCm #36851.docs/cookbook/autoregressive/GLM/GLM-5.2.mdx20260910to20260913and documented theSGLANG_OPT_USE_TOPK_V2=trueenablement with links to the upstream PRs.What is NOT changed
CI States
Latest PR Test (Base): ✅ Run #34834663886
Latest PR Test (Extra): ❌ Run #34834663594
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.