Skip to content

[AMD] Add GLM-5.3-Flash recipes for MI300X, MI325X, and MI355X - #36608

Merged
zijiexia merged 1 commit into
sgl-project:mainfrom
andyluo7:docs/glm-5.3-flash-amd
Aug 27, 2026
Merged

zijiexia merged 1 commit into
sgl-project:mainfrom
andyluo7:docs/glm-5.3-flash-amd

Conversation

@andyluo7

@andyluo7 andyluo7 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Motivation

Add copy-pasteable AMD ROCm deployment recipes for zai-org/GLM-5.3-Flash alongside the existing NVIDIA recipes.

The AMD commands require the engine changes in #36607, which is stacked on the model-support PR #36507.

Modifications

  • Add single-node TP8 ROCm recipes for MI300X, MI325X, and MI355X.
  • Use the validated BF16 KV + TileLang DSA configuration with SGLANG_USE_AITER=1, Triton MoE, and CUDA graphs disabled.
  • Mark MI300X and MI355X as verified from full GSM8K runs.
  • Keep MI325X explicitly unverified: it shares gfx942 with MI300X, but it was not measured directly.
  • Disable the unvalidated AMD MTP/low-latency lane and unvalidated backend combinations.
  • Add accuracy-only benchmark records with exact model revision, engine PR/source manifest, image, and request accounting. No throughput or latency result is presented.

This PR is based on current main (02dfd3782601277bcf7e74c7a836a70011183165). It does not duplicate the separate NVIDIA HiCache/fusion cleanup in #36544.

Accuracy Tests

GPU Recipe status Full GSM8K Request accounting
MI300X / gfx942 Verified 1,284 / 1,319 = 97.35% 1,319 unique records; 100% stop; 0 errors, empty generations, or truncations
MI325X / gfx942 Not verified; inferred from MI300X No MI325X hardware run
MI355X / gfx950 Verified 1,288 / 1,319 = 97.65% 1,319 unique records; 100% stop; 0 errors, empty generations, or truncations

Validated model revision: 3f1971b7b5f7a528c9c4ef6212c8785298a8c24a.

Validated engine source:

  • PR [AMD] Enable GLM-5.3-Flash on gfx942 and gfx950 #36607 head: 9e692c9216c3b5e5c443fecf6b995700eb68d2e4
  • Source manifest SHA256: 2c240e0e01d5fdf04acc485ebfa25f8a1793ba45fb07f165eecedfba7ec1db80
  • MI300X image: lmsysorg/sglang:v0.5.18-rocm720-mi30x
  • MI355X image: lmsysorg/sglang:v0.5.18-rocm720-mi35x

Speed Tests and Profiling

Not measured. These are accuracy/correctness results only.

Documentation validation

  • node docs/scripts/check_cookbook_configs.mjs
  • focused pre-commit run --files ...
  • npx --yes mintlify broken-links
  • npx --yes mintlify validate

All passed. The final rebase changed only the parent with an unrelated test-file commit; the validated docs/ tree hash remained fc5d92e264b3271bd347f5acd91e06dde38fba2f.

Checklist

  • Document platform, image, model revision, TP/DP topology, and required environment settings.
  • Mark measured and inferred hardware separately.
  • Run the cookbook config checker and docs validation.
  • Avoid performance claims without a matched throughput benchmark.

CI States

Latest PR Test (Base): ✅ Run #33037066265
Latest PR Test (Extra): ❌ Run #33037402765
Latest PR Test (AMD ROCm 7.2): ➖ No AMD PR run found for this commit.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 27, 2026
@zijiexia
zijiexia enabled auto-merge (squash) August 27, 2026 05:12
@zijiexia
zijiexia merged commit 0f7b5b8 into sgl-project:main Aug 27, 2026
114 of 123 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation run-ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants