Repository navigation
[GLM-5.2] Mark MI350X verified - #1065
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Verified on 8x MI350X with vllm/vllm-openai-rocm:nightly: the AMD FP8 command serves --max-model-len 524288 with a 2,205,424-token KV cache and 4.21x maximum concurrency, MTP's mean acceptance length is 3.62 out of 5, the glm47 reasoning and tool parsers work, needle retrieval is correct from 9.5K to 105K prompt tokens, and the recipe's benchmark completes 32/32. The command builder's MI350X command (Rust frontend, full 1M context) also serves. Signed-off-by: Jin Tao <jin.tao@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the GLM-5.2.yaml model configuration to add support and documentation for the AMD MI350X GPU. Specifically, it marks the mi350x as verified, updates the last modified date, and adds performance metrics and references for running FP8 on 8x MI350X. There are no review comments, and I have no feedback to provide.
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
Review effort: Lite
Findings: None
What changed in this PR
Updates the GLM-5.2 model YAML to reflect MI350X validation status and document MI350X-specific performance notes for the AMD FP8 serving recipe.
Changes:
- Marks
mi350xasverifiedin the supported hardware matrix. - Updates the AMD FP8 guide section title to include MI350X.
- Adds measured KV-cache/concurrency and MTP acceptance metrics for 8× MI350X.
| File | Description |
|---|---|
| models/zai-org/GLM-5.2.yaml | Updates metadata and guide text to include MI350X verification and adds MI350X benchmark notes for the FP8 AMD recipe. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
What this changes
The MI350X counterpart of #1036. All in
models/zai-org/GLM-5.2.yaml:mi350x: verified.The serve command and the AMD image (
vllm/vllm-openai-rocm:nightly) are unchanged.Validation on 8x MI350X (gfx950),
vllm/vllm-openai-rocm:nightlyvLLM
0.30.1rc1.dev558+g0cbac6cd1(nightly-0cbac6cd1305f710e12193596b27488397bcb205, the currentnightly),zai-org/GLM-5.2-FP8atf33c6dc. The guide's AMD FP8 command, as written:--max-model-len 524288at--gpu-memory-utilization 0.80: 2,205,424-token KV cache, 4.21x maximum concurrencyglm47reasoning parser fillsreasoning;chat_template_kwargs: {"enable_thinking": false}returns plain content withreasoning_tokens: 0;reasoning_efforthigh and max both answer correctlyglm47tool parser returns a well-formedget_weathercall, and the follow-up turn with the tool result answers correctly--no-enable-prefix-caching: 32/32 requests, 423.73 output tok/s, median TTFT 22.2 s, median TPOT 34.3 msThe command builder's MI350X command (Rust frontend, FP8, MTP on, default
--max-model-len) serves the full 1,048,576-token window with a 2,199,520-token KV cache (2.10x) and passes the same reasoning, tool-calling, needle and MTP checks (mean acceptance length 3.35).