[AMD] Update v4 amd cookbook - #28423
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds comprehensive support and documentation for running DeepSeek-V4 on AMD ROCm GPUs (specifically MI300X and MI355X). It introduces AMD-specific Docker run configurations, updates the deployment snippet to dynamically generate the correct Docker flags based on the GPU vendor, and adds benchmark mappings and detailed deployment configurations (including environment variables and CLI flags) for various quantization strategies (FP4 and FP8) on AMD hardware. I have no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Motivation
In the current SGLang Cookbook, the DeepSeek-V4 model only documents Docker / server commands for NVIDIA GPUs. This PR adds the AMD GPU (MI300X, MI355X) variants.
Modifications
deepseek-v4.jsx: MI300X (Flash FP8) and MI355X (Flash / Pro, FP4 / FP8), single-node, TP-only at low concurrency and TP+DP at high concurrency, with EAGLE MTP.modelNames(FP4:deepseek-ai/..., FP8:sgl-project/...-FP8) and add thelmsysorg/sglang-rocmROCm images todockerImages._deployment.jsxDocker wrapper vendor-aware: AMD emits--device=/dev/kfd/--device=/dev/dri/--group-add videoinstead of--gpus all.DeepSeek-V4.mdx: add an AMD ROCm Docker install section and an AMD configuration note.Verification
Built the docs locally with
mint devand confirmed the DeepSeek-V4 page renders correctly (Deploy matrix, AMD cells, and Docker command generation all preview without issues).Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ✅ Run #27608625543
Latest PR Test (Extra): ❌ Run #27608625058