Skip to content

[AMD] Update v4 amd cookbook - #28423

Merged
HaiShaw merged 1 commit into
sgl-project:mainfrom
1am9trash:update-v4-amd-cookbook
Jun 17, 2026
Merged

HaiShaw merged 1 commit into
sgl-project:mainfrom
1am9trash:update-v4-amd-cookbook

Conversation

@1am9trash

@1am9trash 1am9trash commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Motivation

In the current SGLang Cookbook, the DeepSeek-V4 model only documents Docker / server commands for NVIDIA GPUs. This PR adds the AMD GPU (MI300X, MI355X) variants.

Modifications

  • Add AMD verified cells to deepseek-v4.jsx: MI300X (Flash FP8) and MI355X (Flash / Pro, FP4 / FP8), single-node, TP-only at low concurrency and TP+DP at high concurrency, with EAGLE MTP.
  • Map AMD checkpoints in modelNames (FP4: deepseek-ai/..., FP8: sgl-project/...-FP8) and add the lmsysorg/sglang-rocm ROCm images to dockerImages.
  • Make the _deployment.jsx Docker wrapper vendor-aware: AMD emits --device=/dev/kfd/--device=/dev/dri/--group-add video instead of --gpus all.
  • Update DeepSeek-V4.mdx: add an AMD ROCm Docker install section and an AMD configuration note.

Verification

Built the docs locally with mint dev and confirmed the DeepSeek-V4 page renders correctly (Deploy matrix, AMD cells, and Docker command generation all preview without issues).

image

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ✅ Run #27608625543
Latest PR Test (Extra): ❌ Run #27608625058

@github-actions github-actions Bot added documentation Improvements or additions to documentation deepseek labels Jun 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds comprehensive support and documentation for running DeepSeek-V4 on AMD ROCm GPUs (specifically MI300X and MI355X). It introduces AMD-specific Docker run configurations, updates the deployment snippet to dynamically generate the correct Docker flags based on the GPU vendor, and adds benchmark mappings and detailed deployment configurations (including environment variables and CLI flags) for various quantization strategies (FP4 and FP8) on AMD hardware. I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@HaiShaw
HaiShaw merged commit 0d651e6 into sgl-project:main Jun 17, 2026
77 of 81 checks passed
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
jakki-amd pushed a commit to jakki-amd/sglang that referenced this pull request Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants