Skip to content

[AMD] Fix DSV4 JIT build on rocm - #31131

Merged
HaiShaw merged 4 commits into
mainfrom
amd_fix_deepseekv4_0714
Jul 15, 2026
Merged

[AMD] Fix DSV4 JIT build on rocm #31131
HaiShaw merged 4 commits into
mainfrom
amd_fix_deepseekv4_0714

Conversation

@yctseng0211

@yctseng0211 yctseng0211 commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Motivation

The Jul-14 scheduled Nightly Test (AMD MI355X 2N 1P1D Disagg) (run 29305625618) failed on every DeepSeek-V4 config (flash/pro × fp8/fp4 × base/dp8ep8/mtp) at scheduler init:

utils.cuh:245: error: use of undeclared identifier 'cudaDevAttrComputeCapabilityMajor' → ninja exited with status 1 → Rank 0 scheduler died during initialization (exit code: -3)

#30438 added getSMVersion() to jit_kernel/include/sgl_kernel/utils.cuh using the CUDA-only enums cudaDevAttrComputeCapabilityMajor/Minor. The JIT path (tvm-ffi + hipcc) does not run hipify — unlike the compiled sgl-kernel, whose ROCm build hipifies cudaDevAttr* → hipDeviceAttribute* — so on gfx950 those identifiers are undeclared, the DeepSeek-V4 compress-plan JIT fails to build, and the scheduler dies. (Kimi-K2.6 doesn't build this JIT, so it was unaffected.)

Modifications

Adopting the reviewed approach from #31141 (thanks @kangwangamd, @DarkSharpness): instead of guarding the CUDA-only calls with #ifndef USE_ROCM, move getSMVersion out of utils.cuh into runtime.cuh, where the cuda*hip* attribute shims already live — so it compiles on both CUDA and HIP with no guard.

  • Add the cudaDevAttrComputeCapabilityMinorhipDeviceAttributeComputeCapabilityMinor #define (the Major one was already present).
  • Add get_cc_minor() mirroring the existing get_cc_major(), and express get_sm_version(id) = get_cc_major(id) * 10 + get_cc_minor(id) to deduplicate the compute-capability queries.
  • Remove getSMVersion from utils.cuh. It has no caller in the JIT tree (its only reference was its own definition; SM120 gating is done in Python via is_sm120_supported()), so there is no use-site change. No CUDA-path behavior change.

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #29352408580
Latest PR Test (Extra): ✅ Run #29384613963

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@yctseng0211

yctseng0211 commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

Verification run triggered: run 29313160939 on this branch, config dsv4pro-fp8-1k1k-1p1d — the exact config that failed at init in the Jul-14 scheduled nightly (run 29305625618, all DSV4 configs). It should now compile the DeepSeek-V4 compress-plan JIT on ROCm/gfx950 and the scheduler should start.

@yctseng0211 yctseng0211 changed the title [AMD] Guard CUDA-only getSMVersion for ROCm (fix DSV4 JIT build on rocm) [AMD] Fix DSV4 JIT build on rocm - guard CUDA-only getSMVersion for ROCm Jul 14, 2026
@yctseng0211

yctseng0211 commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

https://github.com/sgl-project/sglang/actions/runs/29316462484
AMD CI passed
image

@yctseng0211
yctseng0211 marked this pull request as ready for review July 14, 2026 08:19
@DarkSharpness

Copy link
Copy Markdown
Collaborator

sorry i somehow missed this PR (I review the PR from latest to this...). might be a duplicate of #31141

…get_cc_major)

Adopt the reviewed approach from #31141 (per @DarkSharpness): move the CUDA-only compute-capability query out of utils.cuh into runtime.cuh where the cuda*->hip* attribute shims already live, so it compiles on both CUDA and HIP with no #ifndef USE_ROCM guard. Add the cudaDevAttrComputeCapabilityMinor->hip #define + get_cc_minor, and express get_sm_version(id) = get_cc_major(id) * 10 + get_cc_minor(id). No JIT-tree caller, so no use-site change.

Co-authored-by: kangwangamd <kangwang@amd.com>
@yctseng0211
yctseng0211 force-pushed the amd_fix_deepseekv4_0714 branch from 454a8e4 to 1da00ab Compare July 14, 2026 09:11
@yctseng0211

Copy link
Copy Markdown
Collaborator Author

@yctseng0211

Copy link
Copy Markdown
Collaborator Author

@DarkSharpness Sure, we didn't coordinate on this upfront. I've since synced with @kangwangamd (author of #31141).

The reason I kept this as a separate PR: the MI355X 2N 1P1D disaggregation nightly only runs on a branch inside sgl-project/sglang (the workflow is gated on github.repository == 'sgl-project/sglang'), so we need an upstream branch to actually drive the hardware tests — a fork branch can't trigger it.

So I've brought the reviewed changes from #31141 over to this branch (move getSMVersion into runtime.cuh as get_sm_version, dedup with get_cc_major / get_cc_minor, drop the USE_ROCM guard) and added @kangwangamd as co-author. Happy to close whichever PR you'd prefer to keep.

@yctseng0211

yctseng0211 commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

@yctseng0211 yctseng0211 changed the title [AMD] Fix DSV4 JIT build on rocm - guard CUDA-only getSMVersion for ROCm [AMD] Fix DSV4 JIT build on rocm Jul 14, 2026
@b8zhong

b8zhong commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

/rerun-failed-ci

@yctseng0211

Copy link
Copy Markdown
Collaborator Author

PR Test Base is finished, the only one failure is https://github.com/sgl-project/sglang/actions/runs/29352408580/job/87255423678?pr=31131#logs
which is a known issue

image

irrelevant to this PR

@HaiShaw
HaiShaw merged commit a8b6043 into main Jul 15, 2026
385 of 439 checks passed
@HaiShaw
HaiShaw deleted the amd_fix_deepseekv4_0714 branch July 15, 2026 16:58
roopaksrivastav pushed a commit to roopaksrivastav/sglang that referenced this pull request Jul 16, 2026
Co-authored-by: kangwangamd <kangwang@amd.com>
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
Co-authored-by: kangwangamd <kangwang@amd.com>
jakki-amd pushed a commit to jakki-amd/sglang that referenced this pull request Sep 9, 2026
Co-authored-by: kangwangamd <kangwang@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants