Skip to content

[AMD][bugfix] Fix dsv4 server launch - #39875

Merged
HaiShaw merged 1 commit into
sgl-project:mainfrom
RolaoDenthu:dsv4/fix-error
Sep 17, 2026
Merged

HaiShaw merged 1 commit into
sgl-project:mainfrom
RolaoDenthu:dsv4/fix-error

Conversation

@RolaoDenthu

@RolaoDenthu RolaoDenthu commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

#39652 (13d593b6cf) breaks the JIT build. It added the V4.1 row quantizers behind#ifndef USE_ROCM in kv_layout.cuh,
branch that a V4 instantiation discards, but a discarded branch is still parsed and the
name does not depend on the template parameters, so lookup runs on it whatever the layout
being compiled, and every ROCm build of those headers fails:

fused_norm_rope_v2.cuh:507:18: error: no member named 'v41' in namespace 'sglang::deepseek_v4'

#37810 (a813224e78) breaks target-verify metadata. It added the req_pool_indices_repeated parameter to _attach_unified_kv_prefill_meta and updated one of its two call sites. The other one, in make_forward_metadata_from_raw_verify, is left one positional argument short, so num_tokens never binds and the call raises TypeError.

Modifications

  • kv_layout.cuh: declare a v41::store_row stub under USE_ROCM so the name resolves there. Its static_assert depends on kLayout, so it fires only when a kernel really instantiates a V4.1 layout — which the Python side already refuses on ROCm (compress.py, attn.py and elementwise.py all assert "the V4.1 KV layouts are CUDA (sm100) only"). Nothing inside the #ifndef USE_ROCM branch changed, so CUDA codegen is untouched.
  • deepseek_v4_backend_hip_radix.py: pass req_pool_indices_repeated from make_forward_metadata_from_raw_verify as well.

Accuracy Tests

gsm8k 1319 acc: 0.947

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #35168940597
Latest PR Test (Extra): ❌ Run #35168940370
Latest PR Test (AMD ROCm 10): ❌ Run #35168940507

-Declare v41::store_row on ROCm to fix the JIT build
-Pass req_pool_indices_repeated when building verify metadata
@HaiShaw

HaiShaw commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

cc @AMD-yanfeiwang @hnyls2002

@HaiShaw

HaiShaw commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

HIP/ROCm specific

@HaiShaw
HaiShaw merged commit 7780882 into sgl-project:main Sep 17, 2026
103 of 119 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants