Skip to content

[AMD] Fix DeepSeekV4 server cutlass error #30374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
HaiShaw merged 5 commits into from
Jul 7, 2026
Merged

[AMD] Fix DeepSeekV4 server cutlass error #30374

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
494629a
fix
At1a8 Jul 7, 2026
619b7d3
rm
At1a8 Jul 7, 2026
cfe60e3
Merge branch 'main' into fangyuan/fix_dsv4_server_cutlass_error
At1a8 Jul 7, 2026
769827e
fix
At1a8 Jul 7, 2026
37679e4
Merge branch 'main' into fangyuan/fix_dsv4_server_cutlass_error
b8zhong Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion python/sglang/jit_kernel/dsa/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
from .cutedsl_paged_mqa_logits import CuteDSLPagedMQALogitsRunner, pick_dsl_expand
from sglang.srt.utils import is_hip

from .paged_mqa_logits import (
aiter_paged_mqa_logits,
cutedsl_paged_mqa_logits,
deepgemm_paged_mqa_logits_native,
deepgemm_paged_mqa_logits_split,
)

if not is_hip():
# Preserve the original eager import behavior on non-ROCm platforms.
from .cutedsl_paged_mqa_logits import CuteDSLPagedMQALogitsRunner, pick_dsl_expand

__all__ = [
"CuteDSLPagedMQALogitsRunner",
"pick_dsl_expand",
Expand Down
7 changes: 6 additions & 1 deletion python/sglang/srt/layers/attention/dsa/dsa_indexer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
cutedsl_paged_mqa_logits,
deepgemm_paged_mqa_logits_native,
deepgemm_paged_mqa_logits_split,
pick_dsl_expand,
)
from sglang.jit_kernel.fused_store_index_cache import (
can_use_dsa_fused_store,
Expand Down Expand Up @@ -65,6 +64,11 @@
_is_cuda = is_cuda()
_is_hip = is_hip()
_is_npu = is_npu()
if not _is_hip:
# Preserve the original eager import behavior on non-ROCm platforms.
from sglang.jit_kernel.dsa import pick_dsl_expand
Comment on lines +67 to +69

@gemini-code-assist gemini-code-assist Bot Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When _is_hip is True, pick_dsl_expand is not imported and remains completely undefined in the module namespace. Although the code path that uses it (use_cute_dsl) is not reachable on ROCm, static analysis tools (such as Pyright, mypy, or flake8) will still flag pick_dsl_expand as an undefined name.\n\nDefining a fallback value of None when _is_hip is True resolves this static analysis issue.

if not _is_hip:\n    # Preserve the original eager import behavior on non-ROCm platforms.\n    from sglang.jit_kernel.dsa import pick_dsl_expand\nelse:\n    pick_dsl_expand = None

@kkHuang-amd kkHuang-amd Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check this review comment and fix that

@At1a8 At1a8 Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check this review comment and fix that

Thanks for your comments! I've addressed the issue based on your suggestions.
image

else:
pick_dsl_expand = None
_use_aiter = get_bool_env_var("SGLANG_USE_AITER") and _is_hip
_is_fp8_fnuz = is_fp8_fnuz()
_is_gfx95_supported = is_gfx95_supported()
Expand Down Expand Up @@ -906,6 +910,7 @@ def _get_topk_paged(
and forward_batch.forward_mode.is_target_verify()
and next_n >= 2
):
assert pick_dsl_expand is not None, "Not supported on AMD/ROCm. "
dsl_expand_factor, dsl_atom = pick_dsl_expand(
next_n,
batch_size=B,
Expand Down
Loading