Skip to content

[AMD][DeepSeek V4] Set SGLANG_OPT_FLASHMLA_SPARSE_PREFILL to false on hip code path - #30237

Merged
HaiShaw merged 1 commit into
sgl-project:mainfrom
1am9trash:fix-flashmla-flag-condition
Jul 6, 2026
Merged

[AMD][DeepSeek V4] Set SGLANG_OPT_FLASHMLA_SPARSE_PREFILL to false on hip code path#30237
HaiShaw merged 1 commit into
sgl-project:mainfrom
1am9trash:fix-flashmla-flag-condition

Conversation

@1am9trash

@1am9trash 1am9trash commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

co-authored: @bingxche, @michaelzhang-ai

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #28777920839
Latest PR Test (Extra): ❌ Run #28777920522

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the DeepSeek-V4 hook for ROCm/HIP platforms by removing the check for whether the SGLANG_OPT_FLASHMLA_SPARSE_PREFILL environment variable is set. This change unconditionally disables the sparse prefill kernel on ROCm/HIP, preventing users from explicitly overriding this behavior. The reviewer recommends restoring the check to allow explicit overrides as documented in the warning message.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread python/sglang/srt/arg_groups/deepseek_v4_hook.py
@HaiShaw
HaiShaw merged commit 80decc7 into sgl-project:main Jul 6, 2026
91 of 106 checks passed
@michaelzhang-ai

michaelzhang-ai commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Post-merge validation note (full DSV4-Flash disagg matrix on image ...-mi35x-20260706, which contains this PR):

Run: https://github.com/sgl-project/sglang/actions/runs/28817206537

This PR does what it intends — every prefill + decode log across all 8 flash configs shows Disabling SGLANG_OPT_FLASHMLA_SPARSE_PREFILL ... on ROCm/HIP, i.e. sparse prefill is confirmed off at runtime.

However, DeepSeek-V4-Flash disagg is still failing, and with sparse prefill confirmed disabled the cause is now clearly a separate MORI/RDMA transport issue, not the sparse path:

[application][error] RegisterRdmaMemoryRegion failed! size:7345537024, accessFlag:15, errno:22 (Invalid argument)
Fatal Python error: Aborted   (sglang/srt/utils/watchdog.py)
→ prefill worker aborts → "PD path not serving"

This is uniform across all 8 flash configs (fp8/fp4 × base/mtp/dp8ep8/dp8ep8-mtp), so it's systematic — a ~6.8 GB RDMA memory-region registration failing with EINVAL. Likely needs the MORI/disagg owners; cc @HaiShaw @1am9trash @bingxche @yctseng0211.

Net: #30237 correctly disables sparse prefill on HIP, but does not by itself green the Flash disagg nightly because of this RDMA registration failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants