Skip to content

[Fix] [Intel] dp-attention: Skip the CustomAllReduceV2 import on XPU - #46417

Merged
mingfeima merged 2 commits into
sgl-project:mainfrom
arathi-hlab:fix/xpu-dp-attention-tvm-ffi-import
Oct 26, 2026
Merged

mingfeima merged 2 commits into
sgl-project:mainfrom
arathi-hlab:fix/xpu-dp-attention-tvm-ffi-import

Conversation

@arathi-hlab

@arathi-hlab arathi-hlab commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

#46138 added a module-level from sglang.srt.distributed.device_communicators.custom_all_reduce_v2 import CustomAllReduceV2 to dp_attention.py. custom_all_reduce_v2 imports sglang/kernels/ops/communication/all_reduce.py, which does import tvm_ffi at module level. tvm_ffi is not installed in the XPU image (intel/sglang-dev:latest), and dp_attention.py is imported almost everywhere, so importing sglang on XPU now fails. For example, the XPU diffusion job dies before running any test:

File ".../sglang/srt/layers/dp_attention.py", line 20, in <module>
  from sglang.srt.distributed.device_communicators.custom_all_reduce_v2 import (
File ".../sglang/kernels/ops/communication/all_reduce.py", line 7, in <module>
  import tvm_ffi
ModuleNotFoundError: No module named 'tvm_ffi'

https://github.com/sgl-project/sglang/actions/runs/40179531469/job/121127481365

Modifications

On XPU only, skip the import and bind CustomAllReduceV2 = (). The class's only use is isinstance(comm, CustomAllReduceV2) in _decode_sum_len_fits_custom_all_reduce, and isinstance(x, ()) is always False. That is correct on XPU, which cannot create a v2 custom all-reduce.

Checklist

  • pre-commit passes
  • XPU CI green

CI States

Latest PR Test (Base): ✅ Run #40180912773
Latest PR Test (Extra): ❌ Run #40180912700
Latest PR Test (AMD ROCm 10): ❌ Run #40180912717

sgl-project#46138 added a module-level CustomAllReduceV2 import to dp_attention.py.
custom_all_reduce_v2 pulls in kernels/ops/communication/all_reduce.py,
which does "import tvm_ffi". tvm_ffi is not installed in the XPU image, so
importing sglang now fails on XPU, e.g. the diffusion test runner:

  ModuleNotFoundError: No module named 'tvm_ffi'

On XPU only, skip the import and bind CustomAllReduceV2 to an empty tuple,
so the isinstance check in _decode_sum_len_fits_custom_all_reduce returns
False (XPU never uses the v2 custom all-reduce). Other platforms keep the
module-level import unchanged.
@arathi-hlab arathi-hlab added run-ci CI: run the baseline test suite on this PR xpu intel gpu with device `torch.xpu` intel labels Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

intel run-ci CI: run the baseline test suite on this PR xpu intel gpu with device `torch.xpu`

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants