Skip to content

[XPU] Fix causal_conv1d_update_xpu signature for GDN speculative decode - #397

Closed
dayanandav wants to merge 1 commit into
sgl-project:mainfrom
dayanandav:fix/causal-conv1d-update-spec-decode-kwargs
Closed

dayanandav wants to merge 1 commit into
sgl-project:mainfrom
dayanandav:fix/causal-conv1d-update-spec-decode-kwargs

Conversation

@dayanandav

Copy link
Copy Markdown

Motivation

Serving a hybrid GDN model (e.g. Qwen3.5) on XPU with speculative decoding crashes during warmup:

TypeError: causal_conv1d_update_xpu() got an unexpected keyword argument 'intermediate_conv_window'

The GDN target-verify path in sglang (python/sglang/srt/layers/attention/linear/gdn_backend.py) calls
causal_conv1d_update with five speculative-decoding tree arguments:

  • intermediate_conv_window
  • intermediate_state_indices
  • retrieve_next_token
  • retrieve_next_sibling
  • retrieve_parent_token

causal_conv1d_update_xpu only accepted arguments up to pad_slot_id, so the call raises TypeError
before reaching the kernel. Only the TARGET_VERIFY path is affected; non-speculative decode is fine.

Fixes sgl-project/sglang#34720

Modifications

  • python/sgl_kernel/mamba.py: causal_conv1d_update_xpu now accepts and ignores unknown keyword
    arguments via **kwargs, matching what causal_conv1d_fn_xpu in the same file already does. This
    keeps the XPU wrapper tolerant of upstream signature changes instead of hard-failing at warmup.
  • tests/test_causal_conv1d.py: new test_causal_conv1d_update_ignores_spec_decode_kwargs regression
    test that reproduces the failing call site (all five kwargs) and asserts both the output and the
    in-place conv_state update still match the reference.

Note for reviewers

The tree-topology arguments are accepted but not honored by the SYCL kernel — no intermediate conv
window is checkpointed per draft token. This unblocks the crash and is correct for linear-chain draft
topologies, but tree-based drafting (topk > 1) would need the checkpointing implemented in
src/sycl/causal_conv1d.cpp to be numerically correct. Happy to follow up with either the kernel-side
support or an explicit guard/warning for the tree case if maintainers prefer failing loudly over
silently ignoring.

Accuracy Test

tests/test_causal_conv1d.py on Intel XPU — full file, including the new test:

761 passed in 16.33s

Benchmark & Profiling

Not applicable — signature-only change, no kernel or performance impact.

Checklist

🤖 Generated with Claude Code

The GDN target-verify path in sglang calls `causal_conv1d_update` with
speculative-decoding tree arguments (`intermediate_conv_window`,
`intermediate_state_indices`, `retrieve_next_token`,
`retrieve_next_sibling`, `retrieve_parent_token`). The XPU wrapper only
accepted arguments up to `pad_slot_id`, so serving a hybrid GDN model
(e.g. Qwen3.5) with `--device xpu --speculative-algorithm NEXTN` crashed
during warmup with:

    TypeError: causal_conv1d_update_xpu() got an unexpected keyword
    argument 'intermediate_conv_window'

Accept and ignore unknown keyword arguments, matching what
`causal_conv1d_fn_xpu` already does, so the XPU wrapper stays tolerant of
upstream signature changes. Add a regression test covering the kwargs
from the failing call site.

Fixes sgl-project/sglang#34720

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dayanandav

Copy link
Copy Markdown
Author

I recommend this solution on kernel side, @jiayisunx @Xia-Weiwen requested for your review.

@jiayisunx

Copy link
Copy Markdown
Collaborator

Currently, we don't need to optimize the SYCL conv1d kernels anymore. As Weiwen mentioned, there will be two backends for linear attention on XPU: Triton and intel_xpu. For the Triton backend, we always use Triton kernels, so I removed the SYCL conv1d kernels from SGLang in [GDN] remove XPU path of causal_conv1d_fn and causal_conv1d_update by jiayisunx · Pull Request #3577…. Therefore, the SYCL conv1d kernels are no longer used by SGLang.

@jiayisunx jiayisunx closed this Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] [XPU] Qwen3.5 GDN + speculative decode: causal_conv1d_update_xpu() got an unexpected keyword argument 'intermediate_conv_window'

2 participants