Fix Qwen3.5 GDN multi-item scoring

[daii-0818]

Motivation

Qwen3.5 hybrid models use recurrent GDN layers alongside full-attention layers. Multi-item scoring already gives the full-attention layers an item-branching mask, but the GDN prefill path previously processed the packed suffix as one continuous recurrent sequence. As a result, later items could inherit convolution and SSM state from earlier items.

Fixes #31969.

Modifications

• Add a read-only final-state mode to the Triton chunk gated-delta-rule path while preserving in-place updates by default.
• Build GDN-specific MIS metadata once per forward for compact query and item segments.
• Run GDN MIS prefill in two phases: compute each request query once, then branch every item from the query-end convolution and SSM states.
• Keep item recurrence read-only against the request SSM pool and use unique cloned convolution slots per item.
• Reject unsupported combinations, including non-Triton GDN prefill, page-major layout, cached prefixes, non-prefill requests, and hybrid linear backends without MIS support.
• Add kernel, causal-convolution, backend, metadata, validation, and Qwen3.5 end-to-end coverage.

Reproduction

python3 -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-4B \
  --trust-remote-code \
  --dtype bfloat16 \
  --enable-mis \
  --attention-backend flashinfer \
  --linear-attn-prefill-backend triton \
  --disable-radix-cache \
  --disable-cuda-graph \
  --chunked-prefill-size -1 \
  --mem-fraction-static 0.8 \
  --host 0.0.0.0 \
  --port 30000

Accuracy Tests

Tested on one NVIDIA H20 with Qwen/Qwen3.5-4B in BF16, FlashInfer full attention, and Triton linear-attention prefill.

• Qwen3.5 MIS end-to-end: 3 tests passed, covering empty queries, concurrent requests, varied item counts and lengths, and sibling modification/reordering.
• Focused recurrent, causal-convolution, GDN backend, metadata, and validation suite: 27 tests passed with 46 subtests.
• Full recurrent and causal-convolution test files: 275 tests passed.
• Multi-item scores match per-item MIS scoring with atol=2e-2, rtol=2e-2.
• Formatting, import checks, spelling, syntax compilation, CI registration validation, and whitespace checks passed.

Speed Tests and Profiling

Single H20, Qwen/Qwen3.5-4B, query length 120, item length 180, 10 items per request, 10 warmup requests, followed by three 60-second runs. CUDA graph, radix cache, and chunked prefill were disabled in both modes.

Mode	Items/s/GPU	P50 latency	P99 latency
Independent pointwise scoring	52.80	189.06 ms	195.25 ms
Multi-item scoring	62.75	158.74 ms	167.21 ms

This is an 18.84% throughput increase, with P50 latency reduced by 16.04% and P99 latency reduced by 14.36%.

Checklist

• Format the code with the repository formatters and static checks.
• Add unit and end-to-end tests.
• Documentation is not required because this does not change the public API or Score API schema.
• Provide accuracy and speed benchmark results.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger the required CI tests.
4. After green CI and required approvals, ask Merge Oncalls to merge the PR.

---

CI States

Latest PR Test (Base): ⏳ Run #34441014344
Latest PR Test (Extra): ❌ Run #34441014096
Latest PR Test (AMD ROCm 10): ⏳ Run #34441014287

[yuan-luo]

Please account for token padding here.
In DP-attention / attention-TP padding case may fail. DP-attention / attention-TP may pad forward_batch.input_ids via _pad_inputs_to_size() without extending extend_seq_lens_cpu, input_ids.numel() can legitimately exceed sum(seq_lens), causing valid MIS requests to fail with this ValueError.

[daii-0818]

Thanks for catching this. Fixed in 29af803.

The metadata validation now allows trailing DP-attention / attention-TP padding while still rejecting logical sequence lengths that exceed the input tensor. I also zero-initialize the output so padded positions cannot contain uninitialized values.

Added CPU regression coverage for both boundaries and verified the full Triton GDN backend suite on H20 (11/11 passed).

[nvpohanh]

/tag-and-rerun-ci

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

/rerun-failed-ci

[daii-0818]

/rerun-failed-ci

[daii-0818]

@nvpohanh The GDN/CPU tests are now passing after the latest fix. The remaining failures appear platform-specific: NPU performance baseline, invalid B300 model path, AMD HIPBLAS/IPC timeouts, and missing CPU kernel ops on Arm/Xeon. Could the corresponding platform owners help confirm whether these are infrastructure/baseline issues?

[daii-0818]

/rerun-failed-ci

[nvpohanh]

@daii-0818 please fix the lint issue. thanks

[daii-0818]

/rerun-failed-ci

[daii-0818]

Hi @yuan-luo, your token-padding feedback has been addressed, and the relevant NVIDIA CI suites are passing. The remaining failures appear to be infrastructure-related.

Could you please review the latest changes and merge the PR if everything looks good? Thanks!

[nvpohanh]

All NV pipelines have passed.

[nvpohanh]

@yuan-luo could you review this? Thanks!

[nvpohanh]

/rerun-tests test/registered/e2e/gdn/test_qwen35_gdn_multi_item_scoring.py

[github-actions]

Results for /rerun-tests test/registered/e2e/gdn/test_qwen35_gdn_multi_item_scoring.py:

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/gdn/test_qwen35_gdn_multi_item_scoring.py

[yuan-luo]

LGTM

[daii-0818]

/rerun-failed-ci