Fix Qwen3.5 & Qwen3-Next linear attention cu_seqlens missing

[huang3eng]

Summary

This PR fixes a varlen/packed-sequence bug in the linear attention implementations for both Qwen3.5 and Qwen3-Next.

Previously, cu_seqlens was forwarded to ShortConvolution, but not to chunk_gated_delta_rule. As a result, the convolution respected packed sequence boundaries, while the gated delta rule kernel treated the packed token stream as a single continuous sequence.

This could cause cross-sequence state leakage when multiple samples were packed together in the same micro-batch.

Root cause

In slime's thd packed-sequence path, a micro-batch is flattened into a single token stream and then reshaped to physical batch size 1, with logical sequence boundaries represented by cu_seqlens.

By the time the Qwen linear attention modules run, the inputs are effectively:

• hidden_states: [1, T, hidden]
• cu_seqlens: the boundaries between logical sequences inside the packed stream

This matches the varlen contract expected by upstream flash-linear-attention: when cu_seqlens is provided, the physical batch size should be 1.

However, the previous implementation only passed cu_seqlens to ShortConvolution and omitted it from chunk_gated_delta_rule, so the recurrent state inside the chunk kernel was not reset at packed-sequence boundaries.

Changes

• Forward cu_seqlens to chunk_gated_delta_rule in:
  • slime_plugins/models/qwen3_5.py
  • slime_plugins/models/qwen3_next.py
• Add a regression test to ensure:
  • the packed thd path reaches the chunk kernel with batch_size == 1
  • cu_seqlens is actually forwarded to the chunk kernel for both implementations

Reproduction and verification

The issue can be observed with a simple packed-input sanity check:

• Pack two logical sequences as seq1 || seq2
• Keep seq2 fixed
• Change only seq1

Before this fix

• The output on seq2 changed when only seq1 changed
• The packed suffix corresponding to seq2 also did not match the result of running seq2 alone

This indicates that the chunk kernel was carrying recurrent state across packed-sequence boundaries instead of resetting at each boundary.

After this fix

• Changing seq1 no longer affects the output on seq2
• The packed suffix corresponding to seq2 matches running seq2 alone up to numerical tolerance

This confirms that packed-sequence boundaries are now handled consistently by both ShortConvolution and chunk_gated_delta_rule.

Why this is correct

This change aligns the gated delta rule path with:

• the existing varlen behavior already used by ShortConvolution
• the upstream flash-linear-attention API for packed variable-length inputs

Without this fix, packed samples after the first one could be influenced by earlier samples in the same packed stream.

Validation

I validated this change with:

• a targeted smoke/regression test that monkeypatches the chunk kernel and confirms:
  • q.shape[0] == 1
  • cu_seqlens is forwarded correctly
• a minimal regression test added in:
  • tests/test_qwen3_linear_attention_cu_seqlens.py
• a syntax check via:
  • python -m compileall

Impact

This only affects the packed/varlen linear attention path for Qwen3.5 and Qwen3-Next.

It should not change behavior for non-packed cases, but it fixes incorrect cross-sequence interaction for packed training/inference.

[huang3eng]

@Zhuohao-Li Please help review it. 🫶🫶