Skip to content

[Bugfix] Remove incorrect assertion in causal_conv1d_update for Qwen3.5 GDN layersfix: remove incorrect assertion in causal_conv1d_update for GDN index…#36324

Open
Rks2302 wants to merge 4 commits intovllm-project:mainfrom
Rks2302:fix/gdn-causal-conv1d-assertion
Open

Conversation

@Rks2302
Copy link

@Rks2302 Rks2302 commented Mar 7, 2026

Summary

Removes incorrect assertion in causal_conv1d_update that causes AssertionError
when running Qwen3.5 models with GDN (Gated Delta Networks) layers.

Root Cause

Line 1161 in causal_conv1d.py:

assert num_cache_lines >= batch

This assertion is incorrect when conv_state_indices is provided. In that case:

  • conv_state is a shared cache pool, not a per-batch tensor
  • num_cache_lines is the state length (typically 4-6), NOT the batch dimension
  • The actual batch validation is already correctly handled in the if/else block above

The correct validations are already present:

  • Without indices: assert conv_state.size(0) >= batch
  • With indices: assert batch == conv_state_indices.shape[0]

The removed assertion is therefore redundant and incorrect.

Error Reproduced On

  • GPU: NVIDIA RTX 5090 (Blackwell, sm_120)
  • vLLM: 0.17.0
  • Model: Qwen3.5-35B-A3B-AWQ, Qwen3.5-27B-AWQ
  • CUDA: 12.8

Related Issues

@Rks2302 Rks2302 requested a review from tdoublep as a code owner March 7, 2026 11:21
@mergify mergify bot added qwen Related to Qwen models bug Something isn't working labels Mar 7, 2026
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes an incorrect assertion in causal_conv1d_update that causes an AssertionError when running Qwen3.5 models with Gated Delta Networks (GDN) layers. The assertion assert num_cache_lines >= batch is invalid when conv_state_indices is provided, as conv_state is a shared cache pool in that scenario, and num_cache_lines represents the cache size, not a per-batch dimension. The removal of this assertion is correct as the necessary batch validation is already handled by other checks.

vllmellm and others added 2 commits March 7, 2026 18:40
…on MI300x (vllm-project#36247)

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: Rks2302 <rahulksharma2302@gmail.com>
…ed conv_state

Signed-off-by: Rks2302 <rahulksharma2302@gmail.com>
@Rks2302 Rks2302 force-pushed the fix/gdn-causal-conv1d-assertion branch from 70f1470 to 444783f Compare March 7, 2026 13:11
@mergify mergify bot added the deepseek Related to DeepSeek models label Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working deepseek Related to DeepSeek models qwen Related to Qwen models

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants