Skip to content

Disable dual stream execution of input projection for Qwen3#38152

Merged
zou3519 merged 1 commit intovllm-project:mainfrom
xyang16:disable_dual_stream
Mar 26, 2026
Merged

Disable dual stream execution of input projection for Qwen3#38152
zou3519 merged 1 commit intovllm-project:mainfrom
xyang16:disable_dual_stream

Conversation

@xyang16
Copy link
Contributor

@xyang16 xyang16 commented Mar 25, 2026

Purpose

Currently dual stream execution requires custom ops that passes the layer_name as a string. This will regress cold compile times by ~4x. So this PR temporarily revert dual stream optimization in Qwen3 and Qwen3.5 models.

TODO: Re-enable dual stream after #38123 and upgrade to Pytorch 2.11.

Test Plan

Tested with Qwen3 and Qwen3.5 models.

vllm serve Qwen/Qwen3-Next-80B-A3B-Instruct \
    --tensor-parallel-size 1 \
    --max-num-seqs 16 \
    --no-enable-prefix-caching
vllm serve Qwen/Qwen3.5-35B-A3B \
    --tensor-parallel-size 1 \
    --max-num-seqs 16 \
    --no-enable-prefix-caching

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@xyang16 xyang16 requested a review from sighingnow as a code owner March 25, 2026 21:55
Copy link

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify bot added the qwen Related to Qwen models label Mar 25, 2026
Signed-off-by: Xin Yang <xyangx@amazon.com>
@xyang16 xyang16 force-pushed the disable_dual_stream branch from b73fec8 to 5f1ce9c Compare March 25, 2026 21:59
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the input projection logic within the Qwen3_5 and Qwen3_Next models. It removes the custom gdn_in_proj operation, its fake implementation, and the associated _forward_in_proj method that utilized maybe_execute_in_parallel. The input projection now directly calls self.in_proj_qkvz and self.in_proj_ba instead of relying on the custom operation. There are no review comments to address.

@xyang16
Copy link
Contributor Author

xyang16 commented Mar 25, 2026

cc @zou3519 @benchislett

Copy link
Collaborator

@zou3519 zou3519 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, sorry for the trouble

@zou3519 zou3519 added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 25, 2026
@zou3519 zou3519 enabled auto-merge (squash) March 25, 2026 23:52
@zou3519 zou3519 merged commit 9704a5c into vllm-project:main Mar 26, 2026
58 checks passed
@xyang16 xyang16 deleted the disable_dual_stream branch March 26, 2026 03:37
RhizoNymph pushed a commit to RhizoNymph/vllm that referenced this pull request Mar 26, 2026
HenryTangDev pushed a commit to HenryTangMain/vllm that referenced this pull request Mar 27, 2026
malaiwah pushed a commit to malaiwah/vllm that referenced this pull request Mar 27, 2026
…ject#38152)

Signed-off-by: Xin Yang <xyangx@amazon.com>
Signed-off-by: Michel Belleau <michel.belleau@malaiwah.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants