Skip to content

[MUSA] Fix SwiGLU to use fused op - #6364

Merged
gcanlin merged 1 commit into
vllm-project:mainfrom
yeahdongcn:xd/silu_mul
Aug 20, 2026
Merged

gcanlin merged 1 commit into
vllm-project:mainfrom
yeahdongcn:xd/silu_mul

Conversation

@yeahdongcn

@yeahdongcn yeahdongcn commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

PLEASE FILL IN THE PR DESCRIPTION HERE.

Purpose

Allow SwiGLU on MUSA to use the fused op.

Test Plan

vLLM Version:

vLLM-Omni Commit:

Test Result

Fair A/B Comparison: Baseline → Final

Version TP8 Median Throughput Improvement
Baseline 105.351 s —
#6281 only 101.492 s +3.802%
#6283 + #6364 104.801 s +0.525%
Final combined 101.811 s +3.477%

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.
(anything written below this line will be removed by GitHub Actions)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@yeahdongcn

Copy link
Copy Markdown
Contributor Author

This is a follow-up to #6283
Please @gcanlin take a look. Thanks!

@vllm-omni-review-bot

Copy link
Copy Markdown

This PR appears to belong to: docs/design/module/diffusion/diffusion_model_integration.md.

Module owners: @fhfuih @Bounty-hunter @wtomin @RuixiangMa

@yeahdongcn, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

Comment thread vllm_omni/diffusion/layers/activation.py Outdated
@Bounty-hunter

Bounty-hunter commented Aug 19, 2026 •

Copy link
Copy Markdown
Collaborator

need to add forward_hip, It seem that self.op only init when current_omni_platform.is_cuda() or current_omni_platform.is_rocm() or current_omni_platform.is_musa() in __init__

@yeahdongcn

yeahdongcn commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor Author

need to add forward_hip, It seem that self.op only init when current_omni_platform.is_cuda() or current_omni_platform.is_rocm() or current_omni_platform.is_musa() in __init__

rocm uses forward_hip (which falls back to forward_cuda by default), so leaving it as is should work.

@Bounty-hunter Bounty-hunter left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hsliuustc0106 hsliuustc0106 added the Kernel optimization Codes related to optimize kernel execution to improve hardware utilization label Aug 20, 2026
Comment thread vllm_omni/diffusion/layers/activation.py
@gcanlin gcanlin added the ready label to trigger buildkite CI label Aug 20, 2026
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
@gcanlin gcanlin added ready label to trigger buildkite CI and removed ready label to trigger buildkite CI labels Aug 20, 2026
@gcanlin
gcanlin merged commit b58feb1 into vllm-project:main Aug 20, 2026
8 of 9 checks passed
fan2956 pushed a commit to fan2956/vllm-omni that referenced this pull request Aug 23, 2026
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
fan2956 pushed a commit to fan2956/vllm-omni that referenced this pull request Aug 23, 2026
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
AndyZhou952 pushed a commit to AndyZhou952/vllm-omni that referenced this pull request Aug 26, 2026
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
JoseCarlosGarcia95 pushed a commit to valendra-tech/vllm-omni that referenced this pull request Sep 5, 2026
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
khairulkabir1661 pushed a commit to khairulkabir1661/vllm-omni that referenced this pull request Sep 25, 2026
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Kernel optimization Codes related to optimize kernel execution to improve hardware utilization ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants