[Diffusion]: Diffusion Ring Attention support #273
Merged
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
ZJY0516
reviewed
Dec 11, 2025
Member
ZJY0516
left a comment
ZJY0516
left a comment
There was a problem hiding this comment.
For better project structure, the ring_attn folder should be placed under the attention directory.
58 tasks
Collaborator
|
cc @wtomin |
wtomin
reviewed
Dec 24, 2025
wtomin
reviewed
Dec 24, 2025
wtomin
reviewed
Dec 24, 2025
wtomin
reviewed
Dec 24, 2025
SamitHuang
reviewed
Dec 24, 2025
wtomin
reviewed
Dec 24, 2025
ZJY0516
reviewed
Dec 24, 2025
Member
ZJY0516
left a comment
ZJY0516
left a comment
There was a problem hiding this comment.
please remove some debug lines
- Implement ring-based P2P communication pattern - Add ring_flash_attn backend - Update documentation - Add unit tests Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
ZJY0516
changed the title
ZJY0516
reviewed
Dec 30, 2025
ZJY0516
reviewed
Dec 30, 2025
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
…nication issue in ring Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
ZJY0516
reviewed
Dec 31, 2025
ZJY0516
approved these changes
Dec 31, 2025
Signed-off-by: mxuax <mxuax@connect.ust.hk>
1 task
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
5 tasks
princepride
pushed a commit
to princepride/vllm-omni
that referenced
this pull request
Jan 10, 2026
Signed-off-by: mxuax <mxuax@connect.ust.hk> Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
ZJY0516
added a commit
to LawJarp-A/vllm-omni
that referenced
this pull request
Jan 10, 2026
Signed-off-by: mxuax <mxuax@connect.ust.hk> Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
54 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR follows Diffusion Ulysses-Sequence-Parallelism support PR#189, allowing users to set Ring Attention for diffusion model, e.g., qwen-image.
Purpose
Create Ring Attention Module (vllm_omni/diffusion/ring_attn)
Refactor Distributed State (vllm_omni/diffusion/distributed/parallel_state.py)
Implement Ring Attention Logic (vllm_omni/diffusion/attention/layer.py)
Modify the generic Attention layer to support Ring Attention and Hybrid (Ulysses + Ring) modes.
Verification and Testing
Create a new test file to verify the correctness of the Ring Attention implementation on the Qwen model components.
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)