[Feature][Model] Support text-only Qwen3.5 checkpoints - #14885
Merged
Merged
Conversation
Backport the ModelSlim packed-module mappings and RoPE selection for text-only Qwen3.5 checkpoints from PR vllm-project#14238, together with focused unit tests. Signed-off-by: Erkong <Erkong@noreply.gitcode.com>
Contributor
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Contributor
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request backports essential support for text-only Qwen3.5 checkpoints into the vLLM Ascend integration. By refining the RoPE selection logic and extending ModelSlim configuration mappings, the changes prevent incorrect multimodal path execution for text-only models and enable proper weight resolution for quantized checkpoints. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Attention][Feature] Support Qwen3.5 text and MoE text modelsSuggested PR Summary:
### What this PR does / why we need it?
This PR adds support for Qwen3.5 text and Qwen3.5 MoE text models. It refactors the RoPE selection logic in `AscendQwen3NextAttention` by introducing a helper `_uses_multimodal_rope` to check if the model uses multimodal RoPE (mRoPE) based on the model type and the presence of `mrope_section` in `rotary_emb`. Additionally, it adds packed module mappings for `qwen3_5_text` and `qwen3_5_moe_text` in the ModelSlim quantization configuration.
Feedback:
A review comment suggests improving the robustness of `_uses_multimodal_rope` by adding defensive checks for `None` values or missing attributes on `attention`, `config`, and `rotary_emb` to prevent potential `AttributeError` or `TypeError` exceptions.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tested via new unit tests in `tests/ut/patch/worker/test_patch_qwen3_5_mtp.py` and `tests/ut/quantization/test_modelslim_config.py`.
Comment on lines
+59
to
+61
| def _uses_multimodal_rope(attention: Qwen3NextAttention) -> bool: | ||
| """Return whether a Qwen3.5 attention layer exposes multimodal RoPE.""" | ||
| return "qwen3_5" in attention.config.model_type and hasattr(attention.rotary_emb, "mrope_section") |
Contributor
There was a problem hiding this comment.
To ensure robustness and adhere to defensive programming principles, we should guard against cases where attention, attention.config, or attention.rotary_emb might be None or missing their expected attributes. This prevents potential AttributeError or TypeError exceptions during runtime or when testing with partial mock objects.
def _uses_multimodal_rope(attention: Qwen3NextAttention) -> bool:
"""Return whether a Qwen3.5 attention layer exposes multimodal RoPE."""
config = getattr(attention, "config", None)
if config is None:
return False
model_type = getattr(config, "model_type", "")
if "qwen3_5" not in model_type:
return False
rotary_emb = getattr(attention, "rotary_emb", None)
return rotary_emb is not None and hasattr(rotary_emb, "mrope_section")References
- Enforce defensive programming by ensuring appropriate null/None checks or other language-idiomatic guards exist before object property accesses.
Karryking3
changed the title
Contributor
Author
|
/rerun Rerun (failed jobs only):
|
LoganJane
approved these changes
Aug 25, 2026
kunpengW-code
approved these changes
Aug 25, 2026
pisceskkk
pushed a commit
to pisceskkk/vllm-ascend
that referenced
this pull request
Aug 26, 2026
…14885) ### What this PR does / why we need it? This PR backports the Qwen3.5 text-only RoPE handling and ModelSlim packed-module mappings from [vLLM PR #50734](vllm-project/vllm#50734). Text-only Qwen3.5-family checkpoints use the model types qwen3_5_text or qwen3_5_moe_text. Unlike multimodal Qwen3.5 checkpoints, they do not expose multimodal RoPE fields such as mrope_section. Before this change, the Ascend Qwen3.5 attention patch selected the multimodal RoPE path based only on the qwen3_5 model-type prefix. This could cause text-only checkpoints to access missing multimodal RoPE attributes. ModelSlim also lacked packed-module mappings for the text-only dense and MoE model types. The change: - selects the multimodal RoPE path only when mrope_section is available; - keeps text-only Qwen3.5 checkpoints on the standard RoPE path; - adds ModelSlim packed-module mappings for qwen3_5_text; - adds ModelSlim packed-module and expert mappings for qwen3_5_moe_text; - adds focused unit coverage for RoPE selection and ModelSlim mappings. This is a scoped backport. It does not include the model registration, hybrid-cache wrappers, MTP configuration conversion, or documentation changes from the source PR. Related PR: [vllm-project/vllm#50734](vllm-project/vllm#50734) ### Does this PR introduce _any_ user-facing change? Yes. When the corresponding text-only Qwen3.5 model integration is available, checkpoints using qwen3_5_text or qwen3_5_moe_text no longer incorrectly enter the multimodal RoPE path. ModelSlim W8A8 checkpoints using these model types can also resolve the existing packed QKV, gate/up projection, GDN projection, and MoE expert mappings. ### How was this patch tested? Added unit coverage for: - standard RoPE selection for qwen3_5_moe_text; - multimodal RoPE selection when mrope_section exists; - ModelSlim packed-module mappings for qwen3_5_text; - ModelSlim expert mappings for qwen3_5_moe_text. git diff --check passed. Python syntax compilation passed for all four changed files. Targeted pytest execution was not available in the local environment because pytest is not installed. Ruff was not available in the local environment, so no Ruff result is claimed. No real Ascend NPU or checkpoint inference validation was performed for this scoped backport. vLLM version: current branch dependency vLLM main: current branch dependency - vLLM version: v0.27.1 - vLLM main: vllm-project/vllm@ba07e4a Signed-off-by: Erkong <Erkong@noreply.gitcode.com> Co-authored-by: Erkong <Erkong@noreply.gitcode.com> Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
frankie-ys
pushed a commit
to Csrayz/vllm-ascend
that referenced
this pull request
Aug 26, 2026
…14885) ### What this PR does / why we need it? This PR backports the Qwen3.5 text-only RoPE handling and ModelSlim packed-module mappings from [vLLM PR #50734](vllm-project/vllm#50734). Text-only Qwen3.5-family checkpoints use the model types qwen3_5_text or qwen3_5_moe_text. Unlike multimodal Qwen3.5 checkpoints, they do not expose multimodal RoPE fields such as mrope_section. Before this change, the Ascend Qwen3.5 attention patch selected the multimodal RoPE path based only on the qwen3_5 model-type prefix. This could cause text-only checkpoints to access missing multimodal RoPE attributes. ModelSlim also lacked packed-module mappings for the text-only dense and MoE model types. The change: - selects the multimodal RoPE path only when mrope_section is available; - keeps text-only Qwen3.5 checkpoints on the standard RoPE path; - adds ModelSlim packed-module mappings for qwen3_5_text; - adds ModelSlim packed-module and expert mappings for qwen3_5_moe_text; - adds focused unit coverage for RoPE selection and ModelSlim mappings. This is a scoped backport. It does not include the model registration, hybrid-cache wrappers, MTP configuration conversion, or documentation changes from the source PR. Related PR: [vllm-project/vllm#50734](vllm-project/vllm#50734) ### Does this PR introduce _any_ user-facing change? Yes. When the corresponding text-only Qwen3.5 model integration is available, checkpoints using qwen3_5_text or qwen3_5_moe_text no longer incorrectly enter the multimodal RoPE path. ModelSlim W8A8 checkpoints using these model types can also resolve the existing packed QKV, gate/up projection, GDN projection, and MoE expert mappings. ### How was this patch tested? Added unit coverage for: - standard RoPE selection for qwen3_5_moe_text; - multimodal RoPE selection when mrope_section exists; - ModelSlim packed-module mappings for qwen3_5_text; - ModelSlim expert mappings for qwen3_5_moe_text. git diff --check passed. Python syntax compilation passed for all four changed files. Targeted pytest execution was not available in the local environment because pytest is not installed. Ruff was not available in the local environment, so no Ruff result is claimed. No real Ascend NPU or checkpoint inference validation was performed for this scoped backport. vLLM version: current branch dependency vLLM main: current branch dependency - vLLM version: v0.27.1 - vLLM main: vllm-project/vllm@ba07e4a Signed-off-by: Erkong <Erkong@noreply.gitcode.com> Co-authored-by: Erkong <Erkong@noreply.gitcode.com>
Lethobenthos20
pushed a commit
to Lethobenthos20/vllm-ascend
that referenced
this pull request
Sep 4, 2026
…14885) ### What this PR does / why we need it? This PR backports the Qwen3.5 text-only RoPE handling and ModelSlim packed-module mappings from [vLLM PR #50734](vllm-project/vllm#50734). Text-only Qwen3.5-family checkpoints use the model types qwen3_5_text or qwen3_5_moe_text. Unlike multimodal Qwen3.5 checkpoints, they do not expose multimodal RoPE fields such as mrope_section. Before this change, the Ascend Qwen3.5 attention patch selected the multimodal RoPE path based only on the qwen3_5 model-type prefix. This could cause text-only checkpoints to access missing multimodal RoPE attributes. ModelSlim also lacked packed-module mappings for the text-only dense and MoE model types. The change: - selects the multimodal RoPE path only when mrope_section is available; - keeps text-only Qwen3.5 checkpoints on the standard RoPE path; - adds ModelSlim packed-module mappings for qwen3_5_text; - adds ModelSlim packed-module and expert mappings for qwen3_5_moe_text; - adds focused unit coverage for RoPE selection and ModelSlim mappings. This is a scoped backport. It does not include the model registration, hybrid-cache wrappers, MTP configuration conversion, or documentation changes from the source PR. Related PR: [vllm-project/vllm#50734](vllm-project/vllm#50734) ### Does this PR introduce _any_ user-facing change? Yes. When the corresponding text-only Qwen3.5 model integration is available, checkpoints using qwen3_5_text or qwen3_5_moe_text no longer incorrectly enter the multimodal RoPE path. ModelSlim W8A8 checkpoints using these model types can also resolve the existing packed QKV, gate/up projection, GDN projection, and MoE expert mappings. ### How was this patch tested? Added unit coverage for: - standard RoPE selection for qwen3_5_moe_text; - multimodal RoPE selection when mrope_section exists; - ModelSlim packed-module mappings for qwen3_5_text; - ModelSlim expert mappings for qwen3_5_moe_text. git diff --check passed. Python syntax compilation passed for all four changed files. Targeted pytest execution was not available in the local environment because pytest is not installed. Ruff was not available in the local environment, so no Ruff result is claimed. No real Ascend NPU or checkpoint inference validation was performed for this scoped backport. vLLM version: current branch dependency vLLM main: current branch dependency - vLLM version: v0.27.1 - vLLM main: vllm-project/vllm@ba07e4a Signed-off-by: Erkong <Erkong@noreply.gitcode.com> Co-authored-by: Erkong <Erkong@noreply.gitcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
This PR backports the Qwen3.5 text-only RoPE handling and ModelSlim packed-module mappings from vLLM PR #50734.
Text-only Qwen3.5-family checkpoints use the model types qwen3_5_text or qwen3_5_moe_text. Unlike multimodal Qwen3.5 checkpoints, they do not expose multimodal RoPE fields such as mrope_section.
Before this change, the Ascend Qwen3.5 attention patch selected the multimodal RoPE path based only on the qwen3_5 model-type prefix. This could cause text-only checkpoints to access missing multimodal RoPE attributes. ModelSlim also lacked packed-module mappings for the text-only dense and MoE model types.
The change:
This is a scoped backport. It does not include the model registration, hybrid-cache wrappers, MTP configuration conversion, or documentation changes from the source PR.
Related PR: vllm-project/vllm#50734
Does this PR introduce any user-facing change?
Yes.
When the corresponding text-only Qwen3.5 model integration is available, checkpoints using qwen3_5_text or qwen3_5_moe_text no longer incorrectly enter the multimodal RoPE path.
ModelSlim W8A8 checkpoints using these model types can also resolve the existing packed QKV, gate/up projection, GDN projection, and MoE expert mappings.
How was this patch tested?
Added unit coverage for:
git diff --check passed.
Python syntax compilation passed for all four changed files.
Targeted pytest execution was not available in the local environment because pytest is not installed.
Ruff was not available in the local environment, so no Ruff result is claimed.
No real Ascend NPU or checkpoint inference validation was performed for this scoped backport.
vLLM version: current branch dependency
vLLM main: current branch dependency