Skip to content

Migrate Qwen3-Next and Qwen3.5 models to HybridModel - #4836

Draft
Phlip79 wants to merge 2 commits into
philip/qwen3-hybridfrom
philip/qwen3-linear-attention-hybrid
Draft

Migrate Qwen3-Next and Qwen3.5 models to HybridModel#4836
Phlip79 wants to merge 2 commits into
philip/qwen3-hybridfrom
philip/qwen3-linear-attention-hybrid

Conversation

@Phlip79

@Phlip79 Phlip79 commented Jul 12, 2026

Copy link
Copy Markdown
Member

Summary

  • migrate Qwen3-Next to HybridModel
  • migrate the reusable dense and MoE Qwen3.5 text-decoder architectures to HybridModel
  • share Gated DeltaNet/full-attention schedule translation and logical-to-physical layer mappings
  • update focused bridge tests for both model families

Models modified

Qwen3-Next checkpoint using Qwen3NextForCausalLM:

  • Qwen/Qwen3-Next-80B-A3B-Instruct

Qwen3.5 text-decoder architectures:

  • qwen3_5_text using Qwen3_5ForCausalLM
  • qwen3_5_moe_text using Qwen3_5MoeForCausalLM

These are reusable language-decoder architectures, not separate public Qwen3.5 checkpoint repositories. They appear as the nested text_config inside the official Qwen/Qwen3.5-* multimodal checkpoints. This PR migrates those decoder bridges and provides language-layer mapping helpers reused by the full multimodal bridges.

The official Qwen/Qwen3.5-* checkpoint repositories, their top-level conditional-generation wrappers, vision models, and end-to-end conversion paths are modified in #4837.

Stack

This is PR 2 of 4. Merge the stack in order:

  1. Migrate Qwen3 dense and MoE models to HybridModel #4747 - Qwen3 dense and MoE
  2. Migrate Qwen3-Next and Qwen3.5 models to HybridModel #4836 - Qwen3-Next and Qwen3.5 text
  3. Migrate Qwen VL models to HybridModel #4837 - Qwen3-VL and Qwen3.5-VL
  4. Migrate Qwen3-ASR and Qwen3-Omni to HybridModel #4838 - Qwen3-ASR and Qwen3-Omni

Validation

Phlip79 added 2 commits July 12, 2026 23:20
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@Phlip79 Phlip79 changed the title feat(qwen): migrate Qwen3-Next and Qwen3.5 models to HybridModel Migrate Qwen3-Next and Qwen3.5 models to HybridModel Jul 13, 2026
@Phlip79 Phlip79 added the needs-more-tests Requires additional L0 and L1 test coverage before merge label Jul 13, 2026
@Phlip79
Phlip79 force-pushed the philip/qwen3-hybrid branch from 42f6005 to eddbe88 Compare July 23, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-more-tests Requires additional L0 and L1 test coverage before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant