Skip to content

Migrate Qwen VL models to HybridModel - #4837

Draft
Phlip79 wants to merge 1 commit into
philip/qwen3-linear-attention-hybridfrom
philip/qwen3-vl-hybrid
Draft

Migrate Qwen VL models to HybridModel#4837
Phlip79 wants to merge 1 commit into
philip/qwen3-linear-attention-hybridfrom
philip/qwen3-vl-hybrid

Conversation

@Phlip79

@Phlip79 Phlip79 commented Jul 12, 2026

Copy link
Copy Markdown
Member

Summary

  • migrate dense and MoE Qwen3-VL models to HybridModel
  • migrate the full official dense and MoE Qwen3.5 multimodal checkpoints on the shared Hybrid language runtime from Migrate Qwen3-Next and Qwen3.5 models to HybridModel #4836
  • update multimodal model construction, RoPE, conversion mappings, and recipe integration
  • add focused provider, bridge, model, and recipe coverage

Models modified

Qwen3-VL checkpoints using Qwen3VLForConditionalGeneration or Qwen3VLMoeForConditionalGeneration:

  • Qwen/Qwen3-VL-8B-Instruct
  • Qwen/Qwen3-VL-30B-A3B-Instruct
  • Qwen/Qwen3-VL-235B-A22B-Instruct

Official dense Qwen3.5 multimodal checkpoints using Qwen3_5ForConditionalGeneration:

  • Qwen/Qwen3.5-0.8B
  • Qwen/Qwen3.5-2B
  • Qwen/Qwen3.5-4B
  • Qwen/Qwen3.5-9B
  • Qwen/Qwen3.5-27B

Official MoE Qwen3.5 multimodal checkpoints using Qwen3_5MoeForConditionalGeneration:

  • Qwen/Qwen3.5-35B-A3B
  • Qwen/Qwen3.5-122B-A10B
  • Qwen/Qwen3.5-397B-A17B

Each official Qwen3.5 repository above has a top-level multimodal configuration containing a nested qwen3_5_text or qwen3_5_moe_text decoder. PR #4836 migrates those reusable decoder architectures; this PR owns the public checkpoint IDs and migrates their top-level conditional-generation wrappers, vision models, prefixed conversion mappings, and recipes.

Stack

This is PR 3 of 4. Merge the stack in order:

  1. Migrate Qwen3 dense and MoE models to HybridModel #4747 - Qwen3 dense and MoE
  2. Migrate Qwen3-Next and Qwen3.5 models to HybridModel #4836 - Qwen3-Next and Qwen3.5 text
  3. Migrate Qwen VL models to HybridModel #4837 - Qwen3-VL and Qwen3.5-VL
  4. Migrate Qwen3-ASR and Qwen3-Omni to HybridModel #4838 - Qwen3-ASR and Qwen3-Omni

Validation

Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@Phlip79 Phlip79 changed the title feat(qwen-vl): migrate Qwen3 and Qwen3.5 models to HybridModel Migrate Qwen3 and Qwen3.5 models to HybridModel Jul 13, 2026
@Phlip79 Phlip79 changed the title Migrate Qwen3 and Qwen3.5 models to HybridModel Migrate Qwen3 and 3.5 VL models to HybridModel Jul 13, 2026
@Phlip79 Phlip79 changed the title Migrate Qwen3 and 3.5 VL models to HybridModel Migrate Qwen VL models to HybridModel Jul 13, 2026
@Phlip79 Phlip79 added needs-more-tests Requires additional L0 and L1 test coverage before merge full-test-suite labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-test-suite needs-more-tests Requires additional L0 and L1 test coverage before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant