Fix Quark/AWQ weight loading for Qwen3-VL-4B text model - #2143
Merged
kunal-vaishnavi merged 3 commits intoMay 18, 2026
Merged
Conversation
Contributor
Author
|
@microsoft-github-policy-service agree company="AMD" |
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes Qwen3-VL-4B text model weight loading when building ONNX from quantized Hugging Face checkpoints (Quark/AWQ/GPTQ) or GGUF inputs by routing those cases through the shared quantized/GGUF loader and expanding the quantized tensor name-normalization logic to handle VLM checkpoint conventions.
Changes:
- Route
Qwen3VLTextModel.load_weightsto the base-class loader for quantized checkpoints and.ggufinputs (keeping the existing HF FP loading path unchanged). - Extend
QuantizedModelweight-key handling to (a) skip vision-tower tensors in VLM checkpoints, (b) normalizemodel.language_model.*tomodel.*, and (c) normalize Quarkweight_quantizer.{scale,zero_point}naming to the existingweight_{scale,zero_point}convention.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/python/py/models/quantized_model.py |
Adds VLM/Quark-specific tensor-key filtering and normalization so existing LLM parsing logic can consume Qwen3-VL language weights. |
src/python/py/models/builders/qwen.py |
Ensures Qwen3-VL text builder uses the shared quantized/GGUF loading path for non-FP checkpoints. |
…ression tests Extract the inline VLM/Quark key-normalisation logic from QuantizedModel.__init__ into a module-level normalize_vlm_weight_name() function so it can be unit-tested independently without writing safetensors files or constructing a full model. Add six regression tests covering: - Vision-tower key filtering (model.visual.*, model.vision.*, visual.*) - Pass-through for non-vision/non-VLM keys - model.language_model.* -> model.* prefix rewriting - Quark weight_quantizer.scale -> weight_scale renaming - Quark weight_quantizer.zero_point -> weight_zero_point renaming - Combined VLM prefix + Quark renaming in a single key Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
kunal-vaishnavi
previously approved these changes
May 15, 2026
kunal-vaishnavi
approved these changes
May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes Quark/AWQ weight loading for the Qwen3-VL-4B text model when building the ONNX model from a quantized Hugging Face checkpoint.
Previously:
Qwen3VLTextModel.load_weightsalways went throughQwen3VLForConditionalGeneration.from_pretrained, which doesn't understand quantized checkpoints (Quark / AWQ / GPTQ) or.gguffiles.QuantizedModelloader didn't handle VLM checkpoint conventions (vision-tower weights,model.language_model.*prefix) or Quark'sweight_quantizer.{scale,zero_point}naming.Changes
src/python/py/models/builders/qwen.pyQwen3VLTextModel.load_weights, route quantized models (self.quant_type is not None) and.ggufinputs to the base class implementation, which loads weights viaQuantizedModel. Plain HF FP checkpoints continue to useQwen3VLForConditionalGeneration.from_pretrainedas before.src/python/py/models/quantized_model.pymodel.visual.*,model.vision.*,visual.*) so the LLM weight loader only consumes language-model weights.model.language_model.*tomodel.*so the existing LLM regex/tensor-parsing logic keeps working unchanged.*.weight_quantizer.scale→*.weight_scale*.weight_quantizer.zero_point→*.weight_zero_point