[a17t]

PR #3 — fix(models): detect unquantized MTP weights from checkpoint metadata

Summary

Some GPTQ/AWQ exporters ship MTP tensors unquantized but do not write
the expected -:...mtp... entries into quantization_config.dynamic. The
existing dynamic-pattern check on rdna_extras
(vllm/model_executor/models/qwen3_5_mtp.py:153) only catches the
explicit-exclusion case. On those checkpoints the MTP drafter's experts
get constructed quantized (w2_qweight...), but the loader maps the
fused BF16 names to w2_weight → AttributeError at weight load,
breaking MTP on the entire 35B-MoE family.

This PR adds a complementary suffix-based detection: if the
checkpoint's mtp.* param names carry no quantized-style suffixes
(.qweight, .qzeros, .g_idx, .weight_packed, .scales, .zeros,
.weight_scale, .scale), the MTP drafter is built with quantization
disabled so the fused/per-expert name mappings load the BF16 tensors
as-is.

Diff scope

Single file: vllm/model_executor/models/qwen3_5_mtp.py. +48 / −2 lines.

+ _QUANT_PARAM_SUFFIXES = (
+     ".qweight", ".qzeros", ".g_idx", ".weight_packed",
+     ".scales",  ".zeros",  ".weight_scale", ".scale",
+ )
+
+ def _mtp_weights_unquantized(model_config) -> bool:
+     """Whether the checkpoint stores its mtp.* tensors unquantized."""
+     try:
+         from vllm.transformers_utils.config import (
+             get_safetensors_params_metadata,
+         )
+         params = get_safetensors_params_metadata(model_config.model)
+     except Exception:
+         return False
+     mtp_names = [name for name in params if name.startswith("mtp.")]
+     if not mtp_names:
+         return False
+     return not any(
+         name.endswith(sfx) for name in mtp_names
+         for sfx in _QUANT_PARAM_SUFFIXES
+     )

  # GPTQ: quantized checkpoints may exclude MTP from quantization via
  # quantization_config.dynamic with "-:pattern" entries. When detected,
  # disable quantization for MTP layers so they use unquantized params.
+ # Some exporters skip MTP WITHOUT writing such an entry (e.g.
+ # btbtyler09 Qwen3.6-35B GPTQ stores every mtp.* tensor unquantized
+ # bf16, experts fused as experts.gate_up_proj/down_proj); detect that
+ # from the checkpoint's safetensors param names instead.
  original_quant = vllm_config.quant_config
  if quant_config and quant_config.get_name() not in ("modelopt_fp4",):
      hf_qc = getattr(model_config.hf_config, "quantization_config", None)
+     excludes_mtp = False
      if isinstance(hf_qc, dict):
          dynamic = hf_qc.get("dynamic", {})
-         if any(k.startswith("-:") and "mtp" in k for k in dynamic):
-             vllm_config.quant_config = None
+         excludes_mtp = any(
+             k.startswith("-:") and "mtp" in k for k in dynamic
+         )
+     if not excludes_mtp:
+         excludes_mtp = _mtp_weights_unquantized(model_config)
+     if excludes_mtp:
+         vllm_config.quant_config = None

Why this is net-new value on this fork

Their tree already has the dynamic-pattern detection at the same
location — _mtp_weights_unquantized is a complementary trigger for
checkpoints that omit the explicit -:mtp.* exclusion. This catches:

• btbtyler09 Qwen3.6-35B-A3B-GPTQ-4bit (no dynamic entries;
  MTP stored as plain BF16; experts fused as
  experts.gate_up_proj / experts.down_proj).
• Any future GPTQ/AWQ exporter that follows the same "skip dynamic,
  ship unquantized MTP" pattern.
• AWQ-style checkpoints that use modules_to_not_convert: ["mtp"] are
  unaffected — that path is handled separately by quant_utils.py.

Verification

The detection function was unit-validated against local checkpoints
(via vllm.transformers_utils.config.get_safetensors_params_metadata):

Checkpoint	Total params	mtp.* params	quant-suffix on mtp.*
QuantTrio/Qwen3.6-27B-AWQ	1,892	15	0 / 15
QuantTrio/Qwen3.6-35B-A3B-AWQ	92,356	785	0 / 785

Both correctly detected as mtp_weights_unquantized=True.

End-to-end MTP load + run (validated on vllm-upstream, gfx1030):

Checkpoint	Before	After
btbtyler09/Qwen3.6-35B-A3B-GPTQ-4bit MTP=2	AttributeError at weight load	runs, 64.2 t/s
QuantTrio/Qwen3.6-35B-A3B-AWQ MTP=2	(per-expert BF16 MTP names)	runs

Tests

No new tests required — the existing tests/models/test_qwen3_5_mtp.py
covers the dynamic-pattern path; our suffix-based check is a fallback
inside the same conditional. Manual validation above shows the
detection function reads both local-snapshot and Hub-fetched metadata
correctly.

License

Apache-2.0, matches the file's SPDX header.

AI assistance

This PR was prepared with AI assistance (opencode). The detection
logic is identical to vllm-upstream commit 8fb6a44c47 (2026-08-27);
only the surrounding context comments and function placement were
adapted for the rdna_extras v0.29.0 base.

Commit

fix(models): detect unquantized MTP weights from checkpoint metadata

Single-file change in vllm/model_executor/models/qwen3_5_mtp.py. +48/-2.
Ports vllm-upstream 8fb6a44c47 onto rdna_extras.

Co-authored-by: opencode <noreply@opencode.ai>

[BlivionIaG]

Landed via the local integration merge: rdna_extras is now at 2a5e89368 (merge of this PR's commit e86265f30 on top of 68a635ed8), so the fork carries the unquantized-MTP detection verbatim.

Integration note — this fix is what made the MTP path progress on the V620 stack: our AWQ checkpoint (wtdcode/Qwen3.8-Flash-Next-AWQ-W4A16) has 31 mtp.* tensors with no quant suffixes and dynamic: None in its quantization config, so the drafter was previously built quantized while the checkpoint ships BF16 tensors. With this PR the MTP2 run no longer hangs during profile_run; it now reaches the draft sampling path and fails later, and precisely, in the draft vocab all-gather (_ALLGATHER_BASE, PG ID 2, 60 s watchdog timeout) — a separate follow-up we are chasing now.

Thanks for the port — clean, safe fallback, and it keeps the -:mtp dynamic-entry path as the first trigger.