[Qwen3.5] pass quant_config to VocabParallelEmbedding so the embedding table can be quantized

[malaiwah]

VocabParallelEmbedding already supports quantization: it calls
quant_config.get_quant_method(self, prefix) and refuses a method that does not implement
embedding(). Both Qwen3.5 model files construct it without passing the config:

self.embed_tokens = VocabParallelEmbedding(self.vocab_size, config.hidden_size)

so the hook can never fire and the table stays BF16 no matter what the backend supports.

Why it matters

On Qwen3.8-27B the input embedding table is 248,320 x 5,120 = 2.543 GB resident - second
only to the MLP stack, larger than the whole attention stack after quantization, and the only
large tensor in the model that is never multiplied (it is a gather).

With an int8 overlay behind it (EXL3 side, #318) the effect on a 32 GB card is:

	BF16 embeddings	int8 embeddings
resident weights	19.31 GiB	18.13 GiB
max --max-model-len that starts	229,376	262,144 (native)
KV allocated	240,080 tokens	279,007 tokens
mean KLD, 136 held-out contexts	0.009673	0.009738 (+0.000065, CI [+4.6e-06, +1.3e-04])
multimodal, 30 synthetic cases	24/30	24/30, identical

Verified by use rather than allocation: exact needle retrieval at depths 0.1/0.5/0.9 from
227,334-token prompts on a 5090-sized budget with vision enabled.

Why the MTP file still needs this change

Qwen3_5MultiTokenPredictor materializes an embedding table while its weights load, so it must
pass the quant config for the same hook to be reachable. The current vLLM proposer then aliases
that table to the target model embedding (llm_base_proposer.py logs "Sharing target model
embedding weights with the draft model"). It is therefore not a second resident table, and
quantizing it separately cannot increase steady-state KV capacity or change draft acceptance.
This patch lowers transient load memory and keeps standalone MTP construction consistent; the
native-context MTP gap is elsewhere.

Compatibility

Backends that do not implement embedding() are unaffected: get_quant_method returns None
and the layer falls back to UnquantizedEmbeddingMethod, which is today's behaviour.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• dev/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 178da8f7-4725-43bb-9512-618a7d946f56

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[malaiwah]

Post-publication correction: I re-read the MTP load receipts and found that my original body overstated what the model-file change buys. Both comparison logs say Sharing target model embedding weights with the draft model, and steady-state weight/KV accounting is unchanged when the temporary draft table is narrowed. The body now says explicitly that this table is materialized during load and then aliased: the MTP-side change reduces transient load memory / supports standalone construction; it does not free a second resident table or explain the native-context MTP gap. The main-model int8 measurements and native-context result are unaffected.