fix(gemma2): Add quant_config to embedding layer for GGUF support

[kitaekatt]

Summary

• Adds quant_config parameter to VocabParallelEmbedding in Gemma2 model
• Enables proper GGUF quantized embedding handling

Problem

Gemma2 GGUF models produce garbage output despite loading successfully:

Input: "Say hello in 5 words"
Output: " GHFW側から ThinkmariKeywords!");\rJahre Iliad幺个人..."

Root Cause

Without quant_config, VocabParallelEmbedding uses UnquantizedEmbeddingMethod which calls F.embedding() directly on quantized bytes, interpreting them as float values.

This is the exact same bug that was fixed for DeepSeek in commit aa375dca9 ("#12836 - Missing quant_config in deepseek embedding layer").

The Fix

 self.embed_tokens = VocabParallelEmbedding(
     config.vocab_size,
     config.hidden_size,
+    quant_config=quant_config,
+    prefix=f"{prefix}.embed_tokens",
 )

Comparison

Model	quant_config Passed?	GGUF Works?
Gemma2	❌ NO (before fix)	❌ Garbage
Gemma3	✅ YES	✅ Works
Llama	✅ YES	✅ Works
DeepSeek	✅ YES (after #12836)	✅ Works

Affected Models

All Gemma2 GGUF variants:

• gemma-2-2b-it-GGUF
• gemma-2-9b-GGUF
• gemma-2-27b-GGUF

Test Plan

• Gemma2 GGUF model loads without errors
• Inference produces coherent output (not garbage)
• Compare output quality with Gemma3 GGUF or safetensors Gemma2

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request addresses a critical bug where Gemma2 GGUF models would produce garbage output. The root cause was correctly identified: the VocabParallelEmbedding layer was missing the quant_config parameter, causing quantized weights to be misinterpreted. The provided fix, which passes both quant_config and the layer prefix to the embedding layer, is correct and aligns with the implementation in other models.

While reviewing, I noticed that vllm/model_executor/models/gemma.py seems to have the same vulnerability. Applying a similar fix there would likely resolve the issue for Gemma1 GGUF models as well, ensuring consistent behavior across the Gemma model family. This is a great fix, well done!

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

[kitaekatt]

Validation Results

vLLM	transformers	Cherry-picked PRs	HumanEval	IFEval
HEAD	5.x	#30410, #30411, #30412, #30413, #30424, #30434, #30699, #30702, #31464, #33846	gem2-2b-gguf (42.1%), gemma3-1b (26.8%)	gem2-2b-gguf (65.6%)
HEAD	4.x	#30410, #30411, #30412, #30413, #30424, #30434, #30699, #30702, #31464, #33846	q3-moe-gguf (83.5%)	q3-moe-gguf (85.4%)

Tested on RTX 5090 (Blackwell, SM 120) with all listed PRs cherry-picked together; models listed under each benchmark passed that benchmark in the given environment, while the same models crash or fail without these PRs applied.

Converting from draft to open. Without quant_config on the embedding layer, GGUFEmbeddingMethod is not selected and quantized bytes are read as raw floats, producing gibberish. Validated on RTX 5090 (Blackwell, SM 120).

[kitaekatt]

Closing in favor of consolidated PR #37220, as requested by @Isotr0py in #30434. All fixes from this PR are included in the consolidated version.