fix(gguf): Skip lm_head mapping for models with tied word embeddings

[kitaekatt]

Summary

Fixes RuntimeError: Failed to map GGUF parameters (1): ['lm_head.weight'] for models using tied word embeddings.

Changes

For models like Gemma2 that use tie_word_embeddings=True, add lm_head.weight to sideload_params to allow GGUF loading to succeed.

Root Cause

When tie_word_embeddings=True, model shares weights between input embeddings and output projection. GGUF files don't contain separate lm_head.weight tensor.

Testing

Tested with bartowski/gemma-2-2b-it-GGUF - model loads without parameter mapping error.

[gemini-code-assist]

Code Review

This pull request introduces a fix for loading GGUF models with tied word embeddings, such as Gemma2, by correctly skipping the mapping of lm_head.weight. Additionally, it proactively addresses potential data type compatibility issues on newer hardware like NVIDIA's Blackwell GPUs by preventing the use of bfloat16 with GGUF quantization due to precision issues and implementing a mechanism to automatically select a compatible dtype when conflicts arise. The changes are well-implemented and improve the robustness of GGUF model loading. The logic for handling tied embeddings and resolving dtype conflicts is sound. I have no major concerns with this pull request.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @kitaekatt.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[kitaekatt]

Testing performed:

Tested with GGUF models on RTX 5090 (32GB, Blackwell architecture):

• bartowski/NousResearch_Hermes-4-14B-GGUF
• tensorblock/Qwen_Qwen3-30B-A3B-Instruct-2507-GGUF

Ran models through a local benchmark runner for HumanEval and GSM8K. Verified the semaphore leak fix by running repeated model load/unload cycles without resource exhaustion.

Related PRs:

This is part of a series of GGUF pipeline fixes for Blackwell GPU compatibility:

• fix(gguf): Auto-select compatible dtype for GGUF models on Blackwell #30410: Fix GGUF dtype selection for Blackwell compatibility
• fix(gguf): Skip lm_head mapping for models with tied word embeddings #30412 (this PR): Fix tokenizer semaphore leak
• fix(gguf): Use EOS token ID from GGUF metadata instead of HF tokenizer #30434: Shared memory barrier fix

[kitaekatt]

Validation Results

vLLM	transformers	Cherry-picked PRs	HumanEval	IFEval
HEAD	5.x	#30410, #30411, #30412, #30413, #30424, #30434, #30699, #30702, #31464, #33846	gem2-2b-gguf (42.1%), gemma3-1b (26.8%)	gem2-2b-gguf (65.6%)
HEAD	4.x	#30410, #30411, #30412, #30413, #30424, #30434, #30699, #30702, #31464, #33846	q3-moe-gguf (83.5%)	q3-moe-gguf (85.4%)

Tested on RTX 5090 (Blackwell, SM 120) with all listed PRs cherry-picked together; models listed under each benchmark passed that benchmark in the given environment, while the same models crash or fail without these PRs applied.

Rebased to current upstream HEAD and re-validated on RTX 5090 (Blackwell, SM 120). Fix confirmed still necessary — gem2-2b-gguf and gemma3-1b (both with tied embeddings) crash without it.

[kitaekatt]

Hi @22quinn @Isotr0py — this has been sitting without review for a while. Just rebased on latest upstream/main (merge commit), so the branch should now be mergeable. Quick summary: Skip lm_head mapping for models with tied word embeddings:small fix in gguf_loader.py — tied-embedding models were failing when lm_head mapping was attempted. Rebased on latest main.

Would appreciate a look when you have cycles. Thanks!