fix: Force float16 dtype for GGUF models to fix incorrect output#30090

Closed

kitaekatt wants to merge 3 commits into

vllm-project:mainfrom

kitaekatt:fix-gguf-bfloat16-dtype

Commits on Dec 9, 2025

fix: Default GGUF to float16 while preserving bfloat16 option
kitaekatt
committed
fix(gguf): Disable bfloat16 on Blackwell (SM 120+) via device capability check

kitaekatt
and
Isotr0py
committed
fix(gguf): Remove dtype auto-override, rely on quantization layer
kitaekatt
committed