fix: handle vendor-prefixed GGUF quant types (e.g., UD-Q4_K_XL)

[ianliuy]

Summary

Fixes #39198

GGUF model publishers like Unsloth use vendor-prefixed quant type names (e.g., UD-Q4_K_XL for Unsloth Dynamic quantization). These were not recognized by is_valid_gguf_quant_type(), causing the entire GGUF detection chain to fail. The colon separator was never stripped from the model name, and the full string (with :) was passed to HuggingFace APIs, resulting in HFValidationError.

Root Cause

is_valid_gguf_quant_type() only checks exact GGMLQuantizationType enum members + standard size suffixes (_M, _S, _L, _XL, _XS, _XXS). Vendor-prefixed types like UD-Q4_K_XL are not in the enum, so the full detection chain returns False. The : is never stripped, and the full model string is passed as-is to HF's hf_hub_download, which rejects the : character.

Fix

GGML quant type names never contain hyphens - they only use underscores and alphanumerics. A hyphen is therefore a reliable signal of a vendor prefix.

is_valid_gguf_quant_type() now:

1. Tries the existing base validation (backward compatible)
2. If that fails and the string contains a hyphen, splits on the first hyphen and validates the remainder

Example: UD-Q4_K_XL -> strip UD- -> Q4_K_XL -> strip suffix _XL -> Q4_K (valid enum member)

Changes

• vllm/transformers_utils/gguf_utils.py: Extract _is_base_gguf_quant_type() helper; add vendor-prefix stripping; update error message
• tests/transformers_utils/test_utils.py: Add tests for vendor-prefixed quant types across all GGUF utility functions

Testing

• All new vendor-prefix test cases pass (valid: UD-Q4_K_XL, UD-F16, XX-Q4_K_M; invalid: UD-INVALID, UD-, -Q4_K)
• Existing non-prefixed quant types unaffected
• Edge cases covered: empty prefix, empty remainder, double-hyphen

[gemini-code-assist]

Code Review

This pull request introduces support for vendor-prefixed GGUF quantization types, such as Unsloth Dynamic (UD-), by updating the validation logic and adding comprehensive unit tests. The is_valid_gguf_quant_type function was refactored to handle hyphenated prefixes, and error messages were updated to reflect this new support. A review comment suggests using rsplit instead of split when isolating the vendor prefix to ensure robustness if a vendor name itself contains a hyphen.

[gemini-code-assist]

Using split("-", 1) only supports vendor prefixes that do not contain hyphens. If a vendor prefix itself contains a hyphen (e.g., MY-VENDOR-Q4_K), this logic will fail to validate the remainder. Since GGML quantization types are guaranteed not to contain hyphens, using rsplit("-", 1) is a more robust way to isolate the quantization type from any vendor prefix.

Suggested change

	prefix, remainder = gguf_quant_type.split("-", 1)
	prefix, remainder = gguf_quant_type.rsplit("-", 1)

[ianliuy]

Good catch! Applied in db8856f. Since GGML quant types never contain hyphens, rsplit correctly isolates the quant type from any multi-part vendor prefix (e.g., MY-VENDOR-Q4_K_XL -> remainder Q4_K_XL).

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ianliuy.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[ianliuy]

Closing this was superseded by #39471 which merged a similar fix (is_nonstandard_gguf_quant_type with rsplit). Same root cause analysis, same approach. Thanks!