Deterministic Hadamard KQ rotation

[captainpete]

Purpose

First incremental step toward serving compressed-tensors checkpoints that use transform_config for Hadamard KV cache rotation (SpinQuant R3). See vllm-project#28538 and vllm-project/compressed-tensors#436.

Scope of this PR:

• Deterministic Hadamard only (hadamard type, randomize=False)
• FP16/BF16 weights; no INT4 or FP8 KV cache quantization
• K_CACHE and Q_ATTN transform locations only
• Llama-family models (other architectures need one line added to their load_weights)

What is not yet supported (raises NotImplementedError at model load):

• random-hadamard type and randomize=True (require loading a per-layer rotation matrix from the checkpoint)
• ROCm (CUDA-only kernel)

compressed-tensors PR#436 introduced TransformConfig, TransformScheme, and TransformArgs as the schema for describing these rotations. This PR reads that schema from the checkpoint's quantization_config and applies it at inference.

Changes:

• CompressedTensorsKVCacheMethod.create_weights() reads transform_config, identifies layers with K_CACHE or Q_ATTN transform locations, and sets _kq_attn_transform = True on the corresponding Attention module
• Attention.forward() applies ops.hadacore_transform to Q and K after reshape, immediately before the paged attention call
• Validation at model load raises clear errors for unsupported configurations: random-hadamard, randomize=True, ROCm, head_dim mismatch, non-power-of-two head_dim, and conflict with FP8 KV scales
• is_hadamard_transform_weight() in weight_utils.py handles stored rotation matrices in R3 checkpoints (R3_q_attn.weight, R3_k_cache.weight). The deterministic FWHT reconstructs H from head_dim alone, so these weights are skipped on load following the same pattern as maybe_remap_kv_scale_name

The rotation is checkpoint-driven: it fires when the checkpoint carries transform_config, with no CLI argument required. This aligns with the direction in #sig-quantization to deprecate the quantization= CLI arg for offline quantization.

Known limitation: is_hadamard_transform_weight() is currently called only in llama.py. Other models need the same call added when R3 checkpoints are produced for those architectures. This is the same per-model propagation pattern as maybe_remap_kv_scale_name, which is already in 64 model files.

Test Plan

Unit tests (no GPU, runs on all platforms):

.venv/bin/python -m pytest tests/quantization/test_hadamard_kv_dispatch.py -v
.venv/bin/python -m pytest tests/model_executor/test_weight_utils.py::TestIsHadamardTransformWeight -v

End-to-end against a real R3 checkpoint (~16GB VRAM, not suitable for CI):

# With rotation (default)
VLLM_WORKER_MULTIPROC_METHOD=spawn .venv/bin/python tests/quantization/test_hadamard_kq_r3_e2e.py

# Baseline (rotation stripped from config.json)
VLLM_WORKER_MULTIPROC_METHOD=spawn .venv/bin/python tests/quantization/test_hadamard_kq_r3_e2e.py --no-rotation

Test Result

Unit tests: all pass (no GPU required).

E2E on nm-testing/Meta-Llama-3-8B-Instruct-spinquantR3, RTX 3090, bf16:

Run	Throughput
Hadamard rotation enabled	240.7 tok/s
Baseline (rotation stripped)	240.5 tok/s

Throughput is indistinguishable between the two runs (0.1% difference, within noise). Both runs produce coherent outputs with no crashes or NaNs. The accuracy benefit of Hadamard rotation is only measurable under INT4/FP8 KV cache quantization; the bf16 run confirms correctness of the dispatch and kernel execution path.