[WindChimeRan]

Superseded by #859, which is part of the native GitHub stack with branches in this repository. The original description is retained below.

Stacked on #856. The base branch mirrors its current head; rebase and retarget this PR to main after #856 lands.

Motivation

• Hunyuan Dense applies Q/K normalization after RoPE ([Bugfix] Make QK norm placement architecture-aware #664); OLMo2 normalizes the full projections before splitting heads. The MLX paged-attention wrapper needs architecture-specific contracts to preserve these differences.
• Run those models through upstream Transformers definitions in vLLM, so their forward methods own this ordering and the MPS backend receives prepared Q/K/V.

Summary

• Enable unquantized Hunyuan Dense V1 and OLMo2 on the opt-in MPS/MRv2 path, using the existing 128-dimensional attention kernels.
• Resolve head size through vLLM's model metadata API. No new model wrappers, normalization-placement flags or kernels are needed.
• Add one parameterized regression to the existing test file: non-uniform norm weights, CPU-reference token/logprob parity, chunked prefill, decode and prefix reuse for both architectures.
• Validate the real tencent/Hunyuan-1.8B-Instruct and allenai/OLMo-2-0425-1B-Instruct checkpoints in FP16: 24/24 greedy tokens matched the CPU reference for each across three chat prompts; repeated-prefix outputs also matched. Maximum chat-probe logit relative RMS error was 0.99% / 0.25%. An additional raw long-prompt Hunyuan probe measured 1.03% versus CPU and 0.68% versus dense MPS, with matching argmax.
• Local regression: 272 passed, 13 skipped; lint and type checks passed.
• Other attention widths, sliding-window, MoE and hybrid-state families remain separate follow-ups. Extended probes and downloaded checkpoints stay local.