cuda : fix Turbo4 VEC decode without dot2

[mgeldi]

Overview

Fixes #207.

The manual Turbo4 VEC value decoder used on CUDA builds without V_DOT2_F32_F16_AVAILABLE expands four rows per thread, while the scheduler selected eight rows. That row-ownership mismatch corrupts the Turbo4 V cache.

This change:

• selects four rows for Turbo4 only when the manual decoder is compiled;
• preserves the existing eight-row Turbo4 schedule for the dot2/HIP decoder;
• adds a compile-time assertion next to the manual decoder;
• adds deterministic Turbo4 VEC regression cases for D=128 and D=256.

Additional information

Related: #229 and #253 explored the same regression with broader scheduling changes. This version is deliberately narrower: it leaves the dot2/HIP Turbo4 path unchanged and directly tests the manual decoder at both supported head sizes.

Tested on an NVIDIA GeForce RTX 5090 with CUDA 13.3, SM120a, and a Release build with all quantized flash-attention kernels enabled:

• Unchanged base plus the two new tests: 0/2 passed, both with error 1.000003040.
• Patched focused regression: 2/2 passed.
• Compute Sanitizer memcheck, racecheck, synccheck, and initcheck: 2/2 passed in each mode, with zero errors or hazards.
• test-turbo-quant: passed.
• test-quantize-fns: passed; the generic suite's rotated-domain Turbo2/3/4 skips are expected.
• Full test-backend-ops test -j 1: 21,661/21,661 CUDA0 cases passed against the CPU reference; both new cases executed.
• Focused full FLASH_ATTN_EXT sweep: 10,927/10,927 passed.
• Qwen3.8 27B NVFP4 with Q8_0 K cache, Turbo4 V cache, and flash attention produced the deterministic target output CUDA path healthy.
• llama-bench, five repeats: pp512 5,018.35 +/- 199.67 tok/s; tg128 68.15 +/- 0.47 tok/s.

Runtime coverage is limited to this NVIDIA/CUDA configuration. The HIP dot2 path is preserved by the compile-time branch but was not runtime-tested here; MUSA and older NVIDIA architectures were not available.

Requirements

• I have read and agree with the contributing guidelines.
• AI usage disclosure: YES - AI materially assisted the root-cause analysis, implementation, tests, benchmarks, reviews, and PR preparation. I manually reviewed every changed line, understand the VEC scheduling and decoder contract, verified the claims above, and am prepared to explain and maintain the change.

[mgeldi]

CI triage after the initial run:

• The hosted accelerator workflow passed on CUDA, HIP, and MUSA: run 32820881698.
• HIP quality reports four major-VGPR lines, including both fattn-vec.cuh variants at 259 (255 + 4). The quality run on the exact PR base SHA has the same four lines and counts, so this PR did not increase register pressure.
• Server Ubuntu and Windows fail test_slot_save_restore immediately after state_read_meta: cell_count = 84, dest_seq_id = 0, with 260 tests passed first. An unrelated earlier PR run has the same failure on both platforms.
• Windows/OpenBLAS finishes all test-thread-safety workers and then exits 0xc0000374 during cleanup. An unrelated earlier PR run has the same signature.

The Windows x64 static and Vulkan CPU jobs were cancelled by matrix fail-fast. build-cmake-pkg / linux and the self-hosted matrix are still queued without runners. I have not changed or blindly retried the patch because none of the completed failures differs from the existing baseline behavior.

[TheTom]

Thanks, @mgeldi. This was an easy one to miss in the CI noise, but the patch is focused and well supported. The schedule now matches the manual Turbo4 decoder contract only on builds without dot2, the HIP/dot2 path stays unchanged, and the deterministic D=128/D=256 cases directly cover the regression. The full CUDA backend sweep, sanitizers, and hosted CUDA/HIP/MUSA run are enough here. The red checks are confirmed baseline failures, including the slot restore issue now fixed through #324. Merging this.