docs : add TurboQuant project overview to AGENTS.md and skills

AGENTS.md

@@ -1,4 +1,76 @@
-# Instructions for llama.cpp
+# Instructions for llama.cpp (TurboQuant fork)
+
+## Project Overview
+
+This repo is a fork of [llama.cpp](https://github.com/ggml-org/llama.cpp) (upstream) that adds the TurboQuant feature set on top of a fully-synced upstream base. The local tree always contains all of upstream master plus fork additions; rebasing onto latest upstream is a recurring task.
+
+### What TurboQuant adds
+
+TurboQuant compresses the KV cache far beyond the standard `q8_0` by applying a fixed 128x128 orthonormal Walsh-Hadamard rotation (`GGML_OP_TURBO_WHT`) to cache vectors before quantization, which Gaussianizes the distribution, and inverse-rotating after dequantization. Head dims that are not multiples of 128 are zero-padded. MLA models (DeepSeek) have no separate V cache, so V rotation/padding is skipped for them, and K/V cache types must be identical.
+
+The five fork-only GGML types (registered in `ggml/include/ggml.h`):
+
+| Type   | Enum                       | Purpose                             | Size         |
+|--------|----------------------------|-------------------------------------|--------------|
+| turbo2 | `GGML_TYPE_TURBO2_0` (43)  | KV cache only                       | 2 bits/value |
+| turbo3 | `GGML_TYPE_TURBO3_0` (44)  | KV cache only                       | 3.25 bits    |
+| turbo4 | `GGML_TYPE_TURBO4_0` (47)  | KV cache only                       | 4.25 bits    |
+| TQ3_1S | `GGML_TYPE_TQ3_1S` (45)    | model weights, WHT-rotated Lloyd-Max| 3 bits, block 32 |
+| TQ4_1S | `GGML_TYPE_TQ4_1S` (46)    | model weights                       | 4 bits, block 32 |
+
+Turbo cache types are runtime-only, never stored in GGUF. TQ3_1S/TQ4_1S are first-class weight types with CPU, CUDA/HIP (warp-cooperative mmvq), Metal, Vulkan, and SYCL kernels, exposed as `llama-quantize` targets.
+
+### Key files
+
+- `ggml/src/ggml-turbo-quant.c` - the codec (keep byte-identical to fork tip)
+- `ggml/include/ggml.h` - type enum 43-47, `GGML_OP_TURBO_WHT`
+- `src/llama-kv-cache.cpp` - cache wiring, `get_k_idx`, layer-adaptive precision
+- `src/llama-graph.cpp` - inverse-WHT post-processing (FA and non-FA paths)
+- `ggml/src/ggml-cuda/mmvq-tq.cu` - native TQ dp4a kernels (`GGML_TQ_NATIVE=1`)
+- `ggml/src/ggml-vulkan/` - turbo FA, SET_ROWS, dequant shaders
+- `ggml/src/ggml-metal/ggml-metal.metal` - TurboFlash kernels
+- `docs/KV-cache-quantization.md` - authoritative usage doc (read before touching cache types)
+
+### Usage
+
+```bash
+llama-cli -m model.gguf -c 8192 -ngl 99 --cache-type-k q8_0 --cache-type-v turbo3
+```
+
+Any combination of `f16`/`q8_0`/`turbo2`/`turbo3`/`turbo4` for K and V is supported. Turbo cache types require flash attention; it is auto-enabled with a warning. Quantized V with FA explicitly disabled is an error (upstream behavior). The same flags work in `llama-server`, `llama-bench`, `llama-perplexity`.
+
+### Environment knobs
+
+| Variable                    | Default | Effect |
+|-----------------------------|---------|--------|
+| `TURBO_LAYER_ADAPTIVE`      | `0`     | Layer-adaptive KV precision; `7` = Boundary V (edge layers q8_0, middle turbo) |
+| `TURBO_AUTO_ASYMMETRIC`     | `1`     | Auto-select asymmetric K/V types for large-GQA models |
+| `TURBO_SPARSE_V`            | `1`     | Sparse-V dequant skip in flash attention |
+| `GGML_TQ_NATIVE`            | unset    | `1` opts out of load-time TQ->q8_0 conversion, uses fused native TQ kernels (saves ~1.7x VRAM on decode-heavy workloads) |
+| `LLAMA_ATTN_ROT_K/V_OVERRIDE` | off   | Optional upstream attention rotation (TurboQuant manages its own rotation) |
+
+### Test gates (all must pass before touching quant/backend code)
+
+- `test-turbo-quant` - turbo3 basis MSE=0/Cosine=1.0, turbo4 Cosine=0.9956
+- `test-quantize-fns` - includes TQ3_1S/TQ4_1S and rotated-domain buffer sizing
+- `test-backend-ops` - full sweep on CPU + CUDA0 (23k+ cases on the RTX 5090 dev box)
+- `llama-bench` with `-ctk/-ctv turboN`; type parser accepts `tq3_1s`/`tq4_1s`
+
+### Git workflow
+
+- Remotes: `origin` = TheTom/llama-cpp-turboquant (this repo); the fork remote tracks the upstream TurboQuant fork (same repo, two names); add `upstream` = ggml-org/llama.cpp when syncing
+- Main branches: `feature/turboquant-kv-cache` tracks the upstream TurboQuant fork
+- Upstream master is always fully contained in the tree (verified by rebase parity audits; git log is the record of the last sync point)
+
+### Known pitfalls (each caused a real bug once - check these first on regressions)
+
+- **Metal**: turbo kernels need their `[[host_name]]` instantiations; a missing one is a NULL-pipeline deref on the first turbo KV write.
+- **Vulkan**: SET_ROWS pipeline registration must include TURBO2_0/3_0/4_0 with `require_full_subgroups=true, subgroup_size=32`, or every turbo KV write aborts.
+- **CUDA dispatch**: TQ weights must be excluded from the mmvq path before the fused-TQ branch (`ggml_cuda_should_use_mmvq`), or `GGML_TQ_NATIVE=1` aborts.
+- **CUDA TQ4_1S decode**: the centroid LUT in `mmvq-tq.cu` must use plain shifts, not `__byte_perm` - constant selectors fold differently from runtime ones on some toolchains and silently produce garbage output (NMSE ~1.0). Comment in the file explains; do not "clean up" the shift code.
+- **DeepSeek/MLA**: K and V cache types must be identical; turbo FA auto-enable runs before upstream's quantized-V FA check.
+- **MoE models**: CUDA graphs are auto-disabled for TQ `MUL_MAT_ID`.
+- **gguf-py**: keep model constants deduplicated; stacked-duplicate merge artifacts crash `import gguf`.
 
 > [!IMPORTANT]
 >