examples: add TurboAnchorKV evaluation harness

[TheTom]

What this is

TurboAnchorKV is an experimentally derived two-tier cold-prompt KV algorithm combining anchor projection with TurboQuant residual coding. The selected design is the result of ablation, not a direct copy of AnchorKV:

• deterministic uniform old-context anchors plus a recent bf16 window
• pre-RoPE K projection
• independent K/V nearest-anchor projection with bf16 coefficients
• residual-norm prioritization instead of observation-utility scoring
• block-128 TurboQuant residuals with configurable turbo2, turbo3, turbo4, or f16 fidelity

The intended user benefit, if quality gates can be met, is substantially longer contexts and more concurrent long-context sessions within the same unified-memory budget, with less cold-KV bandwidth and no retraining.

What this PR adds

• llama-turboanchorkv, an offline analyze/PPL/generation harness
• a real packed representation with bf16 anchors and coefficients, uint16 assignments, uint32 O(1) residual-slot maps, and contiguous TurboQuant residual rows
• serialized byte accounting over every packed array
• real-cache pack/reconstruct evaluation with state-matched baseline comparison
• exact PPL, mean KL, and top-1 agreement metrics
• model-free tests for RoPE recovery, uniform anchors, norm ranking, codec quality ordering, partial slot lookup, and packed byte accounting
• deterministic needle-in-a-haystack prompt generation
• paper-style attention/utility and random/post-RoPE controls for ablation
• usage instructions, expected invariants, measured M5 Max results, and known limits

This does not add a compact runtime cache allocation or attention kernel. It reconstructs the packed state into the existing dense cache to test algorithmic quality first.

Implementation lineage

Thanks @giveen for the AnchorKV CPU reference work. The packed storage and slot-map design were adapted from:

• 875a1f370: initial packed storage format
• db851cea8: precomputed O(1) residual-slot indices
• 6ac311a72: residual slot-order bug fix and graph decompression work

The reference cache lifecycle, shared scratch, graph operator, CUDA path, approximate utility scorer, and custom 2-bit codec were reviewed but not imported. The README links the branch and originating commits.

Current evidence

Gemma 4 12B Q4_K_XL, Metal, f16 KV, about 8.5K prefix tokens:

Evaluation	Actual packed ratio	Result
Long identity round-trip	1x	zero PPL delta, zero KL, 1.0 top-1
Docs continuation, turbo2	10.001x	+58.49% PPL, KL 0.685, 78.9% top-1
Docs continuation, turbo4	5.001x	+10.70% PPL over 512 tokens, KL 0.894, 78.7% top-1
Wikitext-2 excerpt, turbo4	5.001x	+23.86% PPL over 512 tokens, KL 1.166, 66.2% top-1
Needle, turbo4	5x	exact 15/15-token baseline match at 10%, 50%, and 90% depths
Needle, turbo2	10x	exact baseline match at 10% depth

The identity control validates the evaluator, packed slot ordering is directly tested, and retrieval remains promising. General next-token quality does not pass a production-parity gate. This PR is therefore a testable research foundation, not a production mode.

How to review and reproduce

See examples/turboanchorkv/README.md for build commands, the identity gate, Wikitext continuation evaluation, needle grids, environment controls, and expected output.

Validated locally with:

• cmake --build build-anchorkv --target llama-turboanchorkv test-turbo-quant -j 10
• ctest --test-dir build-anchorkv -R test-turboanchorkv\|test-turbo-quant --output-on-failure
• ./build-anchorkv/bin/llama-turboanchorkv --self-test
• representative Metal/Gemma analyze, long identity, docs/Wikitext PPL, and four needle runs
• xcrun clang-format --dry-run --Werror examples/turboanchorkv/turboanchorkv.cpp
• git diff --check

Relates to #332.

AI disclosure

AI assistance was used for paper analysis, implementation, test construction, measurements, and documentation. The repository owner requested this work and remains responsible for review and acceptance.

[TheTom]

Thanks @giveen for sharing the AnchorKV CPU reference branch. Its history directly informed the useful storage subset now in this PR:

• 875a1f370 established the packed per-head anchor/residual representation.
• db851cea8 introduced precomputed O(1) residual-slot indices.
• 6ac311a72 documented and fixed the residual slot-order bug while adding the graph decompression path.

TurboAnchorKV now uses the packed-anchor and slot-map ideas with its own ablated algorithm: uniform anchors, residual-norm allocation, bf16 projection metadata, and the existing TurboQuant residual codecs. Reconstruction is performed solely from the packed representation and its byte accounting includes all stored arrays.

We intentionally did not import the broader cache lifecycle, shared scratch, graph op, CUDA path, approximate utility scorer, or custom 2-bit codec. Those remain useful implementation references if the quality gates eventually justify a production kernel.

Validation after the selective port:

• model-free packed/partial-slot test passes, including non-position residual ordering
• 8,871-token identity: zero PPL delta, zero KL, 1.0 top-1
• Wikitext-2 turbo4 at actual 5.001x: +23.86% PPL, KL 1.166, 66.2% top-1
• needle retrieval at 10%, 50%, and 90%: exact 15/15-token baseline match at 5x
• turbo2 10x needle at 10%: exact baseline match

The README links the originating branch and commits so the lineage stays visible. Appreciate the groundwork and especially the slot-order bug history; it gave us the right negative test to add.

[giveen]

My non-computer-programmer brain could only go so far, so I am glad you are picking this up.

[giveen]

Test project /mnt/storage/Projects/llama-cpp-turboquant/build
    Start 64: test-turboanchorkv
64: Test command: /mnt/storage/Projects/llama-cpp-turboquant/build/bin/llama-turboanchorkv "--self-test"
64: Working Directory: /mnt/storage/Projects/llama-cpp-turboquant/build/examples/turboanchorkv
64: Test timeout computed to be: 1500
64: TurboAnchorKV self-test: rope=ok anchors=ok ranking=ok residuals=ok slots=ok
1/1 Test #64: test-turboanchorkv ...............   Passed    0.09 sec

100% tests passed, 0 tests failed out of 1