feat: TurboQuant KV cache compression (V-only, flag-protected)#157

Merged

raullenchai merged 10 commits intomainfrom

feat/turboquant-kv-cache

Apr 20, 2026

Commits on Apr 20, 2026

feat: TurboQuant KV cache compression (V-only, flag-protected)

Your Name
and
claude
committed
fix: codex review — correct Lloyd-Max codebooks, memory estimation, FP32 rotation

Your Name
and
claude
committed
feat: nibble bit-packing + integration tests + memory estimation fix

Your Name
and
claude
committed
test: memory pressure benchmark + compression logging

Your Name
and
claude
committed
fix: TurboQuantKVCache crash in _trim_cache_offset (Gemma 4 bug)

Your Name
and
claude
committed
fix: codex review — server.py flags, trim test, isinstance check, dead code

Your Name
and
claude
committed
docs: add TurboQuant to README features + stress test verified

Your Name
and
claude
committed
refactor: replace boundary-based TurboQuant with PR #1059 cache patch

Your Name
and
claude
committed
revert: restore boundary-based TurboQuant (v2 incompatible with BatchGenerator)

Your Name
and
claude
committed
fix: unify savings percentage in CLI help text (44% → 86%)

Your Name
and
claude
committed