cuda: shared-quantize cache, residual and elementwise-chain fusions (generic half of #338) - #343

Merged

TheTom merged 5 commits into

TheTom:feature/turboquant-kv-cachefrom

jasstrong:pr/cuda-generic-fusions

Sep 3, 2026

Commits on Sep 3, 2026

perf(cuda): per-graph-eval shared-quantize cache for mmvq

jas
and
claude
committed
767b627
perf(cuda): elementwise chain fusion (midi-kernel)
jas
committed
42b990b
test: cover the multi-node CUDA fusions in test-backend-ops

jas
and
claude
committed
2fb1a70
cuda: free held cache buffers newest-first at context teardown
TheTom
committed
f47832e
cuda: take the persistent q8 and TQ pre-rotation cache buffers from device memory, not the pool
TheTom
committed
ad5c1f1