Fix VRAM usage increase on HIP/ROCm with RDNA3 and possibly more - #71

Closed

stragulus wants to merge 2 commits into

TheTom:feature/turboquant-kv-cachefrom

stragulus:bug/vram-usage-rows-with-rocm

Commits on Apr 11, 2026

Implement fixed batch size flash attention kernel invocation to avoid
stragulus
committed
6266acf

Commits on Apr 12, 2026

The batched flash attention kernel changes introduced two issues that this commit fixes.
stragulus
committed
f519788