[Bugfix] Fix hybrid KV manager for quantized per-token-head KV cache - #40308
Closed
lesj0610 wants to merge 4 commits into
Closed
[Bugfix] Fix hybrid KV manager for quantized per-token-head KV cache#40308lesj0610 wants to merge 4 commits into
lesj0610 wants to merge 4 commits into
Loading