Add KV cache quantization for prefix cache memory reduction#62

waybarrios merged 9 commits into

mainwaybarrios/vllm-mlx:mainfrom

feat/kv-cache-quantizationwaybarrios/vllm-mlx:feat/kv-cache-quantizationCopy head branch name to clipboard