Skip to content

GQA models have not supported prefix caching#2873

Closed
toslunar wants to merge 1 commit intovllm-project:mainfrom
toslunar:prefix-gqa-not-yet
Closed

GQA models have not supported prefix caching#2873
toslunar wants to merge 1 commit intovllm-project:mainfrom
toslunar:prefix-gqa-not-yet

Commits

Commits on Feb 14, 2024