Skip to content

feat: support cuda graph for batched multi-query(prefill/append) attention#277

Merged
yzh119 merged 7 commits intomainfrom
prefill-cuda-graph-new
Jun 2, 2024
Merged

feat: support cuda graph for batched multi-query(prefill/append) attention#277
yzh119 merged 7 commits intomainfrom
prefill-cuda-graph-new

Commits

Commits on Jun 2, 2024