Skip to content

Update repeat KV llama logic for better TP-4 performance#639

Merged
libinta merged 4 commits into
huggingface:mainfrom
puneeshkhanna:repeatKVfix
Jan 24, 2024
Merged

Update repeat KV llama logic for better TP-4 performance#639
libinta merged 4 commits into
huggingface:mainfrom
puneeshkhanna:repeatKVfix

Commits

Commits on Jan 16, 2024

Commits on Jan 17, 2024

Commits on Jan 23, 2024