Skip to content

Move the 1st token finish time to not include 2nd step kv pad time#292

Merged
libinta merged 1 commit into
HabanaAI:habana-mainfrom
shepark:fix_1st_token_latency
Jul 11, 2024
Merged

Move the 1st token finish time to not include 2nd step kv pad time#292
libinta merged 1 commit into
HabanaAI:habana-mainfrom
shepark:fix_1st_token_latency

Commits

Commits on Jul 11, 2024