reland - use fast softmax only on prefill#309

Closed

jaygala223 wants to merge 10 commits into

HabanaAI:v1.17-synapsefrom

jaygala223:jay-softmax-fix-2

Commits on Jul 11, 2024

[SW-191023][PyTorch][Optimum-Habana-fork]: Add flag to run inference with partial dataset (#281)
pramodkumar-habanalabs
authored
Move the 1st token finish time to not include 2nd step kv pad time (#292)
shepark
authored

Commits on Jul 12, 2024

Commits on Jul 15, 2024

[SW-192667] Revert "[SW-185803] Enable FusedSDPA fp8 in Llama FT (#291)" (#295)

Eran Geva
and
Eran Geva
authored
fix data-type mismatch for mlperf_inference accuracy test (#296)

kalyanjk
and
kalyanjkk
authored
Tensor parallel distributed strategy without using deepspeed (#280)

kalyanjk
and
kalyanjkk
authored

Commits on Jul 16, 2024

Commits on Jul 17, 2024

reland - use fast softmax only on prefill
Jay
committed