Add support for Mistral fp8#935

Closed

jiminha wants to merge 100 commits into

huggingface:mainfrom

HabanaAI:ae_mistral_fp8_new

Commits on Jan 26, 2024

Release: v1.10.0
regisss
committed

Commits on Feb 12, 2024

Commits on Feb 17, 2024

Commits on Feb 18, 2024

Commits on Feb 19, 2024

Expose Llama Fused OPs control from run_lora_clm.py (#23)

Vivek Goel
authored and
Bhargav
committed
enable internal kv bucket in llama (#24)

xt574chen
authored and
Bhargav
committed
[SW-173358] add first token prints (#18)

shakkedw
authored and
Bhargav
committed
Enable Flash Attention in recompute and causal modes (#21)

wszczurekhabana
authored and
Bhargav
committed
Fix inference command clip-roberta (#31)

MohitIntel
authored and
Bhargav
committed
Changing backend name (#32)
Bhargav
committed
enable falcon-180b inference (#15)

schoi-habana
authored and
Bhargav
committed
Add support for safetensors and sharded checkpoints (#25)

authored and
Bhargav
committed
Update ckpt loading (#38)

schoi-habana
authored and
Bhargav
committed
Further fixes for performance with internal bucketing. (#36)

Puneesh Khanna
authored and
Bhargav
committed
Update llama-7b command to include eval (#43)

jiminha
authored and
Bhargav
committed
[BridgeTower] Fix for NoneType in clip mediapipe (#45)

MohitIntel
authored and
Bhargav
committed

Commits on Feb 22, 2024

Enable Llama2 70B to run with hqt on single card (#50)

Yantom1
authored and
Bhargav
committed
Fixing tests by making static_shapes False (#60)
Bhargav
authored

Commits on Feb 23, 2024

Change check to False explicitly for use_fused_rope (#62)
hlahkar
authored

Commits on Feb 25, 2024

change quant conf example to use act_maxabs_pow2_weights_pcs_opt_pow2 (#69)
dudilester
authored

Commits on Feb 27, 2024

Commits on Feb 28, 2024

Commits on Feb 29, 2024

Upate stdxl train script with dev_ae_stdxl_ft a70a903 (#83)
yeonsily
authored
Add mark step and inplace residual add in llama model code to reduce memory consumption (#65)
Puneesh Khanna
authored
enable hpu_graph support for wav2vec2-asr (#59)
Vivek Goel
authored
Clean-up BERT-BASE FSDP test (#82)
Vivek Goel
authored

Commits on Mar 2, 2024

Commits on Mar 4, 2024

Split the graphs to run with flash_attention on 1x (#75)

kalyanjk
and
kalyanjkk
authored

Commits on Mar 5, 2024

Commits on Mar 6, 2024

Commits on Mar 7, 2024

Commits on Mar 12, 2024

Commits on Mar 13, 2024

Enable Falcon FP8 inference (#94)

schoi-habana
and
Local Lab User
authored

Commits on Mar 15, 2024

sarkar/mistral optimizations (#112)
ssarkar2
authored

Commits on Mar 17, 2024

Commits on Mar 18, 2024

Added additional check to run with distributed enabled and world_size=1 (#96) (#116)

kalyanjk
and
kalyanjkk
authored

Commits on Mar 19, 2024

Commits on Mar 20, 2024

sarkar/Fix barrier (#124)
ssarkar2
authored

Commits on Mar 21, 2024

Commits on Mar 24, 2024

Encapsulate FSDPA in GaudiLlamaAttention (#129)
dudilester
authored

Commits on Mar 28, 2024

enforce recompute flag on fsdpa quantization (#133)
dudilester
authored

Commits on Mar 29, 2024

Fixed the issue that can't take profile with 1st step. (#135)
hsubramony
authored

Commits on Apr 3, 2024

add flash_attention_causal_mask to run_lm_eval.py (#142)
dudilester
authored

Commits on Apr 5, 2024

Fix get_dtype and convert_into_dtypes (#769) (#144)
IT-Forrest
authored

Commits on Apr 8, 2024

update kvcache mistral (#145)
skaulintel
authored

Commits on Apr 9, 2024

Remove deprecated AOT_HPU_TRAINING_BACKEND (#138)
astachowiczhabana
authored

Commits on Apr 10, 2024

set all fusedrope inputs to bf16 (#140)
skaulintel
authored

Commits on Apr 11, 2024

Commits on Apr 12, 2024

Fix throughput calculation for diffusion models (#715) (#160)

astachowiczhabana
and
skavulya
authored

Commits on Apr 15, 2024

Commits on Apr 16, 2024

Mark scale as const and remove --fp8 flag usage (#156)

Yantom1
and
Eran Geva
authored

Commits on Apr 17, 2024

Remove --fp8 flag from script (#171)
Yantom1
authored

Commits on Apr 18, 2024

[SW-12028] - set ds config "reduce scatter" to false (#173)
vineethanandh
authored

Commits on Apr 21, 2024

Commits on Apr 22, 2024

Fix RoPE data type issue for gpt_neox and stablelm (#177)
mandy-li
authored

Commits on Apr 23, 2024

Add rouge metric evalution for llama 70B with orca datasets (#169)
sushildubey171
authored

Commits on Apr 30, 2024

Added Mistral fp8 support
jiminha
committed