Update Mixtral-8x7B Optimization#836

Closed

jychen21 wants to merge 23 commits into

huggingface:mainfrom

jychen21:update-mixtral-optimizations

Commits on Mar 26, 2024

add reuse_cache support
Jinyan chen
committed
make style
Jinyan chen
committed
make style
Jinyan chen
committed
remove debug code
Jinyan chen
committed
add fp8 support of non-sdpa attn
Jinyan chen
committed
add bucket_internal support of Mixtral
Jinyan chen
committed

Commits on Apr 1, 2024

pick changes
Jinyan chen
committed

Commits on Apr 3, 2024

fit to r1.15 and also fp8 sdpa
Jinyan chen
committed
make style
Jinyan chen
committed

Commits on Apr 9, 2024

support long sequence prompt
Jinyan Chen
committed
Merge branch 'huggingface:main' into update-mixtral-optimizations
Jinyan Chen
authored
make style
Jinyan Chen
committed
make style
Jinyan Chen
committed

Commits on Apr 10, 2024

update long seq support
Jinyan Chen
committed
fix bug for w/o reuse_kvcache
Jinyan Chen
committed
make style
Jinyan Chen
committed

Commits on Apr 11, 2024

fix accuracy of NaiveFA
Jinyan Chen
committed
update thresh of long seq
Jinyan Chen
committed
update bucket size for naive fa
Jinyan Chen
committed
make style
Jinyan Chen
committed
tune bucket size
Jinyan Chen
committed

Commits on Apr 12, 2024

update naive fa
Jinyan Chen
committed
make style
Jinyan Chen
committed