[WIP] Adding GPT2 with Multi Query Attention#21253

Closed

lvwerra wants to merge 11 commits intomainfrom

Commits on Jan 20, 2023

run template
leandro
committed
update modeling file
leandro
committed
update config
leandro
committed
fix style
leandro
committed
undo unnecessary changes
leandro
committed
add MQA Readme
leandro
committed
fill info
leandro
committed

Commits on Jan 23, 2023

remove pruning tests
leandro
committed
remove upcasting tests
leandro
committed
change checkpoint name in tests
leandro
committed
fix style
leandro
committed