ALiBi Implementation#101

Merged

thomasw21 merged 7 commits intobigscience-workshop:mainfrom

Sep 24, 2021

Commits on Sep 10, 2021

alibi implementation
ofirpress
committed

Commits on Sep 15, 2021

works but slow
Ofir Press
committed
efficient implementation
Ofir Press
committed

Commits on Sep 16, 2021

Update transformer.py
ofirpress
authored
Update transformer.py
ofirpress
authored
Remove unnecessary .view()

ofirpress
and
thomasw21
authored
bugfix with beta value in attention computation
Ofir Press
committed