Transformer-kernel - supporting any arbitrary sequence-length#587

Merged

jeffra merged 9 commits intomasterfrom

transformer/support-arbitrary-seqlen

Dec 17, 2020

Commits on Dec 8, 2020

Transformer-kernel - supporting any arbitrary sequence-length
Reza Yazdani
committed
remove seq-len from transformer config
Reza Yazdani
committed
pad seq-len to be 16-aligned
Reza Yazdani
committed

Commits on Dec 9, 2020

resolve the issue with softmax forward when sequence is low
Reza Yazdani
committed

Commits on Dec 11, 2020

Commits on Dec 16, 2020

make the padding more efficient
Reza Yazdani
committed

Commits on Dec 17, 2020