New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Positional Encoding #29

Open

JeremyCJM opened this issue Feb 17, 2021 · 2 comments

JeremyCJM commented Feb 17, 2021

Why the positional encoding is random?

johndpope commented Feb 23, 2021

maybe ask @khanrc
1ac51c0#diff-6d46aff567baa42dfd4c6b8d2f5c956d3ea84b2811ca663ea67159e323344ccb

Contributor

khanrc commented Feb 24, 2021

It is learnable positional embedding, which is initialized randomly and learned during training. Please refer to the paper.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment