New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

能否解释一下EmbeddingExt中的旋转位置编码的作用是什么？ #106

Open

junphine opened this issue Sep 24, 2023 · 0 comments

junphine commented Sep 24, 2023

我理解旋转位置编码应该在取得query和key的向量之后计算，这里为什么是在之前计算？
input_sub具体指的是什么？作为旋转位置编码的x_pos参数传入，发现他大部分场景下始终是0向量，所以旋转位置编码没有起到作用

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment