Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Suport sdpa for RoBERTa and XLM-RoBERTa models #31752

Open
kiszk opened this issue Jul 2, 2024 · 0 comments · May be fixed by #31754
Open

Suport sdpa for RoBERTa and XLM-RoBERTa models #31752

kiszk opened this issue Jul 2, 2024 · 0 comments · May be fixed by #31754
Labels
Feature request Request for a new feature

Comments

@kiszk
Copy link
Contributor

kiszk commented Jul 2, 2024

Feature request

Enable sdpa for RoBERTa and XLM-RoBERTa models

Motivation

While BERT, which is similar to RoBERTa and XLM-RoBERTa, support sdpa, RoBERTa and XLM-RoBERTa do not support sdpa yet. This enablement is straight-forward.

Our applications need latency reduction by sdpa. The performance advantage in BERT is already shown at #28802.

Your contribution

I will submit a PR.
@kiszk kiszk added the Feature request Request for a new feature label Jul 2, 2024
@kiszk kiszk linked a pull request Jul 2, 2024 that will close this issue
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature request Request for a new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[roberta] add sdpa to roberta and xlm-roberta kiszk/transformers
1 participant
@kiszk