Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

QKV fuse #39080

Merged
merged 10 commits into from
Jan 29, 2022
Merged

QKV fuse #39080

merged 10 commits into from
Jan 29, 2022

Conversation

JZ-LIANG
Copy link
Contributor

@JZ-LIANG JZ-LIANG commented Jan 20, 2022 •
edited
Loading

PR types

Performance optimization

PR changes

Others

Describe

Auto Parallel support transformer QKV fuse.
profile result：

  • GPT-Medium & TensorParallel degree = 8
  • 32GB v100
no fuse fuse qkv diff
显存 MB 7347 7347 0.0
速度 token/s 2150 2550 18.6%

aoyulong
aoyulong approved these changes Jan 29, 2022
View reviewed changes
Copy link
Contributor

@aoyulong aoyulong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ForFishes
ForFishes approved these changes Jan 29, 2022
View reviewed changes
Copy link
Member

@ForFishes ForFishes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JZ-LIANG JZ-LIANG merged commit fdedf90 into PaddlePaddle:develop Jan 29, 2022
@JZ-LIANG JZ-LIANG changed the title Auto parallel/qkv fuse QKV fuse Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aoyulong aoyulong aoyulong approved these changes

@ForFishes ForFishes ForFishes approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@JZ-LIANG @aoyulong @ForFishes