Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Break hpu graphs into two for better performance #14656

Merged
merged 16 commits into from
Sep 13, 2022
Merged

Break hpu graphs into two for better performance #14656

merged 16 commits into from
Sep 13, 2022

Conversation

jerome-habana
Copy link
Contributor

Signed-off-by: Jerome [email protected]

What does this PR do?

HPU performs better if there are bigger graphs and the current recommendation is to have fwd+bwd into one graph and optimizer as another. This change is to aid this split

Fixes #<issue_number>

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Sep 12, 2022
@jerome-habana jerome-habana marked this pull request as ready for review September 12, 2022 12:35
@mergify mergify bot added the ready PRs ready to be merged label Sep 12, 2022
@mergify mergify bot added has conflicts and removed ready PRs ready to be merged labels Sep 12, 2022
@mergify mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels Sep 12, 2022
@mergify mergify bot added the ready PRs ready to be merged label Sep 12, 2022
@awaelchli awaelchli added this to the pl:1.7.x milestone Sep 13, 2022
@Borda Borda enabled auto-merge (squash) September 13, 2022 10:14
@mergify mergify bot added has conflicts and removed ready PRs ready to be merged labels Sep 13, 2022
auto-merge was automatically disabled September 13, 2022 11:05

Head branch was pushed to by a user without write access

@mergify mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels Sep 13, 2022
@Borda Borda merged commit bc9dd1f into Lightning-AI:master Sep 13, 2022
@jerome-habana jerome-habana deleted the hpu_graph_break branch September 14, 2022 01:58
kaushikb11 added a commit that referenced this pull request Sep 21, 2022
@jerome-habana @pre-commit-ci
@Borda @kaushikb11
Break hpu graphs into two for better performance (#14656)
9b7c6fa 
Signed-off-by: Jerome <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: Kaushik B <[email protected]>
lexierule pushed a commit that referenced this pull request Sep 22, 2022
@jerome-habana @pre-commit-ci
@Borda @kaushikb11 @lexierule
Break hpu graphs into two for better performance (#14656)
87f01ca 
Signed-off-by: Jerome <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: Kaushik B <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kaushikb11 kaushikb11 kaushikb11 approved these changes

@carmocca carmocca carmocca approved these changes

@justusschock justusschock justusschock approved these changes

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

@awaelchli awaelchli Awaiting requested review from awaelchli

@Borda Borda Awaiting requested review from Borda Borda is a code owner

@rohitgr7 rohitgr7 Awaiting requested review from rohitgr7

@otaj otaj Awaiting requested review from otaj

Assignees

@kaushikb11 kaushikb11

Labels
performance pl Generic label for PyTorch Lightning package ready PRs ready to be merged strategy: hpu (external) HPUParallel
Projects
None yet
Milestone
pl:1.7.x
Development

Successfully merging this pull request may close these issues.
6 participants
@jerome-habana @carmocca @justusschock @kaushikb11 @awaelchli @Borda