Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LRFinder callback #13802

Merged
merged 53 commits into from
Oct 5, 2022
Merged

Add LRFinder callback #13802

merged 53 commits into from
Oct 5, 2022

Conversation

rohitgr7
Copy link
Contributor

@rohitgr7 rohitgr7 commented Jul 22, 2022

What does this PR do?

Part of #11012
will add docs and examples in a follow up.

Added LearningRateFinder callback that allows users to run lr_finder at any time they want during .fit call using a callback.

class CustomLearningRateFinder(LearningRateFinder):
    def on_fit_start(self, trainer, pl_module):
        self.lr_find(trainer, pl_module)
    
    def on_train_epoch_start(self, trainer, pl_module):
        if self.trainer.current_epoch == 10:
            self.lr_find(trainer, pl_module)


lr_cb = CustomLearningRateFinder(...)
trainer = Trainer(..., callbacks=lr_cb)
trainer.fit(...)

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

cc @Borda @justusschock @awaelchli @rohitgr7 @akihironitta

@awaelchli
Copy link
Contributor

Great work @rohitgr7
Team (@Borda @carmocca @tchaton @kaushikb11 etc. can we bring these two PRs in? They have been waiting for a long time. I know we are all not well familiar with the tuner, so let's just review as best as we can and let @rohitgr7 finalize the work?

@Borda
Copy link
Member

Borda commented Sep 9, 2022

Great work @rohitgr7 Team (@Borda @carmocca @tchaton @kaushikb11 etc. can we bring these two PRs in? They have been waiting for a long time. I know we are all not well familiar with the tuner, so let's just review as best as we can and let @rohitgr7 finalize the work?

That would be GREAT!

Co-authored-by: Laverne Henderson <[email protected]>
Co-authored-by: Adrian Wälchli <[email protected]>
@Borda Borda requested a review from otaj as a code owner September 12, 2022 19:53
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add docs on how to use this cb? 🦦

@Borda
Copy link
Member

Borda commented Sep 22, 2022

@rohitgr7, mind checking the collisions? 🦦

@mergify mergify bot added ready PRs ready to be merged has conflicts and removed has conflicts ready PRs ready to be merged labels Sep 24, 2022
Base automatically changed from ref/bs_finder_tuner to master September 27, 2022 12:54
@mergify mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels Sep 27, 2022
@Borda Borda merged commit 7fed7a1 into master Oct 5, 2022
@Borda Borda deleted the ref/lr_finder_callback branch October 5, 2022 11:15
nicolai86 pushed a commit that referenced this pull request Oct 13, 2022
* add BatchSizeFinderCallback callback
* enable fast_dev_run test
* keep tune and remove early_exit
* move exception to setup
* Apply suggestions from code review

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Laverne Henderson <[email protected]>
Co-authored-by: Adrian Wälchli <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Is an improvement or enhancement pl Generic label for PyTorch Lightning package ready PRs ready to be merged refactor tuner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[RFC] Tuner Revamp
6 participants