Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add estimated_stepping_batches property to Trainer #11599

Merged
merged 17 commits into from
Feb 28, 2022
Merged

Add estimated_stepping_batches property to Trainer #11599

merged 17 commits into from
Feb 28, 2022

Conversation

rohitgr7
Copy link
Contributor

@rohitgr7 rohitgr7 commented Jan 24, 2022 •
edited by github-actions bot
Loading

What does this PR do?

Fixes #10760

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

cc @Borda @justusschock @kaushikb11 @awaelchli @ananthsub @ninginthecloud @jjenniferdai @rohitgr7 @akihironitta

@rohitgr7 rohitgr7 added the feature Is an improvement or enhancement label Jan 24, 2022
@rohitgr7 rohitgr7 added this to the 1.6 milestone Jan 24, 2022
@rohitgr7 rohitgr7 mentioned this pull request Jan 24, 2022
Closed
@SeanNaren
Copy link
Contributor

Overall looks clean to me!

@cowwoc cowwoc mentioned this pull request Feb 17, 2022
Closed
@carmocca
Copy link
Contributor

@rohitgr7 Is this waiting on something?

@rohitgr7
Copy link
Contributor Author

hey @carmocca !
just need to add 1 test for TPU and IPU each. will try to get it done today.

@rohitgr7 rohitgr7 marked this pull request as ready for review February 21, 2022 19:45
@rohitgr7 rohitgr7 requested a review from awaelchli February 22, 2022 11:13
carmocca
carmocca approved these changes Feb 23, 2022
View reviewed changes
pytorch_lightning/trainer/trainer.py Outdated Show resolved Hide resolved
tests/trainer/properties/test_estimated_num_training_steps.py Outdated Show resolved Hide resolved
@rohitgr7 rohitgr7 changed the title Add estimated num training steps Add estimated num optimization steps Feb 24, 2022
@mergify mergify bot added ready PRs ready to be merged and removed has conflicts labels Feb 24, 2022
@kaushikb11 kaushikb11 enabled auto-merge (squash) February 25, 2022 13:36
@rohitgr7 rohitgr7 disabled auto-merge February 25, 2022 14:23
Borda
Borda reviewed Feb 25, 2022
View reviewed changes
docs/source/common/trainer.rst Outdated Show resolved Hide resolved
pytorch_lightning/trainer/trainer.py Outdated Show resolved Hide resolved
tchaton
tchaton approved these changes Feb 28, 2022
View reviewed changes
Copy link
Contributor

@tchaton tchaton left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome ! @rohitgr7 Might be worth considering to introduce an estimated_progress too.

@rohitgr7 rohitgr7 changed the title Add estimated num optimization steps Add estimated_stepping_batches property to Trainer Feb 28, 2022
@rohitgr7 rohitgr7 enabled auto-merge (squash) February 28, 2022 11:33
@rohitgr7 rohitgr7 merged commit 17bb815 into master Feb 28, 2022
@rohitgr7 rohitgr7 deleted the feat/num_training_steps branch February 28, 2022 12:40
Reason239
Reason239 reviewed Nov 24, 2022
View reviewed changes
docs/source/common/optimizers.rst
scheduler = torch.optim.lr_scheduler.OneCycleLR(
optimizer, max_lr=1e-3, total_steps=self.trainer.estimated_stepping_batches
)
return [optimizer], [scheduler]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi!

I think this code example may be misleading, bacause (if I get it right) the default scheduler interval is 'epoch', so here the scheduler would only update every epoch (and wouldn't complete the cycle, because the total_steps is set to estimated_stepping_batches, not number of epochs). I tested it in Colab

Should it be like this?

    return [optimizer], [{"scheduler": scheduler, "interval": "step"}]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me. Would you like to send a PR updating this example?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Reason239 Reason239 Reason239 left review comments

@kaushikb11 kaushikb11 kaushikb11 approved these changes

@carmocca carmocca carmocca approved these changes

@Borda Borda Borda left review comments

@awaelchli awaelchli awaelchli approved these changes

@ananthsub ananthsub ananthsub left review comments

@tchaton tchaton tchaton approved these changes

@williamFalcon williamFalcon Awaiting requested review from williamFalcon

@SeanNaren SeanNaren Awaiting requested review from SeanNaren

@justusschock justusschock Awaiting requested review from justusschock justusschock is a code owner

@edenlightning edenlightning Awaiting requested review from edenlightning

Assignees

@rohitgr7 rohitgr7

Labels
feature Is an improvement or enhancement ready PRs ready to be merged trainer
Projects
None yet
Milestone
pl:1.6
Development

Successfully merging this pull request may close these issues.

Add total training steps as a property to trainer
9 participants
@rohitgr7 @SeanNaren @carmocca @ananthsub @awaelchli @Borda @tchaton @kaushikb11 @Reason239