Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds more robust timer duration parsing #19513

Merged
merged 18 commits into from
Feb 24, 2024

Conversation

kylebgorman
Copy link
Contributor

@kylebgorman kylebgorman commented Feb 22, 2024

What does this PR do?

Failure to provide the timer duration string (e.g., via --max_time) in the appropriate format currently results in an uninformative IndexError. This uses a simple regex (and the built-in re) for more robust input validation and instead throws a misconfiguration exception.

I experimented with datetime.strptime for this, but it is not well-suited to the job because it is designed for absolute times, not deltas, and therefore does not support the idea of "0 days".

Fixes #19454.

I am submitting as is, but reviewers, feel free to recommend testing.

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--19513.org.readthedocs.build/en/19513/

Failure to provide the timer duration string (e.g., via --max_time) in
the sensible format currently results in an uninformative IndexError.
This uses a simple regex for more robust input validation and instead
throws a misconfiguration exception.

I experimented with `datetime.strptime` for this, but it is not
well-suited to the job because it is not designed for absolute times,
not deltas, and therefore does not support the idea of "0 days".

Closes Lightning-AI#19454.
@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Feb 22, 2024
Copy link
Contributor

@awaelchli awaelchli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kylebgorman This is already great!

Would you like to try adding a unit test for this?
It would go into the file tests/tests_pytorch/callbacks/test_timer.py. You can use

with pytest.raises(Misconfiguration, match="Expected DD:HH:MM:SS format"):
    Timer(...)

to test your code.

Otherwise I'm happy to help or do it.

@awaelchli awaelchli added feature Is an improvement or enhancement callback: timer labels Feb 23, 2024
@awaelchli awaelchli added this to the 2.3 milestone Feb 23, 2024
@awaelchli awaelchli added the community This PR is from the community label Feb 23, 2024
@kylebgorman
Copy link
Contributor Author

I greatly appreciate the suggestions, particularly about wording of the exception. I'll add a unit test shortly.

@kylebgorman kylebgorman requested a review from Borda as a code owner February 23, 2024 00:38
Copy link
Contributor

@awaelchli awaelchli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's great, thanks a lot for adding the test!
Feel free to also add an entry in the "Added" section of the changelog in src/lightning/pytorch/CHANGELOG.md

@kylebgorman kylebgorman changed the title Adds more robust timer interval parsing Adds more robust timer duration parsing Feb 23, 2024
@kylebgorman
Copy link
Contributor Author

That's great, thanks a lot for adding the test! Feel free to also add an entry in the "Added" section of the changelog in src/lightning/pytorch/CHANGELOG.md

Done.

@mergify mergify bot added the ready PRs ready to be merged label Feb 23, 2024
Copy link

codecov bot commented Feb 23, 2024

Codecov Report

Merging #19513 (f478d73) into master (0f4522c) will decrease coverage by 35%.
The diff coverage is 100%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #19513      +/-   ##
==========================================
- Coverage      84%      48%     -35%     
==========================================
  Files         450      442       -8     
  Lines       38264    38113     -151     
==========================================
- Hits        31996    18444   -13552     
- Misses       6268    19669   +13401     

@awaelchli awaelchli merged commit 63188f9 into Lightning-AI:master Feb 24, 2024
87 of 88 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
callback: timer community This PR is from the community feature Is an improvement or enhancement pl Generic label for PyTorch Lightning package ready PRs ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Input validation for Trainer's min_time and max_time when passed as string
3 participants