Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Vision DataModules #400

Merged

Conversation

chris-clem
Copy link
Contributor

@chris-clem chris-clem commented Nov 24, 2020

What does this PR do?

As proposed in #395, this PR creates a BaseDataModule, that can be used as the parent class for several existing DataModules like BinaryMNISTDataModule, CIFAR10DataModule, FashionMNISTDataModule, and MNISTDataModule.

This removes duplicate code in the DataModules and thus makes maintaining them easier.

Fixes #395

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Nov 24, 2020

Codecov Report

Merging #400 (4cf20b6) into master (347a63d) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #400   +/-   ##
=======================================
  Coverage   80.40%   80.40%           
=======================================
  Files         101      101           
  Lines        5685     5685           
=======================================
  Hits         4571     4571           
  Misses       1114     1114           
Flag Coverage Δ
cpu 24.98% <ø> (ø)
pytest 24.98% <ø> (ø)
unittests 79.96% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 347a63d...4cf20b6. Read the comment docs.

@Borda Borda requested a review from nateraw November 24, 2020 13:46
@ananyahjha93 ananyahjha93 self-requested a review November 24, 2020 16:40
Copy link
Contributor

@nateraw nateraw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work 😄 . couple notes.

num_workers: int = 16,
normalize: bool = False,
seed: int = 42,
batch_size: int = 32,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of eval_batch_size and train_batch_size being separate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would break a lot of tests. Should I still do it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this change to separate PR?
in theory, how much the train and valid batch size can have an effect?

pl_bolts/datamodules/base_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/base_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/base_datamodule.py Outdated Show resolved Hide resolved
@chris-clem chris-clem requested a review from nateraw November 25, 2020 15:38
@akihironitta akihironitta added datamodule Anything related to datamodules refactoring labels Nov 25, 2020
@Borda
Copy link
Member

Borda commented Nov 26, 2020

@ananyahjha93 @nateraw is it ready to go?

@chris-clem
Copy link
Contributor Author

Any updates?

@Borda Borda requested a review from akihironitta December 5, 2020 00:59
Copy link
Contributor

@akihironitta akihironitta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chris-clem It looks good to me.
@nateraw @ananyahjha93 Can you also have a look?

@Borda
Copy link
Member

Borda commented Dec 8, 2020

@chris-clem can you pls resolve conflits?

@chris-clem chris-clem force-pushed the feature/395_refactor-vision-dms branch from eb01b40 to b17a5d8 Compare December 9, 2020 15:31
@chris-clem
Copy link
Contributor Author

@chris-clem can you pls resolve conflits?

I just did it

@chris-clem chris-clem requested a review from Borda December 10, 2020 08:25
@Borda
Copy link
Member

Borda commented Dec 14, 2020

@chris-clem we have merged better importing with if/else instead of try/except mind pls update the PR..
sorry for the inconvenience, but I think it shall land soon :]

Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we pls also add typing?

pl_bolts/datamodules/base_datamodule.py Outdated Show resolved Hide resolved
@Borda Borda self-assigned this Dec 14, 2020
@chris-clem
Copy link
Contributor Author

@chris-clem we have merged better importing with if/else instead of try/except mind pls update the PR..
sorry for the inconvenience, but I think it shall land soon :]

No problem, I updated the PR

@chris-clem chris-clem requested a review from Borda December 15, 2020 16:07
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will implement some other comments as direct commits here

pl_bolts/datamodules/base_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/base_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/binary_mnist_datamodule.py Outdated Show resolved Hide resolved
@Borda
Copy link
Member

Borda commented Dec 16, 2020

@chris-clem @akihironitta I have made some more suggestions and rebased on master, mind check it...

Copy link
Contributor

@akihironitta akihironitta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chris-clem @Borda The changes basically look great to me! I left some comments, so could you have a look?

pl_bolts/datamodules/binary_mnist_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/binary_mnist_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/binary_mnist_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/vision_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/cifar10_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/vision_datamodule.py Outdated Show resolved Hide resolved
@Borda
Copy link
Member

Borda commented Dec 17, 2020

@chris-clem mind resolve conflicts and #400 (comment)

CHANGELOG.md Outdated Show resolved Hide resolved
chris-clem and others added 5 commits December 17, 2020 18:56
# Conflicts:
#	pl_bolts/datamodules/binary_mnist_datamodule.py
#	pl_bolts/datamodules/cifar10_datamodule.py
#	pl_bolts/datamodules/fashion_mnist_datamodule.py
#	pl_bolts/datamodules/mnist_datamodule.py
… into feature/395_refactor-vision-dms

# Conflicts:
#	pl_bolts/datamodules/cifar10_datamodule.py
#	pl_bolts/datamodules/fashion_mnist_datamodule.py
#	pl_bolts/datamodules/mnist_datamodule.py
pl_bolts/datamodules/cifar10_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/vision_datamodule.py Outdated Show resolved Hide resolved
pl_bolts/datamodules/vision_datamodule.py Outdated Show resolved Hide resolved
Copy link
Contributor

@akihironitta akihironitta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chris-clem It looks valid to me :]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datamodule Anything related to datamodules refactoring
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor Vision DataModules
4 participants