Pytorch video #216

tchaton · 2021-04-15T09:21:02Z

What does this PR do?

Fixes # (issue)

This PR integrates PyTorchVideo to Lightning Flash: https://pytorchvideo.org

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests? [not needed for typos/docs]
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

kaushikb11 · 2021-04-15T09:29:55Z

flash/vision/video/classification/model.py

+        if not model_kwargs:
+            model_kwargs = {}
+
+        model_kwargs["pretrained"] = pretrained


Need to add a check if the User is passing pretrained in model_kwargs, or else the default will be passed to the Model.

kaushikb11 · 2021-04-15T09:33:35Z

flash/vision/video/classification/model.py

+
+        if isinstance(model, nn.Module):
+            self.model = model
+        elif isinstance(model, str):


Kindly add a check for _PYTORCH_VIDEO_AVAILABLE here, or else self.models will be an empty FlashRegistry.

flash/vision/video/classification/model.py

flash/vision/video/classification/data.py

flash_examples/finetuning/video_classification.py

flash/vision/video/classification/data.py

…ightning-flash into pytorch_video

Co-authored-by: Kaushik B <[email protected]>

codecov · 2021-04-15T10:18:37Z

Codecov Report

Merging #216 (43e2bc3) into master (2b39eae) will decrease coverage by 0.01%.
The diff coverage is 80.78%.

@@            Coverage Diff             @@
##           master     #216      +/-   ##
==========================================
- Coverage   87.38%   87.37%   -0.02%     
==========================================
  Files          59       62       +3     
  Lines        3123     3294     +171     
==========================================
+ Hits         2729     2878     +149     
- Misses        394      416      +22

Flag	Coverage Δ
unittests	`87.37% <80.78%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
flash/core/classification.py	`92.85% <ø> (-0.11%)`	⬇️
flash/text/classification/data.py	`83.49% <ø> (+1.94%)`	⬆️
flash/text/seq2seq/translation/data.py	`90.00% <ø> (ø)`
flash/text/seq2seq/translation/model.py	`72.22% <ø> (ø)`
flash/text/seq2seq/summarization/metric.py	`96.55% <33.33%> (ø)`
flash/core/model.py	`90.77% <60.00%> (-0.03%)`	⬇️
flash/video/classification/data.py	`77.52% <77.52%> (ø)`
flash/text/seq2seq/core/data.py	`80.26% <78.94%> (-0.39%)`	⬇️
flash/video/classification/model.py	`87.50% <87.50%> (ø)`
flash/core/finetuning.py	`88.88% <100.00%> (+0.20%)`	⬆️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b39eae...43e2bc3. Read the comment docs.

…ightning-flash into pytorch_video

flash/vision/video/classification/data.py

tullie · 2021-04-15T19:51:48Z

flash/vision/video/classification/data.py

+            decoder=self.decoder,
+        )
+        if self.training:
+            dataset.num_classes = len(np.unique([s[1]['label'] for s in ds._labeled_videos]))


This is a good little line of code, probably should move it to PTV and just expose a num_classes property (also means we wouldn't need to access the private _labeled_videos property. Will add this as a PTV issue and we can change it later.

flash/vision/video/classification/data.py

tests/video/test_video_classifier.py

pep8speaks · 2021-04-16T08:55:34Z

Hello @tchaton! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-04-30 14:35:36 UTC

ethanwharris

This looks awesome 😍 the examples are super clean and easy to follow! Just some small comments 😃

flash/core/model.py

flash/data/auto_dataset.py

flash/data/data_module.py

flash/data/data_pipeline.py

flash/data/process.py

…ightning-flash into pytorch_video

edgarriba · 2021-04-30T07:57:43Z

flash/video/__init__.py

@@ -0,0 +1,2 @@
+from flash.video.classification.data import VideoClassificationData
+from flash.video.classification.model import VideoClassifier


add the preprocess

tchaton added 2 commits April 15, 2021 10:13

update

ae190b9

update

a4cb3a3

kaushikb11 reviewed Apr 15, 2021

View reviewed changes

flash/vision/video/classification/model.py Outdated Show resolved Hide resolved

kaushikb11 reviewed Apr 15, 2021

View reviewed changes

flash/vision/video/classification/data.py Outdated Show resolved Hide resolved

Update flash/vision/video/classification/data.py

cdba489

edgarriba reviewed Apr 15, 2021

View reviewed changes

flash_examples/finetuning/video_classification.py Outdated Show resolved Hide resolved

flash/vision/video/classification/data.py Outdated Show resolved Hide resolved

flash/vision/video/classification/data.py Outdated Show resolved Hide resolved

tchaton and others added 3 commits April 15, 2021 11:14

update

c6b1cd7

Merge branch 'pytorch_video' of https://github.com/PyTorchLightning/l…

5f1aae9

…ightning-flash into pytorch_video

Update flash/vision/video/classification/model.py

2c427f6

Co-authored-by: Kaushik B <[email protected]>

tchaton added 6 commits April 15, 2021 11:44

update

0c4a092

Merge branch 'pytorch_video' of https://github.com/PyTorchLightning/l…

db6df41

…ightning-flash into pytorch_video

update

19ea5f1

typo

b21f152

update

aea8214

update

fbc43c8

tullie suggested changes Apr 15, 2021

View reviewed changes

resolve some internal bugs

73e0191

tchaton mentioned this pull request Apr 16, 2021

Uniformize Flash API to TorchVision / TorchVideo #218

Closed

tchaton added 4 commits April 16, 2021 10:08

update on comments

a1ff7b6

move files

3227e77

update

98b4e13

update

9e17b50

ethanwharris mentioned this pull request Apr 16, 2021

Add pre_metric_transform hook #219

Closed

8 tasks

ethanwharris reviewed Apr 16, 2021

View reviewed changes

tchaton added 2 commits April 16, 2021 16:48

update

eb286de

filter for 3.6

b122059

tchaton and others added 14 commits April 27, 2021 12:43

Merge branch 'master' into pytorch_video

3c660ef

update

04382a5

update

cf3ef94

Merge branch 'pytorch_video' of https://github.com/PyTorchLightning/l…

9e9b656

…ightning-flash into pytorch_video

update

754f43c

Updates

7a09783

update

6697e91

update

231171a

update

92aa151

update

939a251

iupdate:

63babc6

update

530367d

update

81733ed

Merge branch 'master' into pytorch_video

fdd85a2

tchaton enabled auto-merge (squash) April 29, 2021 19:38

resolve ci

ed043b3

tchaton disabled auto-merge April 29, 2021 20:06

tchaton added 2 commits April 29, 2021 21:15

update

1735b6f

update

5d80e45

edgarriba reviewed Apr 30, 2021

View reviewed changes

tchaton added 8 commits April 30, 2021 10:33

updates

f201a70

update

aff7657

update

b18457a

update

78dbc3a

update

80f7e71

update

1999639

update

3b0bd8f

update

43e2bc3

tchaton merged commit 0263fd1 into master Apr 30, 2021

tchaton deleted the pytorch_video branch April 30, 2021 14:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pytorch video #216

Pytorch video #216

tchaton commented Apr 15, 2021 •

edited

Loading

kaushikb11 Apr 15, 2021

tchaton Apr 15, 2021

kaushikb11 Apr 15, 2021

codecov bot commented Apr 15, 2021 •

edited

Loading

tullie Apr 15, 2021

tchaton Apr 16, 2021

pep8speaks commented Apr 16, 2021 •

edited

Loading

ethanwharris left a comment

edgarriba Apr 30, 2021

		@@ -0,0 +1,2 @@
		from flash.video.classification.data import VideoClassificationData
		from flash.video.classification.model import VideoClassifier

Pytorch video #216

Pytorch video #216

Conversation

tchaton commented Apr 15, 2021 • edited Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

kaushikb11 Apr 15, 2021

Choose a reason for hiding this comment

tchaton Apr 15, 2021

Choose a reason for hiding this comment

kaushikb11 Apr 15, 2021

Choose a reason for hiding this comment

codecov bot commented Apr 15, 2021 • edited Loading

Codecov Report

tullie Apr 15, 2021

Choose a reason for hiding this comment

tchaton Apr 16, 2021

Choose a reason for hiding this comment

pep8speaks commented Apr 16, 2021 • edited Loading

Comment last updated at 2021-04-30 14:35:36 UTC

ethanwharris left a comment

Choose a reason for hiding this comment

edgarriba Apr 30, 2021

Choose a reason for hiding this comment

tchaton commented Apr 15, 2021 •

edited

Loading

codecov bot commented Apr 15, 2021 •

edited

Loading

pep8speaks commented Apr 16, 2021 •

edited

Loading