added initial semantic segmentation example #751

akshaykulkarni07 · 2020-01-25T14:20:46Z

Before submitting

Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
Did you read the contributor guideline?
Did you make sure to update the docs?
Did you write any new necessary tests?

What does this PR do?

Fixes #635 (Semantic Segmentation example).

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Some additional info

I tried to add a test_step in the LightningModule which logs the predicted segmentation output. Since the default logger is TensorBoard, I expected the images to show up when I use tensorboard logdir=lightning_logs (at the default save path). But, they don't show up. I couldn't find any example which uses the logger properly. So, if someone can point me to an example, then I'll finish that (right now it's commented out).

Also, I have kind of followed the GAN example as a template, so I have put all of the code in a single file. If you prefer, then I can separate the Dataset class into another file. And I have added this as a full_example, so please let me know if you want it in domain_examples or something.

Also, I have used the KITTI dataset (since it is small and enough for an example) but it is not available directly in torchvision. So, it needs to be downloaded separately and the path needs to be given in the hparams. Note that the segmentation datasets available in torchvision (like Pascal VOC) are too large for a simple example.

Also, I couldn't figure out what documentation or tests are required, because none of the other examples have any documentation or tests (that I could find). Please point out if anything is required.

williamFalcon · 2020-02-12T16:22:32Z

@akshaykvnit awesome. can you get the tests to pass and update the docs?

Borda

Thx for this example, and apology for the delay...
Could you pls have look at comments and

get the tests to pass
add general docstring t the file root with reference to resources

pl_examples/full_examples/semantic_segmentation/semseg.py

Borda · 2020-02-13T22:36:01Z

@williamFalcon a bit more general thought is it would be better to write examples as ipython notebooks which we can transform to nice pages in Docs even with images as I did here

akshaykulkarni07 · 2020-02-14T17:34:03Z

@Borda @williamFalcon I have made most of the changes suggested by @Borda in the last review (with the exception of inference usage) and got the tests to pass.

I haven't been able to implement the inference usage properly. That will take some more time.

@williamFalcon a bit more general thought is it would be better to write examples as ipython notebooks which we can transform to nice pages in Docs even with images as I did here

I had initially implemented this in a Jupyter Notebook, but then the usage of hparams (using argparse and taking input from command line) won't be possible as they have been used in all your examples. So, if we need to convert the examples to iPython notebooks, then I could open a new PR, but do mention what can be done about the hparams.

Borda · 2020-02-14T19:49:39Z

I haven't been able to implement the inference usage properly. That will take some more time.

@akshaykvnit do you want to do it here in this PR or would you prefer to make a follow-up PR?

I had initially implemented this in a Jupyter Notebook, but then the usage of hparams (using argparse and taking input from command line) won't be possible as they have been used in all your examples. So, if we need to convert the examples to iPython notebooks, then I could open a new PR, but do mention what can be done about the hparams.

In this case, I have in mind to create a kind of lightning ZOO with the models and their particular trainers and then having notebook would be nice as an extra example with hparams can be simple derived without duplicating extra code... just building from prefabricated bricks... @williamFalcon ?

akshaykulkarni07 · 2020-02-15T07:47:00Z

@akshaykvnit do you want to do it here in this PR or would you prefer to make a follow-up PR?

Yes, I'll make a follow-up PR.

pl_examples/full_examples/semantic_segmentation/semseg.py

Borda

Almost there 🚀 pls add documentation to the classes (better also to methods) as we are generating Docs from all examples...

pl_examples/full_examples/semantic_segmentation/semseg.py

Borda · 2020-02-15T23:19:54Z

pl_examples/full_examples/semantic_segmentation/semseg.py

+
+
+class SegModel(pl.LightningModule):
+    def __init__(self, hparams):


as examples are becoming part of the Docs, we shall add at least basic documentation with a brief description and reference to resources...

I'm not sure what documentation you need here. Because none of the other examples have documentation for the LightningModule class.

I ment someting like:

"""Image segmentation model. This is a basic image segmentation model implemented with Lightning ... References - <link to source / publication>

Done in the latest commit

Borda · 2020-02-15T23:24:12Z

@akshaykvnit do you want to do it here in this PR or would you prefer to make a follow-up PR?

Yes, I'll make a follow-up PR.

btw, just curious how did you verify without inference? just see the validation/test curve?

pep8speaks · 2020-02-16T07:00:53Z

Hello @akshaykvnit! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-02-16 12:52:52 UTC

akshaykulkarni07 · 2020-02-16T08:20:50Z

@akshaykvnit do you want to do it here in this PR or would you prefer to make a follow-up PR?

Yes, I'll make a follow-up PR.

btw, just curious how did you verify without inference? just see the validation/test curve?

@Borda I implemented the reference using normal PyTorch and it works. You can see the results here in the last block of the notebook. I added the same code after trainer.fit(model) and it works properly. Thus, we can be sure that the training works.

However, I figured that you needed inference to work when trainer.test(model) is called. I haven't managed to properly implement in this fashion (some problem with the test_step and test_end functions). However, I shall try to fix that on my own (and open a new PR when done or open an issue if stuck).

Borda · 2020-02-16T09:06:26Z

@williamFalcon how about the docs for examples? I would say it shall contain them as it is part of Docs but probably it is not a blocker... :]

williamFalcon · 2020-02-16T15:53:00Z

@akshaykvnit awesome tutorial!

williamFalcon · 2020-02-17T02:22:31Z

@akshaykvnit @Borda

 ERROR collecting pl_examples/full_examples/semantic_segmentation/semseg.py

 No module named 'torchvision.models.segmentation'

oops, looks like this test fails. Need to fix asap

Borda · 2020-02-17T06:02:14Z

Yeah I have noticed that almost all examples are not tested at all... We shall run them with a minimal data....

akshaykulkarni07 added 2 commits January 25, 2020 19:29

added initial semantic segmentation example

1b856cd

removed unnecessary lines.

c2bd710

akshaykulkarni07 requested review from Borda and williamFalcon January 25, 2020 14:49

Borda added the feature Is an improvement or enhancement label Jan 31, 2020

Borda added this to the 0.6.1 milestone Jan 31, 2020

Borda requested changes Feb 13, 2020

View reviewed changes

akshaykulkarni07 added 2 commits February 14, 2020 22:40

changed according to reviews

a7889d4

minor changes

a2ada02

Borda reviewed Feb 15, 2020

View reviewed changes

pl_examples/full_examples/semantic_segmentation/semseg.py Outdated Show resolved Hide resolved

Borda requested changes Feb 15, 2020

View reviewed changes

Added some documentation for Dataset class

aec1d02

Fixed some long lines

7adfdff

added docstring for LightningModule

e615f6d

williamFalcon merged commit 11db9d2 into Lightning-AI:master Feb 16, 2020

four4fish mentioned this pull request Feb 4, 2022

Rewrite accelerator_connector #11448

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added initial semantic segmentation example #751

added initial semantic segmentation example #751

akshaykulkarni07 commented Jan 25, 2020

williamFalcon commented Feb 12, 2020

Borda left a comment

Borda commented Feb 13, 2020

akshaykulkarni07 commented Feb 14, 2020

Borda commented Feb 14, 2020 •

edited

Loading

akshaykulkarni07 commented Feb 15, 2020

Borda left a comment

Borda Feb 15, 2020

akshaykulkarni07 Feb 16, 2020

Borda Feb 16, 2020

akshaykulkarni07 Feb 16, 2020

Borda commented Feb 15, 2020

pep8speaks commented Feb 16, 2020 •

edited

Loading

akshaykulkarni07 commented Feb 16, 2020 •

edited

Loading

Borda commented Feb 16, 2020

williamFalcon commented Feb 16, 2020

williamFalcon commented Feb 17, 2020 •

edited

Loading

Borda commented Feb 17, 2020



		class SegModel(pl.LightningModule):
		def __init__(self, hparams):

added initial semantic segmentation example #751

added initial semantic segmentation example #751

Conversation

akshaykulkarni07 commented Jan 25, 2020

Before submitting

What does this PR do?

PR review

Some additional info

williamFalcon commented Feb 12, 2020

Borda left a comment

Choose a reason for hiding this comment

Borda commented Feb 13, 2020

akshaykulkarni07 commented Feb 14, 2020

Borda commented Feb 14, 2020 • edited Loading

akshaykulkarni07 commented Feb 15, 2020

Borda left a comment

Choose a reason for hiding this comment

Borda Feb 15, 2020

Choose a reason for hiding this comment

akshaykulkarni07 Feb 16, 2020

Choose a reason for hiding this comment

Borda Feb 16, 2020

Choose a reason for hiding this comment

akshaykulkarni07 Feb 16, 2020

Choose a reason for hiding this comment

Borda commented Feb 15, 2020

pep8speaks commented Feb 16, 2020 • edited Loading

Comment last updated at 2020-02-16 12:52:52 UTC

akshaykulkarni07 commented Feb 16, 2020 • edited Loading

Borda commented Feb 16, 2020

williamFalcon commented Feb 16, 2020

williamFalcon commented Feb 17, 2020 • edited Loading

Borda commented Feb 17, 2020

Borda commented Feb 14, 2020 •

edited

Loading

pep8speaks commented Feb 16, 2020 •

edited

Loading

akshaykulkarni07 commented Feb 16, 2020 •

edited

Loading

williamFalcon commented Feb 17, 2020 •

edited

Loading