feat: Add Detection Task #56

kaushikb11 · 2021-02-02T18:28:35Z

What does this PR do?

Add support for detection

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests? [not needed for typos/docs]
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

codecov · 2021-02-02T18:33:57Z

Codecov Report

Merging #56 (13f5106) into master (7acd8b5) will decrease coverage by 0.09%.
The diff coverage is 84.75%.

@@            Coverage Diff             @@
##           master      #56      +/-   ##
==========================================
- Coverage   86.66%   86.56%   -0.10%     
==========================================
  Files          45       47       +2     
  Lines        1372     1526     +154     
==========================================
+ Hits         1189     1321     +132     
- Misses        183      205      +22

Flag	Coverage Δ
unittests	`86.56% <84.75%> (-0.10%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
flash/core/finetuning.py	`88.46% <ø> (ø)`
flash/vision/detection/model.py	`67.27% <50.00%> (-32.73%)`	⬇️
flash/vision/detection/data.py	`94.06% <94.06%> (ø)`
flash/vision/__init__.py	`100.00% <100.00%> (ø)`
flash/vision/detection/__init__.py	`100.00% <100.00%> (ø)`
flash/vision/detection/finetuning.py	`100.00% <100.00%> (ø)`
flash/vision/classification/data.py	`77.77% <0.00%> (-0.35%)`	⬇️
flash/core/data/datapipeline.py	`100.00% <0.00%> (+3.44%)`	⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7acd8b5...3330387. Read the comment docs.

flash/vision/detection/data.py

teddykoker · 2021-02-02T19:21:28Z

Great work! do you know if there are any small datasets we could use to test?

kaushikb11 · 2021-02-03T04:32:46Z

Great work! do you know if there are any small datasets we could use to test?

I found a toy COCO format dataset link. The author uses it in this repo.

zhiqwang · 2021-02-03T06:08:46Z

Great work! do you know if there are any small datasets we could use to test?

I found a toy COCO format dataset link. The author uses it in this repo.

Hi, ultralytics provided a small coco datasets named coco128 here (It is a subset of COCO Datasets), they are using DarkNet like format, but I've converted it to COCO format, if you are interested on this, I can upload the converted COCO file.

EDIT: ~~I've uploaded the COCO label here.~~ (Check the file instead in https://github.com/zhiqwang/yolov5-rt-stack/releases/download/v0.3.0/coco128.zip)

kaushikb11 · 2021-02-03T06:35:26Z

@zhiqwang Thank you, will look into it!

zhiqwang · 2021-02-03T07:09:11Z

Hi @kaushikb11, I upload the whole folder containing images here for easy of use. The COCO format file are also in this folder.

flash/vision/detection/model.py

kaushikb11 · 2021-02-03T15:59:32Z

Hi @kaushikb11, I upload the whole folder containing images here for easy of use. The COCO format file are also in this folder.

@zhiqwang Could you update file_name in images section from path(images/train2017/000000000612.jpg) to filename(000000000612.jpg)?

zhiqwang · 2021-02-04T02:54:44Z

Hi @kaushikb11 okay, I'll reupload the label file soon.

EDIT: Done, you can redownload the https://github.com/zhiqwang/yolov5-rt-stack/releases/download/v0.3.0/coco128.zip.

kaushikb11 · 2021-02-04T06:08:25Z

@zhiqwang: The annotations don't seem to be right. The categories are out of bounds. FOr eg., you could check id 12 in the annotation section which has category_id 86.

zhiqwang · 2021-02-04T06:09:56Z

Hi @kaushikb11, Sorry, and I'll double check it!

zhiqwang · 2021-02-04T06:21:13Z

Hi @kaushikb11

Indeed, COCO only has 80 classes, but they are not contiguous in their class indices. Indeed, as you can see from the colab notebook (which contains the class names), there are many N/A classes

CLASSES = [
    'N/A', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
    'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'N/A',
    'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse',
    'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'N/A', 'backpack',
    'umbrella', 'N/A', 'N/A', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis',
    'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove',
    'skateboard', 'surfboard', 'tennis racket', 'bottle', 'N/A', 'wine glass',
    'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich',
    'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake',
    'chair', 'couch', 'potted plant', 'bed', 'N/A', 'dining table', 'N/A',
    'N/A', 'toilet', 'N/A', 'tv', 'laptop', 'mouse', 'remote', 'keyboard',
    'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'N/A',
    'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier',
    'toothbrush'
]

So the total number of classes is 91 here.

REF: facebookresearch/detr#23 (comment)

zhiqwang · 2021-02-04T06:28:23Z

Hi @kaushikb11

One option here to solve the category inconsistent problem is through the category mapping like:

json_category_id_to_contiguous_id = {
    v: i for i, v in enumerate(example_coco.getCatIds())
}

Thoughts?

kaushikb11 · 2021-02-04T08:45:54Z

@zhiqwang I have handled the case, we wouldn’t want to change the category mapping. Thanks!

teddykoker

LGTM, just one question. Do you think we could add some tests here? Would be great to get code coverage > 90%

flash/vision/detection/finetuning.py

flash_examples/finetuning/image_detection.py

flash/vision/detection/data.py

tchaton

Awesome work there ! You aced it !

flash/vision/detection/data.py

flash/vision/detection/model.py

flash_examples/finetuning/image_detection.py

tests/vision/detection/test_data.py

flash/vision/detection/data.py

flash/vision/detection/dataset.py

flash/vision/detection/finetuning.py

requirements.txt

justusschock

@kaushikb11 looking good already. just pointing out some things hat should be adressed. The collate thing is the only blocker from my side :)

flash/vision/detection/data.py

carmocca · 2021-02-10T14:54:55Z

flash/vision/detection/data.py

+    def collate(self, samples: Any) -> Any:
+        if not isinstance(samples, Tensor):
+            elem = samples[0]
+            if isinstance(elem, container_abcs.Sequence):


Shoudn't you use typing.Sequence?

edenlightning · 2021-02-10T20:05:45Z

Nice work! Can you please add docs here too?

kaushikb11 · 2021-02-10T20:17:32Z

Nice work! Can you please add docs here too?

Where should we add it? Also will do a follow up PR for docs. Better for review.

Co-authored-by: Carlos Mocholí <[email protected]>

williamFalcon · 2021-02-12T13:20:35Z

@kaushikb11 @Borda @edenlightning can we add docs for this?

kaushikb11 · 2021-02-12T14:02:25Z

@williamFalcon On it!

kaushikb11 changed the title ~~Add DataModules for detection~~ Add support for detection Feb 2, 2021

kaushikb11 added the task flash Task label Feb 2, 2021

teddykoker reviewed Feb 2, 2021

View reviewed changes

flash/vision/detection/data.py Outdated Show resolved Hide resolved

adriantre reviewed Feb 3, 2021

View reviewed changes

flash/vision/detection/model.py Show resolved Hide resolved

kaushikb11 marked this pull request as ready for review February 4, 2021 13:40

kaushikb11 requested review from awaelchli, Borda, carmocca, justusschock and tchaton as code owners February 4, 2021 13:40

teddykoker approved these changes Feb 4, 2021

View reviewed changes

flash/vision/detection/finetuning.py Show resolved Hide resolved

carmocca reviewed Feb 4, 2021

View reviewed changes

flash_examples/finetuning/image_detection.py Outdated Show resolved Hide resolved

kaushikb11 force-pushed the feat/detect branch from 5204570 to 4193afb Compare February 5, 2021 06:36

kaushikb11 added 4 commits February 5, 2021 12:14

add CustomCOCODataset for detection

ce852c8

add steps for the detection task

a8c6315

update steps for detector task

3db086d

add data pipeline

837417a

justusschock reviewed Feb 5, 2021

View reviewed changes

flash/vision/detection/data.py Show resolved Hide resolved

flash/vision/detection/data.py Show resolved Hide resolved

tchaton reviewed Feb 5, 2021

View reviewed changes

kaushikb11 added 2 commits February 5, 2021 18:10

update dummy image shapes

a80b569

add save hyperparameters

9a15863

zhiqwang reviewed Feb 7, 2021

View reviewed changes

flash/vision/detection/data.py Show resolved Hide resolved

fix labels assignment

ff52ffc

carmocca reviewed Feb 8, 2021

View reviewed changes

kaushikb11 added 7 commits February 8, 2021 14:11

add singular data pipeline

fa46955

add updates for predict

6fb5935

add cython to requirements

cef3f45

add cython to requirements

71ebf17

fix failing tests

720b1cc

use download_data with coco128 url

84c5dec

update imports

068c3a4

justusschock suggested changes Feb 9, 2021

View reviewed changes

flash/vision/detection/data.py Show resolved Hide resolved

flash/vision/detection/data.py Show resolved Hide resolved

flash/vision/detection/data.py Outdated Show resolved Hide resolved

flash/vision/detection/data.py Show resolved Hide resolved

kaushikb11 changed the title ~~Add support for detection~~ feat: Add Detection Task Feb 9, 2021

handle degenerated boxes

cbcc2b1

justusschock approved these changes Feb 10, 2021

View reviewed changes

kaushikb11 enabled auto-merge (squash) February 10, 2021 14:19

carmocca reviewed Feb 10, 2021

View reviewed changes

flash/vision/detection/data.py Outdated Show resolved Hide resolved

carmocca reviewed Feb 10, 2021

View reviewed changes

carmocca approved these changes Feb 10, 2021

View reviewed changes

Borda and others added 2 commits February 10, 2021 21:28

Apply suggestions from code review

9f97160

Co-authored-by: Carlos Mocholí <[email protected]>

Merge branch 'master' into feat/detect

3330387

Borda disabled auto-merge February 10, 2021 20:28

Borda merged commit c60fef8 into Lightning-Universe:master Feb 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add Detection Task #56

feat: Add Detection Task #56

kaushikb11 commented Feb 2, 2021 •

edited

Loading

codecov bot commented Feb 2, 2021 •

edited

Loading

teddykoker commented Feb 2, 2021

kaushikb11 commented Feb 3, 2021

zhiqwang commented Feb 3, 2021 •

edited

Loading

kaushikb11 commented Feb 3, 2021

zhiqwang commented Feb 3, 2021

kaushikb11 commented Feb 3, 2021 •

edited

Loading

zhiqwang commented Feb 4, 2021 •

edited

Loading

kaushikb11 commented Feb 4, 2021

zhiqwang commented Feb 4, 2021 •

edited

Loading

zhiqwang commented Feb 4, 2021

zhiqwang commented Feb 4, 2021 •

edited

Loading

kaushikb11 commented Feb 4, 2021

teddykoker left a comment

tchaton left a comment

justusschock left a comment

carmocca Feb 10, 2021

edenlightning commented Feb 10, 2021

kaushikb11 commented Feb 10, 2021

williamFalcon commented Feb 12, 2021

kaushikb11 commented Feb 12, 2021

feat: Add Detection Task #56

feat: Add Detection Task #56

Conversation

kaushikb11 commented Feb 2, 2021 • edited Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

codecov bot commented Feb 2, 2021 • edited Loading

Codecov Report

teddykoker commented Feb 2, 2021

kaushikb11 commented Feb 3, 2021

zhiqwang commented Feb 3, 2021 • edited Loading

kaushikb11 commented Feb 3, 2021

zhiqwang commented Feb 3, 2021

kaushikb11 commented Feb 3, 2021 • edited Loading

zhiqwang commented Feb 4, 2021 • edited Loading

kaushikb11 commented Feb 4, 2021

zhiqwang commented Feb 4, 2021 • edited Loading

zhiqwang commented Feb 4, 2021

zhiqwang commented Feb 4, 2021 • edited Loading

kaushikb11 commented Feb 4, 2021

teddykoker left a comment

Choose a reason for hiding this comment

tchaton left a comment

Choose a reason for hiding this comment

justusschock left a comment

Choose a reason for hiding this comment

carmocca Feb 10, 2021

Choose a reason for hiding this comment

edenlightning commented Feb 10, 2021

kaushikb11 commented Feb 10, 2021

williamFalcon commented Feb 12, 2021

kaushikb11 commented Feb 12, 2021

kaushikb11 commented Feb 2, 2021 •

edited

Loading

codecov bot commented Feb 2, 2021 •

edited

Loading

zhiqwang commented Feb 3, 2021 •

edited

Loading

kaushikb11 commented Feb 3, 2021 •

edited

Loading

zhiqwang commented Feb 4, 2021 •

edited

Loading

zhiqwang commented Feb 4, 2021 •

edited

Loading

zhiqwang commented Feb 4, 2021 •

edited

Loading