Skip to content

Joint Deep Neural Networks for Simultaneous Object and Depth Detection

License

Notifications You must be signed in to change notification settings

AminAlam/Deep-Object

Repository files navigation

Deep-Object

What is this?

This project implements two concatenated deep neural networks which perform joint object detection and depth estimation. We used Mask R-CNN for object detection network, and MiDaS network for depth estimation. This project was done for EE-SUT deep learning course.

Dependencies

You can install the dependencies by running: pip install -r requirements.txt

Training

Networks Inintial Weights

Both of the networks(object detector and depth estimator) were pretrained on large dataseta. We fine-tuned them using [NYU Depth Dataset V2](NYU Depth Dataset V2) dataset.

Data Augmentatoin

We used following transforms as augmentation:

  • Random crop
  • Random horizontal flip
  • Brightness change
  • Random affine rotation (5 to 10 degree)

Loss and Metric Plots

Object Detector

Training

Training Loss

Validation

Validation Accuracy

Depth Estimator

Training

Training Loss

Validation

Validation Accuracy

Joint Object Detection and Depth Estimation BELU

BELU-1 BELU-2
BELU-3 BELU-4

Testing

You can see some test images in the following plots:

Etimated Depth Detected Objects with Estimated Depth
Img Depth 1 Img 2
Img Depth 1 Img Depth 2

GUI

You can use the networks by GUI. First, install the depenedncies and the run the follosing command from the project main directory: ptyhon3 GUI/main.py. You can load more images to the app and move between the results using the slider placed at the bottom of the app page. Note that you shoud first train the models and save them in the ./Models directory. For saving or loading the models, change the load_model and save_model boolians in the train.ipynb Configs section.

About

Joint Deep Neural Networks for Simultaneous Object and Depth Detection

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published