Table of contents

• Overview
• Features
• Dependencies
• Try it out
• Build
• Notes for Linux
• Limitations
• Project screenshots and video snippets
• Multi-language interface
• Drowsiness detection
• Fist gesture
• Palm and V gestures
• Multiple cameras
• Smart interaction with the camera
• Cross-platform

Overview

This project is a thesis work representing a prototype of an intelligent safety system for detecting driver drowsiness using facial video surveillance.

Features

• 🚀 YOLOv5 as an object detector
• 🎮 Interactive interaction with the prototype:
  • Start/stop the recognition system
  • Interaction with a video camera
  • Eye and gesture recognition (V gesture ✌️, Fist gesture ✊, Palm gesture ✋)
  • Warning of a potential emergency situation using warning sound
• 🚀 Recognition (drowsiness, gestures, etc.) takes approximately one second
• ⚙️ Multi-language user interface
• ⚙️ Cross-platform (Windows/Linux)
• ⚙️ Multithreaded application

Dependencies

• C++17:
  • 64-bit MSVC 19.15 and higher on Windows
  • 64-bit GCC 11.2 and higher with Ninja on Linux
• CMake 3.21 and higher
• OpenCV 4.5.4 and higher
• Qt 6.5.3
• NVIDIA drivers
• CUDA with cuDNN (more details about the versions below)

Try it out

For convenience, releases with a fully built project with all the necessary libraries to run are prepared. On Linux, the DDDetector executable is located in the bin folder.

The project releases use CUDA 12.2 with cuDNN v8.9.2 (for Windows) / v8.9.5 (for Linux) for CUDA 12.x (cuDNN libraries are in the release, only CUDA and NVIDIA drivers are mandatory for installation on the computer). For Windows, everything installs in a few clicks, you just need to download and run the installers from the official NVIDIA website. For Linux (the project was tested on Ubuntu 22.04) you can use this instruction or others, also take a look at these notes.

OpenCV libraries have version 4.8.0 in the release with support for the following GPU architectures: 5.0, 5.2, 6.0, 6.1, 7.0, 7.5, 8.0, 8.6, 8.9, 9.0 and require CUDA/cuDNN of the versions mentioned above.

Build

To build and run the project locally, you need to have Git, Python 3.5 and higher, and all necessary dependencies installed on your computer.

Notes for Linux

1. Need binutils and OpenGL packages.
2. On Linux, RUNPATH is set for the executable (libraries are searched for in the lib64 folder at level with bin in the Qt6, opencv, cuda subfolders), so if you want to, you can put framework libraries there, so that the linker will try to find them there first.

From your command line:

# Clone this repository
git clone https://github.com/GTruf/Driver-Drowsiness-Detector.git

# Go into the repository folder
cd Driver-Drowsiness-Detector

# --------------------------
# Build
# Template: python/python3 build.py --qt-cmake-prefix-path=<path-to-qt-cmake> --opencv-dir=<path-to-directory-containing-cmake-config-for-opencv> -j <thread-number>

# Examples:

# On Windows
> python build.py --qt-cmake-prefix-path=C:\Qt\6.5.3\msvc2019_64\lib\cmake --opencv-dir=C:\Development\opencv-4.8.0\build\install -j 8

# On Linux
$ python3 build.py --qt-cmake-prefix-path=~/Qt/6.5.3/gcc_64/lib/cmake --opencv-dir=/usr/local/lib/cmake/opencv4 -j 8

# Go into the build folder after building the project
cd build

# --------------------------
# Run

# Before running, make sure that all dynamic libraries are located 
# where dynamic library loaders can find them when the executable runs

# On Windows, the executable will be in the build folder and can be opened either via cmd or explorer
> DDDetector.exe

# On Linux, the executable will be in the build/bin folder
$ ./bin/DDDetector

Limitations

1. The camera class when selecting the camera device on Windows selects the DSHOW (otherwise MSMF) API, on Linux it selects V4L2. In combination, the MJPEG video compression method is used. The problem is that not all video cameras support such combinations, so in the worst case the number of frames per second from the camera can be very small, in this case you need to choose the right API of the camera together with the right video compression method.
2. The repository uses Git LFS to store some objects, so if you can't get something during the git lfs pull, you can download them from the releases or contact me, and I'll send them to you personally.

Project screenshots and video snippets

Just a reminder that you can personally launch the app and try out all of its functionality.

Multi-language interface

Multi-language.interface.mp4

Drowsiness detection

The drowsiness detection takes about 1-1.25 seconds followed by playing the special warning sound signal. The detection is based on the YOLOv5 model trained on a custom dataset labeled in Roboflow.

Drowsiness.detection.mp4

Fist gesture

This gesture is used to restart the recognition system.

Fist.gesture.mp4

Palm and V gestures

The palm gesture is used to wake up the drowsiness recognition system. The V gesture is used to put the drowsiness recognition system into sleep mode.

Palm.and.V.gestures.mp4

Multiple cameras

Multiple.cameras.mp4

Smart interaction with the camera

The application can handle turning off an active camera, no available cameras, and connecting a new camera.

Smart.interaction.with.the.camera.mp4

Cross-platform

The application is cross-platform (Windows/Linux). For Linux, it was tested on Ubuntu 22.04.