F1 2019 official game car control with the implementation of Deep Learning algorithms. Data generation with lane line detection, with game capture and image processing using PIL and OpenCV.
Acess to keys data was obtained with pydirectinput library. Check the repository and LearnCodeByGaming awesome content (also on Youtube and official website).
Inference code includes GPU support. An NVIDIA GeForce GTX 1060 6GB was used for this project.
- Acess to game capture with OpenCV;
- Road Lane Line Detection with Hough Line Transform;
- Collection of video frames data with encoded pressed and released keys;
- Custom CNN and Mobile Net implementation.
To collect data from the game (keys and frames), use the input.py script. Individual game frames with track delimiter can be acquired with the data_collection.py script.
Check the requirements.txt to see the project dependencies. These include tensorflow --version 2.4 and CUDA 11.
sudo pip install -r requirements.txt
git clone https://github.com/taamfp/F12019DL.git
cd F12019DL
python3 input.py (above 5000 frames)
python3 custom_cnn.py (F1_model.h5)
python3 main.py
Both custom CNN and Mobile Net (paper) models were implemented for the project.
The custom CNN is given with an input size of 350x350, following a Conv2D layer section. A Flatten layer compresses all the information to a 1D array for a fully connected
layer section. The rectifier activation function was used in all layers except the output layer.
The output layer consists of 9 nodes, one for each possible keyboard movement. In this case the softmax activation function was used.
Categorical Cross Entropy function is used in this situation for multi-label task, which can be defined as the following:
where C is the number of classes, yt the true class vector and y_score the model inference vector. Optimization of weight matrix was done with
Adam Optimizer.
To check the inference metrics, use tensorboard --logdir logs/.
- Performance optimization;
- Hyperparameter Tuning;
- Online Inference;
- Implementation of other algorithms.
Feel free to add any sugestions or to contribute to the project 👍 .