Developing Autonomous vehicles (AVs) is a very challenging task in terms of Programming, Optimizing and Deployment , especially if we are working in complicated environments, Such as a mixed traffic environment where the AVs coexist with normal human-driven vehicles (HDVs).
In this project we have developed an efficient and scalable multiagent reinforcement learning model that can be used in different dynamic traffic uses cases where the communication topology could be time-varying. Parameter sharing and local rewards are exploited to foster inter-agency cooperation while achieving great scalability. Later, An action masking scheme is employed to improve the learning efficiency by filtering out invalid/unsafe Actions at each step. In addition, a novel priority-based safety supervisor is developed to significantly reduce the collision rate and greatly expedite the training process. A simulation environment was tuned, based on a previous open-source project published on GitHub, with three different levels of traffic densities. We exploit curriculum learning to efficiently learn harder tasks from trained models under simpler settings.
List of publications our project was based on :
- Approximate Robust Control of Uncertain Dynamical Systems (Dec 2018)
- Interval Prediction for Continuous-Time Systems with Parametric Uncertainties (Apr 2019)
- Practical Open-Loop Optimistic Planning (Apr 2019)
- α^α-Rank: Practically Scaling α-Rank through Stochastic Optimisation (Sep 2019)
- Social Attention for Autonomous Decision-Making in Dense Traffic (Nov 2019)
- Budgeted Reinforcement Learning in Continuous State Space (Dec 2019)
- Multi-View Reinforcement Learning (Dec 2019)
- Reinforcement learning for Dialogue Systems optimization with user adaptation (Dec 2019)
- Distributional Soft Actor Critic for Risk Sensitive Learning (Apr 2020)
- Bi-Level Actor-Critic for Multi-Agent Coordination (Apr 2020)
- Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes (Jun 2020)
- Beyond Prioritized Replay: Sampling States in Model-Based RL via Simulated Priorities (Jul 2020)
- Robust-Adaptive Interval Predictive Control for Linear Uncertain Systems (Jul 2020)
- SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction (Jul 2020)
- Delay-Aware Multi-Agent Reinforcement Learning for Cooperative and Competitive Environments (Aug 2020)
- B-GAP: Behavior-Guided Action Prediction for Autonomous Navigation (Nov 2020)
- Model-based Reinforcement Learning from Signal Temporal Logic Specifications (Nov 2020)
- Robust-Adaptive Control of Linear Systems: beyond Quadratic Costs (Dec 2020)
- Assessing and Accelerating Coverage in Deep Reinforcement Learning (Dec 2020)
- Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing (Jan 2021)
- Interpretable Policy Specification and Synthesis through Natural Language and RL (Jan 2021)
- Deep Reinforcement Learning Techniques in Diversified Domains: A Survey (Feb 2021)
- Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles (Feb 2021)
- Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment (Feb 2021)
- Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge
- Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learning (Apr 2021)
- Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic (May 2021)
- Accelerated Policy Evaluation: Learning Adversarial Environments with Adaptive Importance Sampling (Jun 2021)
- Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic Scenarios (Jul 2021)
- Robust Predictable Control (Sep 2021)
PhD theses
- Reinforcement learning for Dialogue Systems optimization with user adaptation (2019)
- Safe and Efficient Reinforcement Learning for Behavioural Planning in Autonomous Driving (2020)
- Many-agent Reinforcement Learning (2021)
Master theses
- Multi-Agent Reinforcement Learning with Application on Traffic Flow Control (Jun 2021)
- Deep Reinforcement Learning for Automated Parking (Aug 2021)
In this Project, we'll be creating artificially intelligent agents that learn from interacting with their environment, gathering experience, and a system of rewards with deep reinforcement learning (deep RL). Using end-to-end neural networks that translate raw pixels into actions, RL-trained agents are capable of exhibiting intuitive behaviors and performing complex tasks.
Ultimately, our aim will be to train reinforcement learning agents from virtual robotic simulation in 3D and transfer the agent to a real-world robot. Reinforcement learners choose the best action for the agent to perform based on environmental state (like camera inputs) and rewards that provide feedback to the agent about it's performance. Reinforcement learning can learn to behave optimally in it's environment given a policy, or task - like obtaining the reward.
This project requires python3 (>=3.5), The graphics require the installation of pygame, which itself has dependencies that must be installed manually.
sudo apt-get update -y
sudo apt-get install -y python-dev libsdl-image1.2-dev libsdl-mixer1.2-dev
libsdl-ttf2.0-dev libsdl1.2-dev libsmpeg-dev python-numpy subversion libportmidi-dev
ffmpeg libswscale-dev libavformat-dev libavcodec-dev libfreetype6-dev gccSeveral scripts and notebooks to train driving policies are available. Here are a few of them you can run using Google Colab or a your local jupyter Lab :
- Highway with image observations and a CNN model [ Open In Colab](scripts)
- Trajectory Planning on Highway [ Open In Colab](scripts)
- A Model-based Reinforcement Learning tutorial on Parking [ Open In Colab](scripts)
- Parking with Hindsight Experience Replay [ Open In Colab](scripts)
- Intersection with DQN and social attention [ Open In Colab](scripts)
