CERLAB UAV Autonomy Framework

Welcome to the CERLAB UAV Autonomy Framework, a versatile and modular framework for autonomous unmanned aerial vehicles (UAVs). This framework comprises distinct components (simulator, perception, mapping, planning, and control) to achieve autonomous navigation, unknown exploration, and target inspection.

Author: Zhefan Xu, Computational Engineering & Robotics Lab (CERLAB) at Carnegie Mellon University (CMU).

Contact Email: [email protected]

Video Tutorial: YouTube, BiliBili (for China mainland)

If you find this work helpful, kindly show your support by giving us a free ⭐️. Your recognition is truly valued.

News

• 2024-01-10: The GitHub code, video demos, video tutorials and relavant papers for our autonomous UAV framework are released. The authors will actively maintain and update this repo!

Table of Contents

1. The Autonomy Modules Introduction
2. Installation Guide
3. Run Autonomy DEMO
4. PX4 Simulation & Real Flight
5. Citation and Reference
6. Acknowledgement
7. Write at the End

I. The Autonomy Modules Introduction

The funtionality of each autonomy module included in this framework in alphabetical order:

• autonomous_flight: The autonomous flight package integrating all other modules for various tasks. details
• global_planner: The global waypoint planner library for autonomous robots. details
• map_manager: The 3D mapping library for autonomous robots. details
• onboard_detector: The dynamic obstacle detection and tracking algorithm for autonomous robots. details
• remote_control: The Rviz configuration and launch files for easy visualization. details
• time_optimizer: The optimal trajectory time allocation library for autonomous robots. details
• tracking_controller: The trajectory tracking controller for autonomous robots. details
• trajectory_planner: The trajectory olanning library for autonomous robots. details
• uav_simulator: The lightweight Gazebo/ROS-based simulator for unmanned aerial vehicles. details

II. Installation Guide

This repo has been tested on ROS Melodic with Ubuntu 18.04 and ROS Noetic with Ubuntu 20.04 and it depends on the ROS packages: octomap, mavros, and vision_msgs. Installing the package with the following commands:

# step1: install dependencies
sudo apt install ros-${ROS_DISTRO}-octomap* && sudo apt install ros-${ROS_DISTRO}-mavros* && sudo apt install ros-${ROS_DISTRO}-vision-msgs

# step 2: clone this repo to your workspace
cd ~/catkin_ws/src
git clone --recursive https://github.com/Zhefan-Xu/CERLAB-UAV-Autonomy.git

# optional: switch to simulation branch for autonomous_flight
# the default branch is for real flight and PX4 simulation
cd path/to/autonomous_flight
git checkout simulation

# step 3: follow the standard catkin_make procedure
cd ~/catkin_ws
catkin_make

setup environment variable. Add following to your ~/.bashrc

source path/to/uav_simulator/gazeboSetup.bash

III. Run Autonomy DEMO

This section shows the most typical ones: navigation, exploration, and inspection. Note that the default environment and the flight parameters might be different from the demos shown as below. Please check uav_simulator for changing the simulation environments and autonomous_flight for adjusting flight parameters.

Before getting started, please make sure you are in the simulation branch of the submodule autonomous_flight for the following demos (please check the link for detailed explanations):

cd path/to/autonomous_flight
git branch

# if the output says you are not in the simulation branch, please run the following (otherwise please ignore): 
git checkout simulation
cd ~/catkin_ws
catkin_make clean # if you switch the branch for autonomous_flight
catkin_make

a. Autonomous Navigation: Navigating to a given goal position and avoiding collisions.

# start simulator
roslaunch uav_simulator start.launch # recommand to use the corridor env for your first trial

# open the Rviz visualization
roslaunch remote_control dynamic_navigation_rviz.launch # if your test env has dynamic obstacles

# run the navigation program
roslaunch autonomous_flight dynamic_navigation.launch # if your test env has dynamic obstacles

# --------------------------------------------------------------------------------------
# (alternatively, if your test env is purely static, you can run the following instead)
# open the Rviz visualization
roslaunch remote_control navigation_rviz.launch # if your test env only has static obstacles

# run the navigation program
roslaunch autonomous_flight navigation.launch # if your test env only has static obstacles

Once the robot is hovering at the predefined height (check the terminal output messages), you can use the 2D Nav Goal to click a goal point in Rviz and you can see example results shown below:

navigation_demo.mp4

b. Autonomous Exploration: Exploraing an unknown environments and create a map.

# start simulator
roslaunch uav_simulator start.launch # recommand to use the floorplan2 env for your first trial

# open the Rviz visualization
roslaunch remote_control exploration_rviz.launch 

# run the navigation program
roslaunch autonomous_flight dynamic_exploration.launch

The example exploration process is shown in the video demo as below:

exploration_demo.mp4

c. Autonomous Inspection: Navigating to the target and inspecting it with a zig-zag path.

# start simulator
roslaunch uav_simulator start.launch # # recommand to use the tunnel_dynamic_1 env for your first trial

# open the Rviz visualization
roslaunch remote_control inspection_rviz.launch 

# run the navigation program
roslaunch autonomous_flight dynamic_inspection.launch

The example inspection process is shown in the video demo as below:

tunnel_inspection_demo.mp4

IV. PX4 Simulation & Real Flight

This section talks about running this framework in the PX4-based simulation or conducting real flight experiments. Please first follow the PX4 simulation installation guide as provided in uav_simulator.

Before getting started, please make sure you are in the px4 branch of the submodule autonomous_flight for the following demos (please check the link for detailed explanations):

cd path/to/autonomous_flight
git branch

# if the output says you are not in the px4 branch, please run the following (otherwise please ignore): 
git checkout px4
cd ~/catkin_ws
catkin_make clean # if you switch the branch for autonomous_flight
catkin_make

a. PX4 Simulation Experiments

The purpose of having another PX4 simulation (besides the simulator we have shown in the previous section) is to simulate ALL behaviors that we might encounter in the real flight. To run the same demos in the previous section, the only change we need to do is to run the following command to start the simulator instead.

# start PX4 simulator
roslaunch uav_simulator px4_start.launch

b. Real Flight Experiments

Once you have tested the flight in the PX4 simulation, the real flight experiments will have exactly the same behavior as you saw in the simulation. The inputs required for this framework in the real flight experiments are:

• The robot pose/odometry: The framework requires a SLAM/VIO system that can estimate the robot states.
• The depth image: The framework expects the depth image to detect objects and construct the map.

Check all the parameters in the autonomous_flight accordingly before the actual flight!!!

c. Examples of Real Flight Experiments

a. The example of real flight experiment for autonomous navigation:

real_navigation_demo.mp4

b. The example of real flight experiment for autonomous exploration:

real_exploration_demo.mp4

c. The example of real flight experiment for autonomous inspection:

real_inspection_demo.mp4

V. Citation and Reference

If you find this work useful, please consider to cite our papers:

• Zhefan Xu*, Christopher Suzuki*, Xiaoyang Zhan, Kenji Shimada, "Heuristic-based Incremental Probabilistic Roadmap for Efficient UAV Exploration in Dynamic Environments”, IEEE International Conference on Robotics and Automation (ICRA), 2024. [paper] [video].
• Zhefan Xu and Kenji Shimada, “Quadcopter Trajectory Time Minimization and Robust Collision Avoidance via Optimal Time Allocation”, IEEE International Conference on Robotics and Automation (ICRA), 2024. [paper] [video]
• Zhefan Xu*, Xiaoyang Zhan*, Yumeng Xiu, Christopher Suzuki, Kenji Shimada, "Onboard dynamic-object detection and tracking for autonomous robot navigation with RGB-D camera”, IEEE Robotics and Automation Letters (RA-L), 2024. [paper] [video].
• Zhefan Xu, Baihan Chen, Xiaoyang Zhan, Yumeng Xiu, Christopher Suzuki, and Kenji Shimada, “A Vision-Based Autonomous UAV Inspection Framework for Unknown Tunnel Construction Sites With Dynamic Obstacles”, IEEE Robotics and Automation Letters (RA-L), 2023. [paper] [video]
• Zhefan Xu*, Xiaoyang Zhan*, Baihan Chen, Yumeng Xiu, Chenhao Yang, and Kenji Shimada, "A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera”, IEEE International Conference on Robotics and Automation (ICRA), 2023. [paper] [video].
• Zhefan Xu, Yumeng Xiu, Xiaoyang Zhan, Baihan Chen, and Kenji Shimada, “Vision-aided UAV Navigation and Dynamic Obstacle Avoidance using Gradient-based B-spline Trajectory Optimization”, IEEE International Conference on Robotics and Automation (ICRA), 2023. [paper] [video]
• Zhefan Xu, Di Deng, and Kenji Shimada, “Autonomous UAV Exploration of Dynamic Environments via Incremental Sampling and Probabilistic Roadmap”, IEEE Robotics and Automation Letters (RA-L), 2021. [paper] [video]

VI. Acknowledgement

The author would like to express his sincere gratitude to Professor Kenji Shimada for his great support and all CERLAB UAV team members who contribute to the development of this research.

VII. Write at the End

This repository concludes my first two-year Ph.D. work at CMU, and I would like to share it to contribute to the autonomous UAV research community. At the beginning, I truly felt the frustration as an autonomous robot researcher due to the lack of a comprehensive development framework. As a result, I hope my code can provide researchers a comprehensive and easy-to-understand platform and let them focus on algorithm/theory design instead of the software pipeline. In the meanwhile, this research and developed framework are not perfect and can be further improved; thus, I am actively looking for collaborators in research and development to improve the UAV/robot autonomy level. Please don't hesitate to reach out for potential collaboration!