Capstone Project - Wolf Pack Team

This is the Capstone project for the Udacity Self-Driving Car Nanodegree. We developed software to guide a real self-driving car around a test track. Using the Robot Operating System (ROS), we created nodes for traffic light detection and classification, trajectory planning, and control.

Beyond the Requirements

• Reloading PID parameters while the car is running for faster tuning
• Trajectory plans with multiple successive behaviors so that trajectory planner works well at low frequency
• RViz visualization of car, traffic lights, and trajectory plan
• Smoother acceleration with lower message frequency by adding linear interpolation to the "pure pursuit" code and replacing speed with acceleration in twist messages
• Automatic, real-time adjustments to throttle gain
• Color added to the traffic light message format for smoother stops
• UNet architecture for traffic light detection
• Automated testing via rostest

Note: Find the latest version of this project on Github.

---

Team Members

• Eric Lavigne - team lead
• William Xu - visualization and controller
• Alena Kastsiukavets - traffic light detection
• Mihir Rajput - trajectory planner
• Sergey Vandyshev - traffic light classification

---

Contents

• Project Components
  • Visualization
  • Traffic Light Detection
  • Traffic Light Classification
  • Trajectory Planner
  • Waypoint Follower
  • Stability Controller
  • Gain Controller
  • Automated Testing
• Usage
  • Installation
  • Running the Simulator
  • Running the Simulator with Visualisation
  • Troubleshooting
  • Running the Project on a Real Car
  • Running Automated Tests

---

Project Components

• diagram of nodes and messages

Visualization

Traffic Light Detection

• UNet architecture
• NN is pretrained on medical images from Kaggle competition
• Trained on black-and-white images
• Custom loss function is used based on Dice coefficient
• Separate models for simulator (Dice coefficient is 0.81) and Carla (Dice coefficient is 0.66)
• See training code in detector and inference code in tl_detector.py.

Traffic Light Classification

Simulator Model	Carla Model

• Four output classes: GREEN, YELLOW, RED, NONE.
• Test accuracy was 100% for simulator images and 84.7% for Carla images.
• See model and training code for Carla and Simulator in iPython notebooks.
• See inference code in tl_classifier.py.

Trajectory Planner

Decision Function	Ideal Scenario

• Acceleration and deceleration profiles are governed by the kinematic equations of motion.
• Safe distance allows deceleration at 10% of maximum deceleration while still stopping in time.
• If further than safe distance from the traffic light, the car ignores the traffic light's color and accelerates up to cruising speed.
• Within safe distance of a traffic light, car decelerates at whatever rate would result in stopping exactly on the stop line.
• When car within stopping distance (3 meters) of yellow or red light, it will decelerate at maximum deceleration.
• When car is less than hysteresis distance (1 meter) past the stop line at a yellow or red light, it will continue max deceleration.
• When car is more than hysteresis distance (1 meter) past the stop line, or if the traffic light turns green, car will accelerate up to cruising speed.
• See code in waypoint_updater.py.

Waypoint Follower

• Substantial improvement over original "pure pursuit" version provided by Udacity
• Originally determined intended speed based only on first waypoint, which quickly became stale
• Originally reported intended speed to stability controller, which also quickly became stale
• Changed to consider current position and varying lookahead distance depending on speed
• Changed message format from speed to acceleration, which is much more stable
• See code in pure_pursuit_core.cpp#calcAcceleration.

Stability Controller

• Tuned PID parameters with sliders while the simulator was running.
• Tuned for high speed in simulator.
• Needed for moderating the output of waypoint follower and preventing overshoot.
• See code in stability_controller.py.

Gain Controller

• We know that the real car has a much more sensitive throttle and brake than the simulator, but don't have enough information to calculate this difference.
• The effect of throttle on acceleration is also very non-linear depending on car's speed.
• Use a series of linear models for 0-10mph, 10-20mph, 20-30mph, etc.
• For each of these small speed ranges, assume required throttle or brake is linear function of current speed and intended acceleration.
• Collect data from car behavior for on-the-fly calibration of throttle and brake models
• See code in gain_controller.py.

Automated Testing

---

Usage

Installation

• Be sure that your workstation is running Ubuntu 16.04 Xenial Xerus or Ubuntu 14.04 Trusty Tahir. Ubuntu downloads can be found here.

• If using a Virtual Machine to install Ubuntu, use the following configuration as minimum:

  • 2 CPU
  • 2 GB system memory
  • 25 GB of free hard drive space

  The Udacity provided virtual machine has ROS and Dataspeed DBW already installed, so you can skip the next two steps if you are using this.

• Follow these instructions to install ROS

  • ROS Kinetic if you have Ubuntu 16.04.
  • ROS Indigo if you have Ubuntu 14.04.

• Dataspeed DBW

  • Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install (binary)

• Download the Udacity Simulator.

Running the simulator

1. Clone the project repository

git clone https://github.com/udacity/CarND-Capstone.git

2. Install python dependencies

cd CarND-Capstone
pip install -r requirements.txt

3. Make and run styx

cd ros
./rebuild-all.sh
source devel/setup.sh
roslaunch launch/styx.launch

4. Run the simulator

Running the Simulator with Visualisation

cd ros
./rebuild-all.sh
source devel/setup.sh
roslaunch launch/wolfpack.launch use_ground_truth:=true

# running rqt_gui in a different terminal
rqt --perspective-file "$(rospack find wolfpack_visualisation)/launch/rqt.perspective"

Troubleshooting

If the car does not move or behaves poorly, try applying the "eventlet monkey patch" with this modification to the roslaunch command. This helps on some computers and hurts on others, so try both ways if you're having problems.

EVENTLET_MONKEY_PATCH=true roslaunch launch/styx.launch

Running the Project on a Real Car

1. Download training bag that was recorded on the Udacity self-driving car
2. Unzip the file

unzip traffic_light_bag_files.zip

3. Play the bag file

rosbag play -l traffic_light_bag_files/loop_with_traffic_light.bag

4. Launch your project in site mode

cd CarND-Capstone/ros
roslaunch launch/site.launch

Running Automated Tests

To run a single unit test with all the rosout logging in the console

# rostest from ros dir
rostest --text src/waypoint_updater/test/waypoint_updater.test

# nosetest from ros dir
nosetests -s src/twist_controller/test/test_stability_controller.py

To run all ROS unit tests

# from ros dir
catkin_make run_tests

To watch for file changes and run unit tests on any file modifications. If the below commands don't work on windows then change the watch-run.sh, uncomment the inotify command and comment out the existing inotify command

# from ros dir watch src dir and run single test
./watch-run.sh src "rostest --text src/waypoint_updater/test/waypoint_updater.test"

# from ros dir watch src dir and run nosetest
./watch-run.sh src "nosetests -s src/twist_controller/test/test_stability_controller.py"

# from ros dir watch src dir and run all unit tests
./watch-run.sh src "catkin_make run_tests"

Adding ros node tests to new package

read http://wiki.ros.org/rostest/Writing

1. Create a ROS test file e.g. ros/src/waypoint_updater/test/test_waypoint_updater.py

2. Create a launch file with the node to test. Add a node pointing to the test file e.g. ros/src/waypoint_updater/test/waypoint_updater.test

<launch>
  <node pkg="mypkg" type="mynode" name="mynode" />
  <test test-name="test_mynode" pkg="mypkg" type="test_mynode" />
</launch>

3. add to CmakeLists.txt in the module

if(CATKIN_ENABLE_TESTING)
  find_package(rostest REQUIRED)
  add_rostest(test/waypoint_updater.test)
endif()

4. Add the below to the package.xml

  <build_depend>rostest</build_depend>

Adding ros nosetests to new package

1. Create plain Python unit test file
2. Add to CmakeLists.txt in the Testing section

## Add folders to be run by python nosetests
  catkin_add_nosetests(test/test_stability_controller.py)