The Udacity Self Driving Nanodegree Term 1 is mostly about Deep Learning using Tensorflow and Keras to build Convolutional Neural Nets (CNNs).
In Project 3, we are given a simulator that we need to use to gather data and then train a CNN to drive itself around the track. What we're really doing is simple ;), we're training a network to predict a steering angle based off an image.
It took me way longer than I hoped to do this, but I finally figured it out and it's now successfully staying on the track!!
Clone this repo, then open a terminal in the main folder, and run:
python drive.py
Open up the driving simulator program and it should work from there! (The driving simulator would need to be given to you from Udacity)
The data I used was from Udacity, as they had a more stable dataset to work with. I had issues using my keyboard to get realistic images to train the model with.
The main thing I noticed with the data was that on each side of 0 (left or right driving), each bucket isn't totally even. So I wanted to make sure to even out both the sides so the model didn't generalize this and try to go left or right more often than it should.
I used the model that comma.ai open sourced, but I'm pretty sure the NVIDIA model (also contained in the repo) would work well now that I've figured out augmentations better.
I used dropouts between the layers to lower problems with overfitting the model, and I also used data augmentation to avoid overfitting. Specifically,
# Augment brightness so it can handle both night and day
image = augment_brightness_camera_images(image)
trans = np.random.random()
if trans < .2:
# 20% of the time, return the original image
return return_image(image), steering
trans = np.random.random()
if trans > .3:
# Flip the image around center 70% of the time
steering *= -1
image = cv2.flip(image, 1)
trans = np.random.random()
if trans > .5:
# Translate 50% of the images
image, steering = trans_image(image, steering, 150)
trans = np.random.random()
if trans > .8:
# 20% of the time, add a little jitter to the steering to help with 0 steering angles
steering += np.random.uniform(-1, 1) / 60
I used the comma.ai model, a batch size of 128, and an AdamOptimizer. I found a lower learning rate to work much better for my model, so that we didn't reach a false minimum error.
Here are my main takeaways from my work:
- Do not pre-generate the images. I originally made something that generated the images (translations, flips, etc) and then just read those images for training. This didn't work well. I think it's because I basically did
Original Image * CHOICE([Flip, Translation, Brightness])
, but you need to just do the translations on the fly to really give the model enough images to train on - Consider large numbers of EPOCHS. When using dropouts and translations, I wasn't overly worried about overfitting. My late EPOCH versions worked best
- Do all the augmentations above, and make sure it varies. Keep your model guessing/learning
- 128 mini-batches worked better for me than anything larger
- Ask a lot in Slack and the forums!