README.md

Convolutional Neural Network (CNN)

• Week 5 CNN Slides

Session A: What is CNN?

Objectives:

• Understand when and why you might train your own model from scratch versus use a pre-trained model or transfer learning.
• Learn about the Google “Quick, Draw!” dataset.
• Understand how ato work with image data for training your own model.

Quick, Draw! Data

• Data and Documentation
• Video tutorial: Replaying Drawings with node server
• Video tutorial: Replaying Drawings with Google Web API
• Preparing Data as Images for Doodle Classifer Part 1
• Preparing Data as Images for Doodle Classifer Part 2

Examples

• Displaying MNIST
• Displaying Quick, Draw! 28x28 images
• Animating Quick, Draw! paths

Creative Quick, Draw! projects

• Letter collages by Deborah Schmidt
• Face tracking experiment by Neil Mendoza
• Faces of Humanity by Tortue
• Scribbling Speech by Xinyue Yang
• How do you draw a circle?

Other Related Projects:

• Machine Learning for Visualization by Ian Johnson
• MegaPixels: Faces curated by Tactical Tech, design and development by Adam Harvey
• Watch What Neural Networks See by Gene Kogan
• Recognizing Human Facial Expressions With Machine Learning by Angelica Perez

Session B: Doodle Classification

Objectives

• Learn to train an image classifier (no convolutional layers) with ml5.js.
• Learn the distinction between different types of layers of a neural network, specifically “What is a convolutional layer?”
• Learn to feed the input of a graphics canvas into a machine learning model.

Video Tutorials

• What is Convolutional Neural Network Part 1
• What is Convolutional Neural Network Part 2

Convolutional Neural Nets

• Original 1998 "LetNet5" paper: "Gradient-Based Learning Applied to Document Recognition" by Y. Lecun, L. Bottou, Y. Bengio, P. Haffner
• Interactive Node-Link Visualizations of Convolutional Neural Networks
• How computers got shockingly good at recognizing images by Timothy B. Lee
• Image Kernels Explained Visually by Victor Powell
• A visual and intuitive understanding of deep learning, CNNs (0:00 - 9:40) by Octavio Good

Examples

• p5.js Convolution demo
• p5.js Convolution demo -- max pooling
• Training a model with ml5.neuralNetwork() and Google Quick, Draw! images
• Classifying Drawings with ml5's DoodleNet (model trained by @yining1023)

Assignment 5a

Reading

• An Intuitive Explanation of Convolutional Neural Networks by Ujjwal Karn
• Exploring and Visualizing an Open Global Dataset by Google Research

Data Research

In preparation for next Thursday's class, find a dataset that interests you! Here are some places to get started:

• Something you find online. For example, take a look at Kaggle, awesome datasets or this list of datasets.
• Find a dataset that you collect yourself or is already being collected about you. For example, personal data like steps taken per day, browser history, minutes spent on your mobile device, sensor readings, and more.

Post a link to the dataset that interests you on the homework wiki. (I would also suggest taking a peek at the instructions for assignment 5b in next week's material).