Skip to content

How weight initialization affects forward and backward passes of a deep neural network

Notifications You must be signed in to change notification settings

kvmanohar22/DeepNets

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 

Repository files navigation

From Andrej Karpathy's course cs231n:CNNs for Visual Recognition

How weight initialization affects the forward and backprop of a deep Neural Network ?

All the plots were generated with one full forward pass across all the 10 layers of the network with the same activation function

Architecture

There are 10 layers, each layer having 500 units.

Activation Functions

Tanh, ReLU, Sigmoid were used.

Data

Random data points of 1000 training examples are generated from a univariate "normal" (Gaussian) distribution of mean 0 and variance 1. Weights for each layer were generated from the same distribution as that of data points but later on varied to obtain different plots.

About

How weight initialization affects forward and backward passes of a deep neural network

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages