Curated list of ressources for deep learning and machine learning - in progress

You can find below a curated list of ressources for deep learning and machine learning. It contains repos, articles, research papers and ebooks. I am trying to keep this list as minimal as possible!

python review

a python quick reference ⭐ article

deep learning ebook

deep learning reference book that you can download for free ⭐ ebook
to get familiar with mathematical notations, the hundred pages machine learning book ebook

random forest

Random forests are very good for classification but cannot extrapolate.
1️⃣ to get a visual intuition of how random forests work article
2️⃣ to code a random forest from scratch ⚙️⚙️ notebook
this is a very good exercise to brush up your coding skills and get an in-depth knowledge of how random forests work
3️⃣ random forest interpretation ⭐⭐⭐ (tree variance feature importance, partial dependence, tree interpreter) article

neural nets

• activation functions: tanh, relu, leaky relu, sigmoid, softmax
• initialization: He initialization, Xavier initialization
• regularization: L2, L1, dropout, data augmentation, early stopping
• optimization: batch gradient descent or vanilla gradient descent, stochastic gradient deschent, mini batch gradient descent, gradient descent with momentum, RMS prop, Adam optimization, learning rate decay, batch normalization

1️⃣ how to use cyclical learning rates for training neural networks ⭐ paper

time-series

1️⃣ great practical repo to get SOTA results quickly ⭐⭐ notebooks
2️⃣ quick walk through different time-series predictions methods (neural net, cnn, lstm, cnn-lstm, encoder-decoder lstm) article

SOTA algorithms

2017 hive-cote: ensemble of 37 classifiers (no neural net)
2019 rocket: using random convolutional kernels paper

image recognition

• filter, padding, stride, convolution, pooling (max, average), fully connected layer

cnn architectures

year	model	size	top-1 accuracy	top-5 accuracy	parameters	depth
2012	AlexNet
2014	VGG16	528MB	0.713	0.901	138,357,544	23
2015	InceptionV3/GoogleNet	92MB	0.779	0.937	23,851,784	159
2015	ResNet50 ⭐	98MB	0.749	0.921	25,636,712

segmentation

2015 unet paper