Autodiff

Static reverse-mode automatic differentiation in C

This repository consists of a scalar-valued reverse-mode automatic differentiation library, extended into a tensor computation library, used as the foundation of a multilayer perceptron model that scores 96% accuracy on the MNIST database. Also included is a curve fitting demo.

The multilayer perceptron works in two stages: in the first it builds a computation graph for the model then generates C source code that directly computes the gradient of the cost function with respect to model parameters, and in the second it compiles that C source code as a library and uses it for gradient descent. The curve fitting demo, on the other hand, builds a computation graph then runs an interpreter over it in a single stroke.

Run the multilayer perceptron against MNIST with:

make -j2 bin/mlp-fit && bin/mlp-fit

Run the curve fitting demo with:

make bin/curve-fit && bin/curve-fit