A from-scratch implementation of a feedforward neural network (FFNN) in order to classify handwritten digits form the MNIST dataset, with no machine learning libraries.
This was created to understand the mathematics behind neural networks, from the theory thereof to the practical implementations, and more.
Key Features
- Implementation of a FFNN and backpropagation from first principles
- Experimentation with a range of learning rates
- Visualizes accuracy over epochs for different learning rates
- Written purely in Python + NumPy + Matplotlib
Results
- Achieved ~90% test accuracy after training for 50 epochs
- Demonstrates sensitivity of performance to learning rate adjustments
A detailed write-up of the mathematical derivations, experimental methodology, and results is available here:
Alan Smith – Investigating the Relationship Between Learning Rate and Gradient Descent Convergence (PDF)