Micrograd
A tiny autograd engine and neural network library built from scratch in Python, following Andrej Karpathy's lecture. Backpropagation on single numbers, up to training an MLP.
ArchivedSource

/about
Micrograd does for single numbers what PyTorch does for tensors. Every number is wrapped in a Value that remembers how it was computed, building a computation graph during the forward pass. Each operation knows only its local derivative, and the chain rule plus a topological sort does the rest. On top of that sits a small Neuron, Layer and MLP library, trained with gradient descent on a four-example dataset until the loss falls from about 5.27 to 0.0004. Includes a fully annotated notebook and a write-up of the ideas.
/stack
Python