Micrograd

A tiny autograd engine and neural network library built from scratch in Python, following Andrej Karpathy's lecture. Backpropagation on single numbers, up to training an MLP.

ArchivedSource
Micrograd

/about

Micrograd does for single numbers what PyTorch does for tensors. Every number is wrapped in a Value that remembers how it was computed, building a computation graph during the forward pass. Each operation knows only its local derivative, and the chain rule plus a topological sort does the rest. On top of that sits a small Neuron, Layer and MLP library, trained with gradient descent on a four-example dataset until the loss falls from about 5.27 to 0.0004. Includes a fully annotated notebook and a write-up of the ideas.

/stack

Python

/more

DocueArchivedIdeosReviving