This lecture opens Andrej Karpathy’s “Neural Networks: Zero to Hero” series. In it he builds micrograd, a tiny automatic differentiation engine, entirely from scratch in Python. The point is to make backpropagation concrete: rather than treating it as a black box inside a framework, Karpathy shows exactly how gradients flow backward through a computation graph one operation at a time.
He starts with simple scalar values, defines how addition and multiplication propagate gradients, and gradually assembles enough machinery to train a small neural network. By the end the viewer has seen the same core idea that powers PyTorch and TensorFlow, stripped down to a few dozen lines they could write themselves.
Backpropagation is the algorithm that makes modern deep learning possible, and it is often taught with formulas that obscure what is really happening. By building it up from nothing, Karpathy gives a durable, intuitive understanding that transfers to any framework. For anyone serious about learning how neural networks are trained, this is a foundational watch.