backpropagation

The algorithm that computes how much to nudge every weight, working backward from the loss.

Backpropagation computes the gradient of a network's loss with respect to every weight, by applying the chain rule backward from the output layer to the input layer one layer at a time, reusing each layer's already-computed result rather than recomputing it from scratch for every weight individually.

Examples

After a forward pass produces a wrong class score, backprop computes how much nudging each of the network's thousands of weights would have reduced the loss, reusing each layer's already-computed gradient rather than recomputing it from scratch.