exploding gradient
A gradient that grows uncontrollably large as it's propagated backward through many layers, destabilizing training.
Examples
Weights of magnitude 2 feeding a ReLU layer (slope 1) give a per-layer factor of 2; ten layers compound that to 2^{10}=1024.
See also
Point table
Graph settings
Calculator