vanishing gradient
A gradient that shrinks toward zero as it's propagated backward through many layers, stalling learning in early layers.
Examples
Multiplying sigmoid's largest slope, 0.25, by itself ten times gives 0.25^{10}\approx0.00000095 -- a gradient about a million times smaller.
Point table
Graph settings
Calculator