Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

exploding gradient

A gradient that grows uncontrollably large as it's propagated backward through many layers, destabilizing training.

Examples

  • Weights of magnitude 2 feeding a ReLU layer (slope 1) give a per-layer factor of 2; ten layers compound that to 2^{10}=1024.

Read the full section →
Graph settings

Calculator