Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

KL divergence

The extra, avoidable surprise a guess costs above the unavoidable minimum the true distribution's own uncertainty forces.

Examples

For $p=(0.7,0.2,0.1)$ and $q=(0.5,0.3,0.2)$ , $D_{\mathrm{KL}}(p\|q)=H(p,q)-H(p)\approx0.0851$ nats.