Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

variance

A number measuring how spread out a set of values is around their average.

The variance of a quantity is the average squared distance of its values from their mean -- small when the values cluster tightly, large when they are spread out. In attention, scaling scores by 1/\sqrt{d_k} keeps their variance at 1 as vectors get longer.

Examples

  • Test scores of 70, 71, 69 have small variance; scores of 40, 70, 100 have large variance.

Read the full section →
Graph settings

Calculator