catastrophic cancellation

Losing nearly every significant digit by subtracting two large, nearly equal floating-point numbers.

Catastrophic cancellation is the loss of relative precision when two nearly equal rounded quantities are subtracted, as in the sum-of-squares variance formula evaluated in float on values far from zero.

Examples

Computing the variance of 100 002 to 100 009 with the sum-of-squares formula in float returned 0 instead of 4.57.
Subtracting two huge, nearly equal running sums leaves only rounding error when float keeps about 7 significant digits.