Floating Point Has Edges
Compute a variance three algebraically identical ways and watch one of them return a negative number.
Controls
Every value sits near μ = 1e+6, while the spread stays σ = 1.
The stable one-pass update.
Every result on this page is a deterministic function of the seed and the controls.
Relative error of the variance, as the data moves away from zero
Vertical axis is log₁₀ of the relative error against a double-precision two-pass reference.
This sample, three algorithms
n = 500, μ ≈ 1e+6, working in float32.
Addition is not associative
Σ 1/i for i = 1 … 10000, same terms, three orders.
Descending order adds the small terms first, while they can still change the running total. Kahan carries the lost low-order bits forward explicitly.
The edges of the number line
Compare floating-point numbers with a tolerance, never with equality.
When the naive formula reports a negative σ², nothing about the data is wrong. Σx² and (Σx)²/n are two nearly equal enormous numbers, and their difference keeps only the few bits that survived cancellation. The fix is the algorithm, not the dataset.