The Irreducible Floor
Overlapping class densities set a limit no classifier can pass. Try to beat it, then find out why your validation says you did.
Controls
Distance between the two class means.
Unequal spreads make the optimal rule non-monotone.
Predict class 1 at or above this value.
Jittered draws from the two densities.
Every result on this page is a deterministic function of the seed and the controls.
Overlapping densities and the irreducible floor
The shaded overlap is the Bayes error — no classifier, however clever, can remove it.
Error as a function of the threshold
One minimum when the spreads are equal; the curve flattens as overlap grows.
Where the floor comes from
When the accuracy ceiling sits near the majority baseline, accuracy cannot tell a useful model from a constant prediction — that is the Chapter 5 metrics problem.
The Bayes rule assigns each x to whichever weighted density is larger there, and its error is the integral of the smaller one. That number depends only on the data distribution. A deeper network, more trees, or better tuning can approach it but never go under it — and any validation estimate that appears to has leaked.