Fairness Criteria Cannot All Hold

Adjust a classifier and watch demographic parity, equal opportunity, and calibration compete.

Ch 1 1.8

Explorer

Foundations & Data Quality

20 mindifficulty 4/5

Controls

Group A

Base raterₐ60%

Fraction of group A that is truly qualified/positive.

Score separationdₐ2

Distance between the positive and negative score means.

Group sizenₐ1000

Decision thresholdtₐ0

Group B

Base rater_b25%

Score separationd_b2

Group sizen_b1000

Decision thresholdt_b0

Overridden while single threshold is on.

Single threshold for both groups

Uses tₐ for both groups instead of a group-specific threshold.

Group A score distribution

base rate 60%

Group B score distribution

base rate 25%

truly positivetruly negative

Group A confusion counts

n = 1000

505

337

Group B confusion counts

n = 1000

210

119

631

Fairness metrics by group

Compare each criterion across A and B; the gap is what a fairness audit reports.

A: selection rate

56.8%

A: TPR

84.1%

A: FPR

15.9%

A: PPV

88.8%

B: selection rate

32.9%

B: TPR

84.1%

B: FPR

15.9%

B: PPV

63.9%

Demographic parity gap

23.9%

|selection rate A − selection rate B|

Equal opportunity gap

0.0%

|TPR A − TPR B|

Equalized odds gap

0.0%

max(|TPR gap|, |FPR gap|)

Calibration gap

25.0%

|PPV A − PPV B| (calibration-in-the-large)

Forcing demographic parity breaks the others

Holding tₐ fixed, group B's threshold is moved until its selection rate matches group A's exactly.

Base rate gap between groups is 35.0%. When base rates differ and thresholds are forced to equalize selection rates, equal opportunity and calibration cannot also hold — this is Kleinberg–Mullainathan–Raghavan / Chouldechova's impossibility result.

Forced t_b

-0.84

was 0

Demographic parity gap

0.00%

driven to ≈0 by construction

Equal opportunity gap now

12.6%

was 0.0% before forcing

Calibration gap now

46.3%

was 25.0% before forcing

A fair model does not satisfy every fairness definition at once

Demographic parity (equal selection rates), equal opportunity (equal TPR), and calibration (equal PPV / predictive parity) are three different, individually reasonable requirements. Chouldechova (2017) and Kleinberg, Mullainathan & Raghavan (2016) proved that whenever base rates differ across groups and the classifier is not perfect, no single threshold rule — and no single score — can satisfy all three at once except in degenerate cases. The gaps above are the quantitative proof: shrinking one to zero grows another.

Predict before you look

Group A has a 60% true base rate and group B has 25%. If a single shared threshold is used, which fairness gap will be largest?

Scenarios

Teacher-ready starting points.

Documentation

What this shows

Two groups with different base rates and score distributions are scored by a threshold classifier. Selection rate, true/false positive rate, and positive predictive value (calibration-in-the-large) are computed per group, and a dedicated panel forces demographic parity by moving group B's threshold to show what happens to the other two criteria.

Why the three criteria cannot all hold when base rates differ

Each group's scores are modeled as a two-component mixture: negatives at N(−d/2, 1) and positives at N(+d/2, 1), mixed with weight equal to the group's base rate r. A threshold t turns the score into a binary decision.

Demographic parity asks for equal P(decision = 1) across groups. Equal opportunity asks for equal TPR = P(decision = 1 | Y = 1). Calibration/predictive parity asks for equal PPV = P(Y = 1 | decision = 1).

These three quantities are linked through the base rate by Bayes' rule. If TPR and FPR are held equal across groups (equalized odds) but base rates differ, PPV must differ, since PPV = TPR·r / (TPR·r + FPR·(1−r)) is monotone in r for fixed TPR, FPR. Symmetrically, forcing equal selection rates across groups with different base rates forces TPR and FPR apart. Only when r_A = r_B, or the classifier is a perfect separator, can all three coincide.

selection rate = r·TPR + (1−r)·FPR

PPV = r·TPR / (r·TPR + (1−r)·FPR)

equalized odds ⇒ TPR, FPR shared; r_A ≠ r_B ⇒ PPV_A ≠ PPV_B

demographic parity ⇒ selection rates equal; r_A ≠ r_B ⇒ TPR or FPR must differ

Try this

Set both base rates to 0.5 with a single shared threshold. All three gaps should sit near zero.
Drop group B's base rate to 0.2, keep the single threshold on, and watch the demographic parity gap open up even though the decision rule never changed.
Turn off the single-threshold toggle and hand-tune tᵦ until the demographic parity gap is under 1%. Read off the new equal opportunity and calibration gaps.
Compare that hand-tuned result with the automatic 'forced parity' panel — they should roughly agree.
Lower group B's score separation (noisier signal) while keeping base rates and threshold equal to A. See a calibration gap emerge even without a base rate gap.

Common misconception

“A fair model satisfies every fairness definition at once.”

Demographic parity, equal opportunity, and calibration are each defensible on their own, but they are mathematically incompatible whenever the two groups have different base rates and the classifier is imperfect. Picking a 'fair' model means choosing which criterion to prioritize, not finding a model that satisfies all of them — this is a provable impossibility result, not a modeling failure.

DSPA3 §1.8, Ethics and Fairness