On May 23, 2016, ProPublica published “Machine Bias,” an investigation by Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner into COMPAS, a risk-assessment algorithm sold by Northpointe and used in US courts to score how likely a defendant is to reoffend. The reporters obtained COMPAS scores for more than 7,000 people arrested in Broward County, Florida, and checked, over the following two years, who actually went on to be charged with new crimes.
Their headline finding was about error rates. Among defendants who did not reoffend, Black defendants were nearly twice as likely as white defendants to have been labeled high risk - a false-positive rate of about 45% versus 23%. Conversely, white defendants who did go on to reoffend were more often mislabeled as low risk. ProPublica reported that even after controlling for prior crimes, age, and gender, Black defendants were 77% more likely to be flagged as at higher risk of committing a future violent crime. COMPAS does not ask about race directly; it generates scores from a 137-question survey covering factors like criminal history, employment, and education.
Northpointe disputed the analysis, arguing that COMPAS was in fact equally accurate across races by the measure that matters - among people given the same score, the actual reoffense rate was similar regardless of race. Researchers later showed that both sides were partly right: when the underlying base rates differ between groups, it is mathematically impossible for a risk score to satisfy every definition of fairness at once. That impossibility result, sharpened by the COMPAS debate, became central to the field of algorithmic fairness.
Why business readers should care: the story is the classic cautionary tale that a model can look “unbiased” by one statistical definition and “biased” by another, with no way to satisfy both. Any organization deploying a scoring model on people should decide, in advance and explicitly, which notion of fairness it is optimizing for - because the choice is unavoidable and consequential.