In January 2020, Detroit police arrested Robert Williams, a Black man, at his home in front of his wife and two young daughters, and held him for about thirty hours. The arrest stemmed from a 2018 shoplifting of watches at a Shinola store. Investigators had run a blurry surveillance still through face-recognition software, which returned Williams’s expired driver’s license photo as a possible match. As the ACLU put it, Williams was plainly not the man in the footage and was nowhere near the store at the time.
His was the first case of a wrongful arrest caused by a face-recognition false match to be publicly reported in the United States. The ACLU and the University of Michigan Law School’s Civil Rights Litigation Initiative filed suit against the Detroit Police Department on his behalf. The case landed against the backdrop of research, including the Gender Shades study, showing that face-analysis systems misidentify darker-skinned faces and women at far higher rates.
In June 2024 the case settled with what the ACLU called the nation’s strongest police-department restrictions on the technology, including a requirement that face-recognition leads be corroborated by independent evidence before an arrest, mandatory officer training on the technology’s risks, and an audit of past face-recognition-based arrest warrants.
Why a business reader should care: this case shows what happens when a probabilistic tool is treated as a definitive identification. A match score is a lead, not proof, and using it as proof, without human corroboration, produced a concrete and serious harm to an innocent person.