In July 2018 the American Civil Liberties Union ran a test of Amazon’s Rekognition facial recognition service: it compared photos of every then-current member of the US Congress against a database of 25,000 publicly available arrest photographs. Rekognition incorrectly matched 28 members of Congress to mugshots of people who had been arrested.
The errors were not evenly distributed. Nearly 40% of the false matches were of people of color, even though people of color made up only about 20% of Congress at the time. Among those wrongly flagged were six members of the Congressional Black Caucus, including the late civil-rights leader Representative John Lewis. The ACLU also noted the test cost just $12.33 to run, underscoring how cheap and accessible the technology had become.
Amazon disputed the test, arguing the ACLU had used a low confidence threshold that the company did not recommend for law-enforcement use. But the demonstration drew national attention to the prospect of police running face recognition against booking databases, and it became one of the most-cited examples of demographic bias in the technology - a finding consistent with the Gender Shades audit and later NIST vendor tests. The pressure it helped generate fed directly into Amazon’s 2020 decision to pause police use of Rekognition.