In December 2019 the US National Institute of Standards and Technology published “Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects” (NISTIR 8280), an independent evaluation of face recognition algorithms submitted by developers worldwide. Across the algorithms tested, NIST found systematic differences in accuracy across demographic groups defined by sex, age, and country of origin or race.
The largest effects were in false positives - cases where the system wrongly matches two different people. NIST reported that false-positive rates varied widely across groups, with many algorithms producing far more false matches for some populations than for others, and noted that the false-positive differentials were widespread and generally larger than the differences in false-negative rates. Because the study evaluated algorithms from many vendors using the same government test protocol, it provided one of the most authoritative pieces of evidence that demographic bias in face recognition was a broad property of the technology of that era, not the failing of a single product.