NIST report documents undeniable demographic effects in face recognition systems

WPF Press Release

PDF version: WPF Press Release: NIST report documents demographic effects in face recognition systems

NIST issues undeniable scientific documentation of demographic effects in face recognition systems:

World Privacy Forum calls on face recognition industry to accept, acknowledge, and address new NIST findings on wide-scale demographic effects in face recognition

Based on the NIST report findings, WPF sees the need for a neutral, international organization composed of government, technology users, face recognition developers, consumers, privacy regulators, and civil rights organizations to assess the demographic impacts of face recognition systems and determine policies
+1 760-712-4281


DECEMBER 19, 2019

The World Privacy Forum welcomes the NIST report, Face Recognition Vendor Test Part 3: Demographic Effects, which investigated demographic effects of facial recognition algorithms. NIST states in its report: “We found empirical evidence for the existence of demographic differentials in the majority of contemporary face recognition algorithms that we evaluated.”

The detailed and unambiguous NIST findings indicate strongly that the face recognition industry needs to do more to address the issue of demographic effects in face recognition systems.

“We now have a large-scale, scientifically based study that thoroughly documents demographic effects in modern face recognition systems,” said WPF executive director Pam Dixon. “This report raises troubling questions. It is crucial that we do not take a biometric shortcut to address the problems in face recognition by simply adding more biometrics, like iris scans, to the mix. We need to have a meaningful public discussion about biometrics uses, when, where, or if it is appropriate.”

WPF proposes that in addition to other considerations, face recognition be designated as a technology of very high concern, and be subjected to meaningful safety guardrails, including safety provisions such as:

Testing requirements: Biometric technologies available for use would be required to be tested and evaluated by NIST for accuracy and demographic effects before used in real-world applications.

Proven safe prior to launch: The technology must be proven safe and fit for purpose prior to launch, and must be cleared for market by the appropriate government oversight body. For facial recognition, a demographic effect analysis is essential.

Product labeling: The biometric product must be labeled as meeting standards for accuracy and for demographic effects. (Face recognition.)

Certification and training requirements: Users must meet defined qualification so that they know how to operate equipment and interpret results properly.

Ongoing monitoring: Marketplace monitoring must allow for operator reports, consumer complaints, and ongoing testing.

Use controls: Allowed use cases must have significant definitional controls and procedural requirements.

WPF looks forward to reviewing the NIST report thoroughly and responding in more detail to its thoughtful and nuanced findings, as well as engaging with NIST further.


NIST Face Recognition Vendor Test Part 3: Demographic Effects:

World Privacy Forum Biometrics Work, WPF biometrics work blogroll:

About the World Privacy Forum

The World Privacy Forum is a nonprofit, non-partisan 501(C)(3) public interest research group. WPF conducts in-depth research, analysis, and consumer education in the area of data privacy, and focuses on pressing and emerging issues. It is among one of the only privacy- focused NGOs conducting independent, original, longitudinal research. The World Privacy Forum has had notable successes with its research, which has been groundbreaking and consistently ahead of trends. Regarding biometrics, WPF’s peer-reviewed original research on India’s Aadhaar biometric ID was cited twice in the India Supreme Court landmark 2018 privacy decision, one which impacted over a billion people and corrected the most problematic aspects of Aadhaar. WPF was founded in 2003 and works both nationally and internationally. The Forum also works to encourage collaborative efforts among other non-profits.