96% failure.
Let that sink in for a minute.
Detroit police have used highly unreliable facial recognition technology almost exclusively against Black people so far in 2020, according to the Detroit Police Department’s own statistics. The department’s use of the technology gained national attention last week after the American Civil Liberties Union and New York Times brought to light the case of Robert Julian-Borchak Williams, a man who was wrongfully arrested because of the technology.
In a public meeting Monday, Detroit Police Chief James Craig admitted that the technology, developed by a company called DataWorks Plus, almost never brings back a direct match and almost always misidentifies people.
Holy. Shit.
We've known for a while now that facial reconition technology was mostly useless for its intended purposes, but this is the first time that anyone in law enforcement has gone on the record to confirm it, and quantify the futility. In addition to admitting that the technology is used almost exclusively against Black Americans, in spite of a proven track record of struggling to identify people of colour in general, and Black people in particular, the fact that the Detroit police have continued to use the tech in spite of knowing that it failed almost all of the time is gobsmacking. How the fuck has this been allowed to happen?
Todd Pastorini, a general manager at DataWorks Plus, told Motherboard that it does not keep statistics on the software's accuracy in real-world use, and it does not specifically instruct law enforcement how to use the software.
"There's no statistics for that," Pastorini said. [...] Pastorini likened DataWorks Plus' software to automated fingerprint identification systems, where dozens or hundreds of potential matches are returned. It "does not bring back a single candidate," he said. "It's hundreds. They are weighted just like a fingerprint system based on the probe [and what's in the database]."
In other words, facial recognition technology doesn't actually identify anything. It only returns an enormous list of potential matches, and then lets its users sift through them... except that those users are almost certainly not doing that, since the tech was sold to them as a magic box that could identify people accurately based on their facial features.
I would say that this throws serious doubt on the efficacy of all applications of the technology, including more benign ones like Face Unlock on smartphones and laptops. If your device's facial recognition system has to be forgiving enough to still recognize you in spite of its 96% failure rate, how good a job is actually doing keeping people other than you out of your data? In the law enforcement scenario, crap results can at least be identified by the system returning hundreds of potential matches, rather than actually matching a specific face to a specific person, but how would you even know if your laptop's facial recognition tech was working, when its database contains exactly one face: your own?
Facial recognition technology, and those who are selling it, would be bad enough if the tech were only useless, but its actually much, much worse than useless. It's being actively used for racial profiling, and to harrass, detain, arrest, assault, and occasionally murder the mostly Black people that it's misidentifying... something which, in Detroit at least, police are well aware of. This isn't just incompetent; it's fucking evil, and every company selling this high-tech snake oil should be ashamed of themselves today, and bankrupt shortly thereafter.