Race Rec Face Rec

In most research and law enforcement circles, the prospect of using advanced facial recognition technology to determine an individual’s sex or race is an understandable gray area. Though census takers, demographers, and police reports have always concerned themselves with these details, the idea that a computer might reliably judge the racial background of each and every city passerby is more than a little creepy to some and raises the specter of racial profiling.

Recently Benjamin Wales, a graduating student at the Royale College of Art, set out to test peoples sensitivities on the subject by deploying his own "race detection" camera apperatuses [Spy Blog via Rajiv Shah] on the streets of London. And though he published no scientific data about the accuracy of his systems, it was clear that many who came across his art project shared Mr. Wales' mixed feelings on the technology.

Had scientific results actually been published in this project, however, I doubt they would have been terribly good. It turns out that this kind of image classification is awfully difficult to accomplish. To understand why, take a look at this research project at Mitsubishi Electric Research Laboratories (MERL).




In this research project, MERL succeeds in getting what you might consider "pretty good" results using two different image classification routines, Male vs. Female & Asian vs. Non-Asian, on excellent video footage of various faces. On thier own, these overly simple binary classifiers work well enough to justify further research, not well enough for any real world real-time law enforcement or profiling function.

Further, if MERL had added additional race classification outcomes, like black, or Korean, or Latino, etc., the level of accuracy they might be expected to achieve would decline considerably. And if they added enough racial classifications to approximate the actual diversity found in major cities, the algorithm would likely cease to provide any meaningful data at all.

One area where racial classification does show some promise, however, is in the the area of video search. The day is not very far away when a police official might be able to query a city surveillance system for an "Asian women with a red purse" when attempting to track down a suspected kidnaper and her victim. There would be a number of false matches, of course, and a human might still need to review a lot of video to ultimately close in on their intended suspect, but the search would be faster, more focused and sweep up fewer innocent bystanders if some face/person search algorithm was used.

I think this is a more realistic and reasonable use case for face classification...and one that might actually do some good, as well.