Cell Phone App Raises Privacy Concerns

Although Recognizr, a new cell phone app, is still just a prototype, it’s already earning criticism for potential privacy concerns. Here’s how it works: You point your iPhone at someone, take a photo and click “Recognize.” The facial recognition software analyzes the image and identifies the person based on social media sites, such as Facebook and Twitter. It then links the image with their other information, like phone numbers and e-mail addresses.


Here’s a video of the prototype at work:





These days it seems like anything linked to the words “facial recognition” immediately raises privacy concerns. But apparently Recognizr requires a fairly stationary, head-on image to successfully identify someone, so it’s doubtful that you’re going to get “recognized” without your permission. Also, it’s unclear whether the photographer has to be friends with the subject on Facebook or MySpace for the app to access his or her information.

Face Rec Coming to Chicago Transit

From Bob Roberts at WBBM:

Pickpockets and muggers beware -- soon, every turnstile at a CTA 'L' station will have airport-style face recognition cameras, and the transit agency is planning much more comprehensive cameras at all 144 of its 'L' stations. [...] CTA President Richard Rodriguez said the agency's goal is to fully outfit all 'L' stations, but said it is dependent on federal funding, and as a result chose the 29 Green and Red Line stations first.

Rodriguez said all of the cameras will be linked not just to the CTA Control Center but to the city's 911 call center and police, as well. At suburban CTA 'L' stations, CTA has established or intends to establish a similar link with local police agencies and Cook County Sheriff's Police.

CTA's board Wednesday approved a $4.3 million contract with Teleste Corp., of Georgetown, Tex., to install the cameras and related equipment. It is financing the high-resolution turnstile cameras, which will be capable of showing facial details, through a $17.9 million U.S. Dept. of Homeland Security grant.

This type of public deployment of facial recognition technology is extremely difficult. So far, 3VR Security is the only organization to come close to providing adequate technology of this task as showcased in its deployment in South Korea.

US/Canada Radio Frequency IDs Set to Go Live

A lot of outlets are covering this piece of news, Homeland Security to scan fingerprints of travelers exiting the US, but I think a far more interesting tidbit is buried deep in the story:

The DHS is also set to go-live with a trial of RF-enabled biometric identity cards on June 1, aimed at securing the border between the United States and Canada.

[...]

As of June 1, travelers crossing the border will be asked to carry an RF-enabled card which transmits a passport photo image and information about the traveler to border control staff systems, who can then check that photograph against the physical appearance of the traveler or their vehicle.

I think this is a sign of what's to come at the border. For now, these kinds of trials will likely remain limited. Also, given recent problems with more automated verification programs, like in Manchester, these trials will almost certainly require a good deal of manual monitoring and human review for the foreseeable future.

South Korean Study Delivers a Face Rec First (90%+ Accuracy Using Surveillance Video)

Those following the South Korean government’s most recent biometric initiative know it to be one of the most ambitious facial recognition testing and deployment projects ever conceived. The project, in its various phases, has been featured quite extensively on Korean television here and here.

Under the project, the Korean National Police (NPA) sought to deploy facial recognition technology against Seoul’s plentiful video surveillance cameras, both to generate alerts to the presence of targeted individuals, as well as provide a mechanism to quickly search city surveillance archives for persons identified after the fact.

Though the NPA has been quite public on its facial recognition project in general, details regarding the specific technologies and testing results have been closely guarded. However, today after much anticipation, the NPA has finally made its initial findings public. According to the NPA and SK Networks spokesmen, 3VR's facial recognition platform demonstrated the highest percentage of accuracy of any of the eight tested technologies in trials run by the NPA and its testing partner, South Korea's IT giant SK Networks.

Indeed the results, at around 90% accuracy, would seem far better than those achieved in any previous public video facial recognition study. After a grueling multi-year testing process, in 3VR SmartRecorders and SmartCams provided between 85 percent and 92 percent accuracy in recognizing and matching faces in a few crowded, highly-trafficked public train stations in Seoul. In each case, the images analyzed were of fast-moving groups of commuters entering or exiting various transit areas en masse.

Said Sung-Ho Kong of SK Networks:

“In 2008, we performed a live, uncontrolled test of 3VR's facial recognition technology in Seoul subway stations, where the solution was an impressive 85-92 percent accurate, depending on conditions. No other solution approached this level of accuracy, vastly improving our ability to track, find and thwart crime in subways and other highly populated areas, which had previously proven extremely difficult to monitor.”

To better appreciate just how impressive the NPA’s results are, we need to compare them to the best previous facial recognition study. In 2006-2007, a similar facial recognition-use case was evaluated by the German Federal Police. After months of testing in a German subway with lighting and traffic much more controlled than in South Korea, a mediocre 60 percent accuracy rating was achieved.

The improvements in accuracy afforded by 3VR are the result of a unique approach to video facial recognition. While most face rec vendors offer technologies optimized for the comparison of flat, normally lit, passport-style photos, only 3VR's software was built from the ground up to address many of the unique challenges presented by real-time surveillance video.

According to Tim Frederick, director of engineering at 3VR:

“This study demonstrates the breakthrough power of 3VR’s patented facial surveillance technology. Unlike other attempts at high-volume face surveillance, which re-purposed still-image face recognition algorithms, the South Korean study benefited from 3VR’s end-to-end video analysis system, specifically designed for this type of demanding video application.”

Expect more details from 3VR and the NPA shortly, but in the meantime, please check out SDN's Rhianna Daniels' feature story on the announcement, as well as a quick diagram of how the technology works and even some up-close and personal screen grabs of the UI.

UPDATE June 22, 2009:

Here is a translation of the South Korean National Police's Phase 1 testing results:

Face.com Brings Face Rec to Facebook

I was just accepted to the Face.com Alpha release and getting set up with the new service was pretty simple:

1) Face.com sent me an email with a Facebook link to their application.

2) I clicked the link and was asked to log on to my Facebook account.

3) The application then asked for permission to access my photos...and my friends photos.

4) Processing

5) Processing

6) Processing

7) I was presented with bunch of pictures of me that the software had found.



All-in-all not a bad experience and the facial recognition accuracy seems pretty good so far. I'll keep digging and check back in later with what I find out.

You are being watched...in 3D

I just got back from the International Security Show, ISCWest. I’ll be writing up some more thoughts on that in a bit, but in the meantime, I thought I’d tide you over with you a few cool videos I found from one of the exhibitors there.

Feeling Software got its start in 2005 building 3D technologies for video games and the entertainment industry. So, it’s no surprise that their offerings to the world of high-tech surveillance look a lot like a 3D shooter.





The demos are still a little rough around the edges, but in them, I think, one can see the beginnings of a new interface paradigm for surveillance monitoring that succeeds in providing both detail and context in one interface. It's huge potential advance over the brain-numbing eyeball-burning video walls that sit at the center of modern security rooms today.

And in addition to offering better situational awareness, the interface abstraction might be used to provide other features from privacy to integration, as well. 3D tracked models might be highlighted, tagged, or even obscured or removed from monitoring model based on security policies, privacy law, or real-time information from other systems.

One more note: For those interested in facial recognition, tracking, and modeling, you should also check out Feeling Software’s Face Flow demo. Though not quite up to the standard of the Benjamin Button tech I commented on the other week, it's pretty cool none-the-less.

How to Create a Digital Human in 18 Minutes

An detailed explanation of how state-of-the-art facial analysis technology transformed Brad Pitt in to an 87 year old man...then aged him backward.



Wow!

Now the Ads Watch You

From Fox4kc.com:

Watch an advertisement on a video screen in a mall, health club, or grocery store and there's a growing chance the ad is watching you too. Small cameras can now be embedded in the screen or hidden around it to track who looks at the screen and for how long. The makers say the software can determine the viewer's gender, approximate age range, and, in some cases, ethnicity, and can change the ads accordingly.

Tru-Media , the makers of the technology, argue that its next generation advertising systems don’t have privacy implications in that they only enable ads to become more targeted. Actual images and identity information are not transmitted back to their servers; only general counts and other statistics relating to ad viewership are retained.

Still, I think that elderly shoppers might not welcome the idea of ad monitors suddenly switching to commercials for Boniva, Depends, and Quaker Oatmeal as they walk by.

Korean ATMs Exploring Face Rec Options

According to JoongAng Daily, one of three English-language daily newspapers in South Korea, the Korean National Police Agency is currently looking into new facial recognition technologies as part of a comprehensive public security plan to reduce crimes in the country.

Bank machines have been involved in several notable crimes in South Korea, including the highly-publicized murders by confessed serial killer Kang Ho-sun earlier this year. In those murders, an ATM was used to withdraw money from the victims' bank accounts before they were kidnapped and killed.

In the JoongAng Daily article discussing new attempts to further secure the nation's ATMs, the author discusses concerns around the implementation of facial recognition technologies, including problems with complicated instructions, cost and the technical headaches often involved:

"But ATMs with such advanced features still come with a litany of problems. Facial recognition machines were once test-run by several banks in 2005. Users were confused by a string of instructions such as "Raise your head, please" or "Step up to the machine, please," before being able to access their accounts. The disabled had even more trouble."

All of these arguments point to an interesting issue, and ultimately the resounding truth that facial recognition should not be implemented solely in the manner criticized above. Instead, government agencies should focus on using face finding technologies to help police procure better evidence, while also implementing face searching technologies in order to help track down and identify suspects.

Using these technologies, alerts can still be generated if someone with a mask or disguise attempts to use an ATM, but that notice would go directly to a monitoring center for evaluation, rather than immediately freezing all activity on the machine. As we know, when you adversely affect a customer's experience too much they become upset, and when too many people become upset the offending technology must be removed -- in this case, the removal wouldn't help anyone, and thus the complete freezing of ATM machines should be uniformly avoided.

3VR has created its own technology to address this problem by combining face finding, face matching and face searching. It is this synthesis of capabilities that ultimately makes the technology effective and leads to the best, least intrusive and most efficient results for all users -- from the smallest businesses looking to manage access control for a single door to multifaceted government agencies and airports.

Protecting Citizens and Foiling Criminals: California DMV Looks to Biometrics Technology to Prevent Identity Fraud

Some consumer rights groups in California are up in arms around plans for the state Department of Motor Vehicles to use biometric and facial recognition technology to identify citizens and control identity fraud.

The California DMV has applied to use biometric technology to create a database that would contain facial and fingerprint information for all California citizens over the age of sixteen. The technology would be used to verify the identities of applicants for driver's licenses and ID cards and would additionally have the ability to compare new photos against older records, thus confirming the individual's identity and thwarting potential identity thieves.

The DMV claims the new system will reduce fraudulent driver's licenses -- pointing out that more than 1,200 identification cards are linked to the wrong individual each year. They are asking for roughly $63 million over the next five years to institute the system and begin creation of the database. States including New Mexico, Texas, Oregon and Georgia have already implemented biometrics technology and experienced success.

On the other hand, groups including the California American Civil Liberties Union (ACLU), the World Privacy Forum, and the Consumer Federation of California are protesting the proposal and calling on legislators to halt the contract's progress. Because the information contained in the DMV's database would be accessible to government agencies, some claim that it could be used for other purposes besides basic identity recognition. Privacy group leaders also worry about data falling into the wrong hands, and insist that safeguards must be put in place to ensure that the implications surrounding biometric data are fully discussed and understood by the public.

While the privacy concerns of innocent citizens are always valid and it certainly is necessary for people to be cognizant of data being held by government agencies, this case is in fact an excellent use of facial recognition technology. To apply for and receive a DMV identification card, citizens are required by prove their identities often with 2 or 3 other pieces of information. Therefore, the only people who have real privacy in this situation are the fraudsters, criminals and identity thieves who resort to using fake information and documents and never reveal their true identity in official records.

Ultimately, using facial recognition technology to prevent the acquisition of fake IDs in scenarios like this one helps everyone. While concerns that this type of biometric comparison may ultimately be used to track the general public in ways that genuinely impinge on privacy are legitimate, this is why sensible, balanced surveillance policy and technology is so important.

Image via Mercury News

Cheerleaders Demo Face Rec

From InfoSync:

"Face Recognition" is the next wave of visage-geared technology, and who better to demonstrate Panasonic's latest phenomenon than three Miami Dolphins cheerleaders? Though it was a nearly insurmountable task to remain focused, we were able to extract the general concept behind Face Recognition.

Somehow, I don't think Ariana has any trouble with people remembering her name.

iPhoto Offers Face Rec for Cats...Not Dogs

Via Maclife:

A huge disappointment for pet owners, was word that iPhoto's newest feature,
Faces, wouldn't recognize animals according to Apple employees on the show
floor. We even did a few tests at Macworld Expo with images they had of the dog
Luce. Well, we are happy to say that we were able to get iPhoto to recognize
Robbie's Cat, Lola.

Poncho will not be happy.

Faces: The New Fingerprints?

As we learned from the International Biometric Group's recent report, fingerprint scanning is now the biometrics option of choice for law enforcement agencies.

But one sheriff's department in Washington State has gotten a jumpstart in taking things a bit further-- the Pierce County Sheriff's Department was the first in the country to try out the MorphoFace Investigate system, a facial recognition system by Sagem Morpho, and their new technology recently made its inaugural criminal catch.

This past fall, the department caught on to an alleged bank fraud ring, and they were able to break up an organized effort to steal ATM cards and make fraudulent bank withdrawals by doing a database search around the ATM photograph. Two potential mugshot matches came up in less than 5 seconds, and it turned out the two matches were the same person, who happened to have a documented history of identify theft.

Police were able to move quickly, get a search warrant, and find the evidence they needed to make their arrest. Posthaste.

What's critical to note here is that facial recognition technology is adding a new layer to the biometric tools already in place. With this layer, we have access to new levels of speed and accuracy, and as the Pierce County forensic investigations manager says, the facial recognition system "eliminates 80 to 85 percent of the work in booking repeat offenders."

Maybe this year Santa will be making his list and checking it just once?

Security in 3D: Coming Soon to an Airport Near You?

Security engineers left and right have been attempting to crack the code on technology to match CCTV photos of faces with image archives in order to capture known criminals and thieves. Recent research out of Arizona State and Michigan State show that 3D laser scans could be eventually utilized to alleviate problems with lighting and photos angles in public areas with fast moving crowds.


Using a newly developed program, Dirk Colbry from ASU and his MSU colleague, George Stockman, performed 300 laser scans of 111 different faces using a commercial scanner to store images, while a horizontal plane of laser light passed over the subject's face. These images were then manipulated to create 3D models of each person's face.

The results were superb-- with the new system, different scans of the same face were matched even when lighting was unusual or the angle from which the images were taken was off by as much as 30 degrees. The error rate was a startling one percent.

This research demonstrates a new step in the advancement of surveillance technology, although its implementation will need to overcome the high price, slow scan speed and short-range sensors that may serve as roadblocks. The current scanner price of $50,000 needs to drop to about a tenth of that price to encourage widespread deployment. Additionally, scans currently take 2 to 5 seconds, making the technique obsolete in large crowds and only functional at choke points, such as airport and train station security checkpoints, where passengers are forced to wait in queues and in close range of scanning devices.

Being able to accurately identify someone who has paused, short-range, at a checkpoint and presented thier face, or iris, or fingers, or hands to the confidence levels demonstrated here has been possible for quite some time. But sometimes it's hard for me to get too excited about most of those approaches. That's because what we would all like to get out of any new biometric systems that get deployed at the airport are shorter lines and fewer checkpoints not more!

At 3VR, we're constantly looking for new algorithms to improve our facial recognition analysis features to combat the disparities in expression, lighting and angle. But to date, we have limited ourselves to writing algorithims for use in conjunction with conventional video cameras. That's because CCTV, and even new IP camers, are cheap, prolific, and offer something that laser scanning systems can't, the ability to work at a distance with uncooperative subjects.

Though current facial surveillance approaches fall somewhat short of 99%+ accuracy of 3D laser scanners, I can say that some of the techniques we are pioneering today hold strong promise of closing that gap. By processing streams of facial data from standard CCTV video feeds, it's possible to create an extremely accurate facial model; maybe one that will someday rival 3D scans. When will facial surveillance catch up? I can't say exactly. But, I do know that technologies like what we deploying at 3VR will get there long before a $50,000 laser scanner becomes as cheap as a video camera.

Face Rec Hacked! Needs "Liveness" Test

Very interesting article in CNet that highlighted an exciting new trend, but also pointed out that it may not be ready for prime time. Many new laptops, including new models from Lenovo, Asus and Toshiba, have started using facial recognition scans as the primary security mechanism for accessing their devices, rather than fingerprints or passwords. Definitely a cool use of new technologies, but as CNet points out, companies need to be sure they get it right before they introduce it to consumers, who would have no way to know their security was compromised.

In this test, security firm Vietnamese Internetwork Security Center (VISC) demonstrated vulnerabilities in laptops' face recognition-based authentication mechanisms that let anyone log in to a computer easily with a "special" photo of the legit owner, even at the highest authentication level. VISC was able to almost instantly produce a photo of CNet Editor Dong Ngo, taken over the laptop's webcam during a Skype chat, that fooled the computer's facial recognition software and successfully logged into a computer registered to Ngo.

Here's how Ngo described the offending photo:

About five minutes later, the technician produced a rather unflattering picture of me on a piece of letter-size paper. I could hardly agree that it was my mug on the photo. Nonetheless, when used in front of the laptop's camera, the Y430's authentication software was happy enough with the photo and logged in within a second. Pretty scary.

This type of hack is going to be very difficult for taditional facial recognition vendors to overcome. Early algorithms in this biometric field all focus exclusively on comparing one single image to another single image. Even if that image is being extracted from a laptop web camera. There is zero concept of context or "liveness" in this approach, and so it is easily spoofed. 3VR Security, is the only company I know of with a facial recognition platform built from the ground up to analyze streams of faces, like those in a video feed, rather than just single images. With this type of approach, subtle changes in motion, expression, pose, and other varialbles unique to a "live" 3D person can be analyzed at the same time a biomtric match is taking place and the kind of spoofing demonstrated here simply would not work. Maybe it's time for laptop vedors to upgrade their algorithms.

CyberExtruder Gets 2D to 3D Face Patent

At the end of November, CyberExtruder announced that the company had been granted a new patent on their process for creating reliable 3D models of a person’s face from a single or series of 2D images.




Though the enforceability of this patent has yet to be tested, CyberExtruder’s 2D-to-3D conversion is certainly an important innovation to the security and biometrics technology industries. Why? It enables better matching between offline photographs and surveillance video. While the matching of "watch list" images against surveillance video has traditionally achieved mixed results as a result of inadequate lighting, angle, expression, etc., this patent could signify a leap forward in terms of the quality and value of 3D facial images.

In June of 2007, I wrote on how XID was using a similar technology in the “world’s largest” facial recognition access control project. In that instance, XID literally generated hundreds of thousands of variants of an enrolled employee’s face rather than using just a single 2D photo converted to a 3D model. Each day when an employee arrived for work, his or her photo would be taken and compared to the database of generated images rather than a single original. Interestingly, this approach generated huge improvements in the performance and accuracy of the Thailand access control system.



We’ll see if these two companies come into conflict over the new patent, but I don’t think they will. XID’s approach to 2D-to-3D is very different than CyberExtruder's-- almost quick and dirty by comparison. CyberExtruder, on the other hand, has become famous for its hyper-realistic…if sometimes creepy…generated floating 3D heads that lend themselves to applications well beyond security including gaming and movies, and even boast a fan in Phillip Rosedale of SecondLife.

Face Swapping 2.0 - Now Mostly Prettier

In July, I made a post regarding new technology from Columbia University that swapped faces in photos and videos with generic amalgems drived from a photo database. The technique, which had potential privacy benefits...and even certain Hollywood applications...suffered from one major drawback. The modified faces were really really really ugly.

See below (From Columbia University Research Paper):



Tough to stomach, I know, but these results were bound to be approved someday, and thankfully that day is here. The New York Times is reporting that computer scientists in Israel have taken a more beauty-centric approach to face swapping that simply replaces faces in images and video with more idealized representations of the same faces. Their software applies an algorithm covering over 200 different facial measurements that brings a target face closer to the ideal. In most cases, this approach goes well beyond simple airbrushing, dramatically altering a person's appearance and perceived identity.

See below (From Lars Klove for the NYT):



I am not sure either technology is perfectly suitable for privacy protection in video surveillance just yet, but given the choice between the two approaches above, for aesthetic reasons, I'd go with the latter.

2 Years Post Neven Deal...Google Launches Face Rec

Via TechCrunch:

In the anticipated release of Google’s new and improved Picasa, the company will offer facial recognition technology to help you identify friends and family in your pictures without requiring you to tag them by-hand each time you see them.

Launching at noon PDT today, Picasa’s facial recognition technology will ask you to identify people in your pictures that you haven’t tagged yet. Once you do and start uploading more pictures, Picasa starts suggesting tags for people based on the similarity between their face in the picture and the tags you already put in place for them.

The facial recognition technology comes to Picasa thanks to an acquisition Google made in 2006 of Neven Vision, a company that specialized in matching facial detail with images already found in a centralized database. Picasa’a facial recognition technology works in much the same way.

More from the Folks at Image Metrics

This video is impressive...and a bit creepy:

Evansville Airport Gets Face Rec

From the Evansville Courier & Press:

Security is the paramount issue among air travelers these days, but Evansville Regional Airport — like other commercial airports throughout the world — rely on videocassette recorder security systems whose nonspecific tapes can take hours, days and sometimes weeks to decipher when suspicious concerns arise.

That, however, is about to change at the local airport.

The Evansville airport is the first in the nation to get a new technological security system, known as a 3VR (Third-Generation Video Recorder), said Bob Working, the airport manager. The searchable surveillance system uses a Google-like analytical search engine for spewing out valuable information instantly. For example, it displays in an instant on a computer monitor clear images of thousands of faces for identification purposes.

Eric Moss, vice president/director of data services for Gaither Technologies/STC, demonstrates a 3VR security system as company President Steve Rudolph is recorded on video, in the screen at left. The unit features searchable facial recognition and motion-changing events.