Chicago’s Surveillance System Raises Privacy Concerns … Again

It’s official: No other U.S. city has such an extensive and sophisticated video surveillance system as Chicago. According to CBS News, the network of an estimated 10,000 cameras also includes innumerable private cameras and may grow to encompass tiny “covert” cameras in the future:

While critics decry the network as the biggest of Big Brother invasions of privacy, most Chicago residents accept them as a fact of life in a city that has always had a powerful local government and police force. Chicago police point to 4,000 arrests made since 2006 with the help of cameras. And, [a recent study] found crime in one neighborhood—including drug sales, robberies and weapons offenses—decreased significantly after cameras were installed.

Regardless of what the critics say, municipal surveillance is here to stay and will only become more prevalent, as even more cities begin to use cameras for everything from capturing and prosecuting criminals and exonerating the innocent to detecting parking violations. Privacy will continue to be an issue only because most surveillance systems are ineffective, even in Chicago, and certainly not private.

Despite how far surveillance has come, many cities still use technology from the 1980s. At the center of these archaic systems are giant video walls that require far too many employees to monitor. A network of 10,000 cameras may sound impressive, but pinpointing a certain event within that amount of data is nearly impossible without the right intelligent technology. This gap in turn creates an all-or-nothing situation in terms of privacy. The cameras and software can’t narrow in on a particular person, so review requires an overly wide net and involves too many innocent people.

There’s no denying that the world needs both security and privacy; the trick is to find a balance between the two. With so many current surveillance systems falling back on dumb cameras—and requiring an army of people to watch them—we get neither. The only way to bridge the security-privacy gap is with new technologies that enhance monitoring and visualization capabilities. Intelligent search allows officials to focus on particular people and make use of privacy protections like access control, encryption and face blurring.

Once we move past '80 technology, it's not only possibly to balance security and privacy concerns; more than that, we can actually acheive both.

Sitting Down with Rajiv Shah

It has been a busy past few months, but I recently had the opportunity to chat with Rajiv Shah, an adjunct assistant professor in the Department of Communication at the University of Illinois at Chicago, and the author of the Smart Cameras blog. Below are some of my answers to his questions around privacy, vendor comparisons, industry connections to academia and the future of the smart camera market.

Again, very excited to share them with the growing IHF readership. Feel free to make comments on any of the responses or questions, and I will be sure to address them.

Also, check out Rajiv's Smart Cameras blog -- it focuses on developments in Chicago's use of video surveillance, as well as other "smart cameras" that utilize additional sensors and/or computer processing techniques. Cool read and a staple on my blogroll.

1. Privacy: What should the industry approach be towards privacy? Should they incorporate features that protect privacy? Should they have default settings that protect privacy or delete information? Or should we not worry about this? Is there a need for an industry-wide approach to this issue?

Conventional wisdom presents ‘privacy vs. security’ as a zero-sum game, one in which gains in one arena necessitate sacrifices in the other. And while there is certainly much truth in this, it is also equally true that in a modern society neither principle can exist without the other. There can be no security without privacy, and no privacy without security.

Today, any meaningful national security failure could create a response that curtails our civil liberties quite broadly. And conversely, next-generation security technologies deployed without adequate privacy and civil liberties protections likely face the crippling backlash of a concerned public. As a result, the security industry needs to take issues of privacy VERY seriously.

For me, an approach to privacy in the context of surveillance starts with a few key principles designed (1) to narrowly tailor a system’s use and (2) to ensure that system access is adequately controlled and audited. Today, conventional “dumb” surveillance systems offer none of these benefits. A large video wall in a security room or command center does nothing to distinguish between security threats and the average person; these systems cast an unnecessarily wide net, relying on human expertise and interest to filter down to focus in on actual security threats.

The issue of what to delete or keep in terms of surveillance becomes much less important on systems where this kind of ‘all-or-nothing’ approach to data access doesn’t exist. For instance, on a 3VR, an investigator might search through many months worth of video information looking for matches or clues relating to the kidnapping of a little girl. However, because this query is done algorithmically using facial recognition, and because the search request is logged and audited, there is ultimately much less concern about the overall retention of video data. The public generally has very little problem with legitimate surveillance investigation that doesn’t subject them to what they feel is needless voyeurism.

Retention of video also becomes less of a concern in the context of new blurring and encryption algorithms designed to protect individual privacy. These new technologies prevent generally tracking and identification of the pubic using surveillance, while preserving the ability of law enforcement and security officials to detect and investigate crime. To better understand what I mean by this, you should take a look at the recent article in New Scientist on some of what we are working on in 3VR labs right now.

In any case, issues of data retention, encryption, access control and the like are often more policy issues than industry issues. Instead, our focus should be enabling decision and policy makers to make, monitor and enforce these choices themselves. Our solutions should present options to do all of this...and more. Today, most security solutions don’t include any privacy protections whatsoever. That needs to change; asking someone to chose between security and privacy isn’t much choice at all.

2. Comparing Vendor Solutions: What can be done to make it simpler for end users to compare and contrast different solutions? It's very confusing now for end users to sort through claims by tens of companies on effectiveness, costs, technology, etc.

Normally, I would say that the answer to this question solely involves the emergence of various standards groups, independent testing and analysis organizations -- that is because the best response to confusion is nearly always more good information. And, I do think there is some good news on the horizon in both of those areas with new security analysts, bloggers and agencies entering the marketplace of ideas every day.

However, because many new solutions’ claims today are so specific and require real-world deployment for actual evaluation, the only way for end-users to fully educate themselves may be through pilot and testing projects that they conduct themselves. New technologies being offered today represent a quantum leap over previous generations of security and surveillance solutions, and end users will ultimately need to make a very significant investment in time and money to educate themselves on their benefits.

3. Connections to Academia: Explain if anything needs to be done to expand the connection between industry and academia. After all, much of the engineering talent has come directly from universities. Are there any suggestions you have for universities and their research?

The disconnect between commercial markets and academia is a classic problem seen across many industries, but I have noticed is a particular problem in the security industry. And as a partial result, there has been comparatively little innovation at the core of this market in recent decades. The surveillance methods used to catch criminals hasn't changed drastically with investigators still found staring at video walls or fast-forwarding through video stores looking for needles in haystacks. Plus, the innovations responsible for rapid productivity gains in knowledge workers in other industry segments seems to have largely passed this industry by. Who are security’s Googles, Microsofts, and Oracles?

To begin to address this issue, I think that most importantly security needs to become the province of innovative and interesting companies again. Only by tackling big, tough and important problems can the security industry hope to lure academia’s best and brightest, or focus them on its problems.

As for universities and their research, there is one problem faced by the security industry today greater than all others…and that is a crisis of our own making. It’s “information overload.” There are quite simply too many cameras and sensors today generating way too much information today, and the resulting torrent of data threatens to overrun our entire industry. Identify ways to process and sort and make meaningful this flood, and you will have done us all a great service…and there is probably a job waiting for you at 3VR, as well.

4. Future Growth of Smart Cameras: Have cameras hit a period of steady growth or do you foresee a potential boom ahead? If so, what are the crucial factors that you see that are limiting growth of that will cause growth to increase? Do we need to improve technology, better end-user experience, etc.

Cameras have seen explosive growth already -- sales worldwide are booming. Not only that, but the general sense of a 'camera' is evolving dramatically; dumb cameras, smart cameras, cameras that record at 200 frames per second, cameras integrated with iPods – they're popping up all over the place and exploding in ways that people could not have anticipated. Not only are the types of cameras available growing exponentially, but the data being collected by cameras has increased by a geometric factor far beyond that. New cameras have higher resolution, higher frame rates. More of just about everything!

As a result, we're stuck drinking from the firehose for the time being. We're inundated with data and have no idea what to do with it due to the sheer volume we're faced with. It's coming in too quickly to comprehend, and as a result, we've discovered that it's not the volume of data you collect, but what you can do with that video (and how quickly) that matters.

The modality of staring at a wall of video screens broadcasting camera streams broke down a long time ago – and we're better off for it. However, as camera volume, quality and speed explodes, we need to figure out how to comprehend and process this volume of data. If we're going to manage the growth of cameras, they don't need to be smarter – we're already capturing more data than we need – but rather more searchable and enable efficient retrieval of vital information.

Sitting Down with Rob Jenkins

I recently connected with Rob Jenkins, a lecturer at the University at Glasgow and leading authority of facial recognition technology in the UK, to pick his brain about different topics in facial recognition, namely airport security, the future of biometric technology and privacy/related concerns.

Rob had some very insightful, innovative answers to my questions, and I'm excited to share them with the growing IHF readership. Going forward, I'm hoping to have other thought leaders and readers contribute content and commentary to this blog, as I'd like to make this more of a forum for biometric, facial recognition and other technology discussions rather than a one-sided conversation. Feel free to make comments on any of the responses or questions, and I will be sure to address them!

Also, check out Rob's departmental Web site for selected publications on gaze perception and other facial identification topics. Very interesting stuff.

In response to Manchester Airport lowering their matching thresholds, The Telegraph quoted you saying that lowering the passport match level to 30 percent would make the system almost worthless. Another perspective is that the previous levels were causing horrendous queues and customer dissatisfaction. Is there a middle ground here?

There is certainly a middle ground in the sense that we can choose where to strike a balance between rejecting genuine matches and accepting false matches. But reducing either type of error generally increases the other, so it’s a trade-off. There is no ‘sweet spot’ where both types of error are reined in.

Despite the advanced nature of this technology, do you believe that there should still be a human element involved in security checks? If so, do you believe we will ever reach a point where this will no longer be necessary?

The main problem with referring the difficult cases to humans is that humans cannot do the task reliably either - even if we’re trained and experienced. Humans are fantastic at matching familiar faces, but our performance with unfamiliar faces is very poor. If we can somehow incorporate the benefits of familiarity into the technology, then it could be transformed.

Facial recognition technologies are popping up all over -- club entrances, bathroom faucets, online photo services, using cameras in lieu of passwords to access computers -- have they hit the tipping point? Is it only time before we use the technology to unlock our front doors and open our car trunks? What trajectory do you see it taking? Staying in security-based deployments, infiltrating everyday life or a balance between the two?

To some extent I think a tipping point is being ushered in, mostly by people who have something to sell. And it is an idea that some sectors are keen to buy into. So in that sense there is a lot of good will wishing the technology to work. I don’t find the gadget market especially troubling, provided that errors are of relatively little consequence. The real danger is in rushing to large-scale security deployments. For applications such as passport control or forensic face recognition the stakes can be much higher, and we know that the available technology is not yet up to the task.

In the same vein, has facial recognition reached a point where accuracy and reliability now line up with the media's expectations?

In my experience, identification errors tend not to go down well with the public. I often ask audiences how often they would be prepared to be the subject of a misidentification. The answers are in the order of once a decade, even when the imagined consequences are minor. That’s a tall order, given the number of identity checks that some proposals entail. It comes as something of a shock when these demands are compared against current capability. As far as media expectations are concerned, I think there has been a change in tone. Traditionally, the emphasis has been on the implications face recognition for privacy, with the unspoken assumption that it is reliable. These days there is more of an awareness that the technology simply is being phased in, whether it works or not. That changes the focus of the debate.

The "Big Brother" argument -- that citizens are losing their individual privacy rights due to increased public security efforts -- is always present in a discussion about surveillance. Is there a point at which facial recognition and biometric technology infringe on personal freedoms and the right to privacy? Is blurring faces enough? Are there places where surveillance should not be allowed?

I don’t think facial recognition and biometric technology necessarily infringe on privacy. It is certainly possible to imagine applications where privacy concerns don’t arise. However, for the security and surveillance applications that have been at the forefront of public discussion, the tension with privacy is fundamental. The whole purpose of identifying someone is to connect them with some other information, and the nature of that information is a major issue. We can think of face recognition as a key to identity. But focusing on the key tends to distract us from other questions, like What’s behind the lock? As more and more information is stored behind the lock, the reliability of the key becomes increasingly important. As does the question of who has access to the key.

The practice of blurring or pixellating faces to protect identity (as in Google Streetview) is often poorly informed. Although such manipulations can make it more difficult for observers to identify people, this is only the case when the observer is unfamiliar with the faces concerned. When the observer is familiar with the face, blurring or pixellating the image does surprisingly little to impede identification.

People have very different ideas about where surveillance should be allowed, and which places should be out of bounds. I don’t really foresee any wide agreement on the extent of coverage that is desirable or acceptable. The general trend is for rapid expansion, especially in the US and the UK, but my impression is that this trend is not driven by public demand.

The UK has over 4 million cameras -- that's one for every 14 people in the country and 200,000 in London alone. Chicago is working to improve its 'Virtual Shield' and include the entire metropolitan area in its surveillance grid to cut down crime. Yet, criminals still often get away with murder -- literally. Are expectations set too high? Are surveillance grids more of a scare tactic in preventing crime from happening rather than proactive in catching criminals in the act?

It has been known for some time that the unprecedented CCTV coverage in the UK has had little or no effect on crime rates. A recent Home Office report revealed that only 3% of crimes were solved using CCTV footage, and suggests that simple improvements to street lighting would be more effective. Part of the problem is that it is unrealistic for police to monitor CCTV footage on the scale that it is produced. But more importantly, little thought has gone into the use of CCTV evidence in court. It has only recently become clear how poor humans are at matching unfamiliar faces, even when the images are far higher quality than could be obtained from CCTV. We’ve already looked at machine performance in this context. Establishing a match that will stand up in court is very difficult indeed.

The deterrent argument is interesting because the figures imply little or no deterrent value in CCTV. The standard explanation for this is that people assume the cameras are not working, which is a reasonable inference to make if they are not reducing crime. However, I wonder if there is also a paradoxical effect of increasing coverage. After all, the more cameras there are, the less likely it is that any particular camera is being monitored.

Do New Biometric Restrictions Make Video Surveillance Illegal?

This past October, the State of Illinois passed the Biometrics Information Privacy Act, which joins similar laws previously enacted in Texas and Virginia in imposing restrictions on the use of certain biometric data. Essentially, companies can no longer collect customer “biometric identifiers” without first receiving written consent. A "biometric identifier" is defined as "a retina or iris scan, fingerprint, voiceprint, or scan of hand or face geometry."

Wait a minute! Doesn’t every video surveillance system essentially “scan” people’s faces?

Well, they certainly do record pictures of people’s faces -- lots and lots of pictures, in fact. So, how exactly is a series of pictures any different from a scan? Did Texas, Virginia and Illinois just outlaw all facial video surveillance?

As you might expect, upon closer inspection it turns out that the new legislation DOES NOT outlaw traditional surveillance. Some laws specifically exempt video surveillance, and others protect it by definition. A picture of someone, it turns out, is not a “biometric identifier,” which is strictly defined to be an actual biometric measurement tied to an actual identity.

Because surveillance video captures “images” of people and places, and not individualized measurements, and because those images are not each individually identified, they are not technically biometric identifiers.

So what about Minority Report stuff?

Can businesses install retinal scanners and toss ads at you based on who you are? This would seem to be outlawed, unless permission is obtained from the individuals being scanned. And large retailers that serve millions of customers each year aren’t likely to ask each person that passes through their doors to sign a consent form authorizing a biometric scan. So, I think Minority Report stuff may be officially off the table.

Is 3VR video search and facial surveillance affected by these laws?

No. In fact, 3VR’s technology might be the only system built to survive harsh biometric prohibitions like what we’ve seen in Texas, Vermont and Illinois; its unique approach and privacy protections give it the same exempted status afforded traditional video surveillance recording under the law.

Unlike prohibited technologies like hand, iris or fingerprint scanners, 3VR exclusively records and processes only what is permitted under the law -- standard surveillance imagery of scenes and people. When a 3VR SmartRecorder does a search or performs a facial matching analysis, what the system is really doing is comparing actual images to each other using a mathematical language that has nothing to do with any underlying identity or physical measurement derived from the image.

The distinction might seem subtle at first, but 3VR’s approach offers several important privacy protections, most notably the fact that none of the data that 3VR uses to catalog surveillance video contains any information about who a person is or what they actually look like.

More importantly, a 3VR’s searchable surveillance index relates recorded content only to itself and compares only images collected by the system to each other. This is very different from a biometric scanner that seeks to relate every one of its scans back to a specific identity. A 3VR only relates surveillance to identities under very limited circumstances, such as when a watchlist match has been triggered or during an actual investigation. Thankfully, in both cases, bad guys are afforded much less protection under the law than the general public.

What’s next?

In the future, expect privacy and security law to conflict more and legislation to become increasingly complex. There will no doubt be more laws, like the “Know Your Customer” provision of the Patriot Act demanding more aggressive data collection and customer identification, and there will also be many more laws like those recently passed in Texas, Illinois and Vermont protecting consumer privacy. Ultimately, we need both kinds of laws, but to navigate them, we’ll need new technologies and approaches specifically designed to ensure security without sacrificing our privacy. You can look for that here.

A Billion Dollars for Biometrics

The FBI Biometric Center of Excellence is currently working on a Next Generation Identification (NGI) system that will combine iris scans, facial imaging, palm prints and fingerprint identification technologies in one, multi-dimensional system. The platform is intended to succeed the outdated Integrated Automated Fingerprint Identification System (IAFIS) put in place in 1999.

The cost on this ten-year investment? A small price tag of $1 billion dollars.

Plans for NGI include increased fingerprint storage capacity and accelerated fingerprint processing times for high priority criminals, as well as the creation of a special database of names, which will include sexual offenders, wanted persons and terrorists.

"NGI will give us bigger, better, faster capabilities and lead us into the future. We have added additional capabilities to our current system, and are working with the Departments of Homeland Security, Defense, State, and the International law enforcement community in making our communities safer," said Thomas E. Bush, Assistant Director of the FBI's Criminal Justice Information Services (CJIS) Division.

Of course, privacy concerns abound in relation to the amount of personal data stored and collected on average American citizens. The FBI Web site proactively addresses these concerns and notes that the NGI system refuses to expand the categories of people from whom they collect data already, but will rather collect additional data on criminals and terrorists. As Bush said, "The privacy and security of the system is extremely important and we have to ensure the relationship with privacy advocacy groups, and make sure it's not accessible to unauthorized persons."

Eternal viligance.

DHS Privacy Workshop Delivers Best Practices

As sophisticated surveillance grids continue to pop up nationwide, it was only a matter of time before a group of experts convened to discuss the trajectory of Closed Circuit Television (CCTV) technology and explore best practices when installing the technology. While the Department of Homeland Security (DHS) hosted its "CCTV: Developing Privacy Best Practices Workshop" over a year ago, only recently did the executive report summarizing the results of the conference surface.

As the Executive Summary states, the report serves as a best practices guide to avoid crossing privacy boundaries when deploying of CCTV systems as systems become more pervasive in both the private and public spheres. However, rather than advocate installation, the report serves as an objective guide to the various concerns to consider when first pondering the massive investment involved. Attending academics, researchers and government officials deliberated the importance of ensuring safeguards and civil liberties prior to deployment:

"These resources may be useful in helping government agencies build privacy and civil liberties protections into the design and implementation of a CCTV program. Failure to address privacy and civil liberties can undermine public support for the use of CCTV and erode confidence in the government's ability to protect privacy and civil liberties while protecting the Homeland."

Six panels ranging from an in-depth look at CCTV technologies being used today to general international surveillance practices culminated in suggestions on how to comply with major concerns based on the Fair Information Practices Principles (FIPPs), a set of principles that have long served as a framework for protecting privacy within the United States and abroad since 1973.

With the Obama administration taking office earlier this week, we won't know the specifics on his agenda to "Protect Critical Infrastructure", but the new President's stated objectives do include improving airline security, monitoring US ports, safeguarding public transportation and improving border security.

With a more sophisticated surveillance blanket covering the US, a best practices guide is certainly necessary to preserve individual privacy and civil liberties. In following through on his agenda action items, Obama's cabinet might want a copy of this 66-page manual delivered to the White House sooner rather than later.

Getting San Fran Surveillance Right

Surveillance cameras mounted at First and Mission Streets in San Francisco

Big news. Despite doing many things wrong, San Francisco has still managed to reduce non-violent crime by an average of 24% in areas where they deployed surveillance cameras. Yet, a main objective to reduce violent crimes has yet to be accomplished. But, what would it mean if the city did everything right? That question wasn't addressed in a substantial piece in the SF Chronicle today, but the statistics the article provides are making for some pretty interesting food for thought.

The article cites a recently released study that shows that San Francisco's surveillance program has failed in its primary goal of reducing homicide and other violent crime, although it has succeeded in reducing such lesser offenses as burglary, pickpocketing and purse-snatching. While this statistic is important, it should be noted that San Francisco's cameras are not monitored in real time, but rather the footage is ordered by investigators only after the crime is reported; privacy controls prevent city police from monitoring the city's surveillance cameras in real-time. The privacy controls are so stringent, in fact, that I recently wrote about a man forced to spend 69 days in a San Francisco jail waiting for access to surveillance video footage that ultimately exonerated him. The city can and should be doing much better.

Protecting Privacy With Technology vs. Non-Use

There is very little excuse these days for deploying city surveillance and then not monitoring cameras or granting police and attorneys access to footage that helps convict or exonerate. That's because identity protection and search engine technology exists today that enables police to actively monitor and query city surveillance systems without sacrificing privacy. The ACLU recently wrote about one example of this kind of technology from 3VR Security, and these kinds of approaches are getting better every day. Had San Francisco deployed its surveillance system with this kind of privacy protection technology, it would not have been necessary for the city to take the extreme step of locking police out of its surveillance infrastructure.

The Value of Image Quality and Data Storage

In San Francisco, image quality and data storage are two other major concerns. The cameras the city purchased and installed for $700,000 in 2005 are high-resolution, but produce only three frames per second and thus the footage appears choppy (movies are shot at 24 frames per second), making it difficult to identify even license plates – forget about trying to recognize a repeat car thief or vandal. Again technology may provide an answer for the city. Instead of storing everything and an equally poor frame rate and video compression level, why not use face and license plate detection algorithms to smartly track and store relevant information when, and only when, it is detected. Intelligent approaches that separate what is important from what is not have been proven to dramatically improve both the quality and storage longevity of surveillance archives.

A Fully Integrated Platform

Traditional surveillance systems often require an army of people to patrol the system and report suspicious activity to a security manager. However, by using networked systems that apply analytics and reporting, municipalities can service security functions of hundreds with an army of one. By incorporating the latest innovations in face recognition, license plate recognition, video motion alarms and other new technologies to make systems more effective and efficient, municipalities can vastly improve the results of their security systems. As John Honovich points out in a recent contribution to Government Security News, "cameras enable officers to assess and respond over much greater areas at much lower cost. Even in the U.K., famous for its mass public deployment of surveillance cameras, video surveillance costs are only one one-hundredth of police costs."

Creating A Better Solution

Ultimately, video surveillance is nothing new and many cities are capturing hours and hours of footage on a daily basis. However, that endless volume of footage must be managed to be valuable, and even if a surveillance system features the highest quality cameras, the best image resolution, and a fully integrated network, the resulting video footage isn't going to be useful unless it can be stored, analyzed and searched – in real time. Search and analytics would dramatically improve the usability of the footage and make the surveillance network that much more effective in achieving its objective of crime reduction.

With the right ingredients, municipal surveillance has the potential to significantly counter criminal activity, but it takes that correct formula and a scientific approach to have such results. Despite its inefficiencies, the San Francisco municipal surveillance system isn't a disaster by any means. If Newsom can bolster its allotment of the 2009 budget and add the aforementioned features into the existing infrastructure, San Francisco has the opportunity to have one of the largest municipal surveillance systems out there operating smoothly, meeting and perhaps even surpassing expectations.

Adobe Video-Object Manipulation Project Holds Significant Promise

Video Object Manipulation may be going mainstream.

Once the exclusive domain of computer vision geeks in the Department of Defense, and more recently Hollywood, Adobe has now got its hands on these algorithms and every YouTuber with an HD camera will soon have the tools to not only mark up video in fantastic new ways, but literally to bend reality.

What does it mean for security when just about anyone can add, remove or alter people and objects within a video stream to create a perfectly realistic video of something that never happened? Well, in my opinion it’s mostly not good.

But, there is a bright side -- the entrance of consumer-focused companies like Adobe into this industry is likely to help the security professionals as much as the criminals. More experts, new approaches, better tools and easier-to-use interfaces are a welcome addition to security’s video analytic offerings, and can certainly help security personnel fulfill the promise of improved surveillance.

Face Swapping 2.0 - Now Mostly Prettier

In July, I made a post regarding new technology from Columbia University that swapped faces in photos and videos with generic amalgems drived from a photo database. The technique, which had potential privacy benefits...and even certain Hollywood applications...suffered from one major drawback. The modified faces were really really really ugly.

See below (From Columbia University Research Paper):



Tough to stomach, I know, but these results were bound to be approved someday, and thankfully that day is here. The New York Times is reporting that computer scientists in Israel have taken a more beauty-centric approach to face swapping that simply replaces faces in images and video with more idealized representations of the same faces. Their software applies an algorithm covering over 200 different facial measurements that brings a target face closer to the ideal. In most cases, this approach goes well beyond simple airbrushing, dramatically altering a person's appearance and perceived identity.

See below (From Lars Klove for the NYT):



I am not sure either technology is perfectly suitable for privacy protection in video surveillance just yet, but given the choice between the two approaches above, for aesthetic reasons, I'd go with the latter.

Amazing Video Enhancement Technology

Researchers at the University of Washington recently released a video showing a series jaw-drapping video enhancements now possible using various image analytic and modeling techniques. In one demo of interest, simply by mixing a few high-resolution photographs of a scene with a lower quality video stream, the scientists were able to dramatically enhance the quality and resolution of the complete video...by 4x.

The technology could also be used to seemlessly remove private content from surveillance video, truely making proctected images of people or objects invisible. That's probably welcome news to some, but disconcerting to law enforcement professials who already have significant concerns about the reliablity of photos presented them. Video has been generally thought of as much harder to manipulate...no more.

Face Swapping Still Not Pretty

Not since John Travolta and Nicolas Cage swapped mugs in the 1997 non-hit Face/Off, have I been so intrigued/disturbed by the prospect of changing faces. Researchers at Columbia University have developed a system capable of quickly replacing and/or obscuring faces in photographs with new composite or synthetically generated images. Unlike our technology at 3VR and that at Google, which only blurs faces, this technology actually changes them...with mixed results.

Here is an example:

The authors see several applications to this technology from protecting privacy in public images to Hollywood special effects. But, as you can see above, the researchers' favorite application seems to focus on making famous faces less attractive.

Too Much Information Makes Us Less Secure

How many cameras do you need to have before you can't see anything? How much video do you need to store before you can't find what you were looking for? Whatever that level is, the Brits passed it a long time ago when, as the author of this piece quips, the answer to the needle in the haystack problem became to "collect more haystacks."

"The answer in both America and Britain has been to collect more haystacks: useless, indiscriminately acquired information on people who've done nothing to arouse suspicion," he writes. "We even inveigle our citizens to become amateur curtain-twitchers and pecksniffs, demanding that they report "suspicious" activity to the authorities. Between DNA databases, mandatory fingerprinting for visa seekers, CCTV carpet-bombing, and Oyster card data, we've never collected more "security" information than we do today. But does this really make us secure? Is it possible to know too much?"

I think the question is not "Is it possible to know too much?", but rather "Is it possible for too much information to overwhelm our ability to know anything at all?". In which case the answer is a resounding yes! The delta between what security professionals can collect today and what they can process is called the "Security Gap" and it's getting bigger every day.

But, I'm not sure it's time to hoist the white flag just yet, however. A few years ago, after the bombing in London, closing the Security Gap meant assigning over one thousand MI-5 agents to manually review surveillance video. The process took over 6 weeks. Today, using new search and video analysis technology, just a handful of those agents could have probably completed much of the same work in a few days. Search technology in particular is rapidly closing the gap between the data we can collect and store and what we can "know". It's not unlike what happened with the Internet when tools like Google have made sifting through billions of images and pages as simple as typing a few key words. Suddenly this huge heap of information that had been sitting there unused by most became hugely useful when we could quickly and easily sift through it.

The prospect of these same advances in video search being applied to security is likely to greatly tame America and Britain's current jumble of haystacks, but such technology will of course bring with it a new set of problems. For those who have spent years working to close the Security Gap, I am reminded of another question about a dog chasing a firetruck.

"What's he going to do if he catches it?"

Surveillance as Handy Marketing Tool

A lot of new products are coming out that help companies use existing CCTV surveillance technology for marketing purposes--capturing everything from basic data about how many people stopped at a promotional display to more advanced details about particular customers. In addition to the camera-and-box equipped billboards being piloted by TruMedia and Quividi, Google recently announced its partnership with a company called Xuuk to produce a palm-sized camera called the Eyebox that will track how many times people look at both billboards and products in stores. The idea is to provide brick-and-mortar stores or companies the same tracking abilities in real life as they have with Google ads online. Personally, I think using face recognition with an already existent system (like, I don't know, a 3VR system!) makes better financial sense than spending $25,000 for a separate system and cameras, but even above and beyond that, using surveillance systems as marketing tools doesn't exactly help to assuage the public's "Big Brother" concerns about surveillance.

Finally, A Balanced Take on Security Cameras

Okay, in this case, Fox News actually is providing a "fair and balanced" take on things. Apparently security camera neither put a complete stop to crime nor are completely useless privacy invasions. In known, high-crime areas, the deployment and use of surveillance systems can actually help fight crime. Maybe not all crime, but at least enough to make it worth doing. Obviously, privacy is important. I get that. But I think the way to get folks on board with privacy protection is not to make unfounded statements like "cameras have absolutely zero effect on crime." That's just absurd and patently false. Extreme statements like that will just keep people on opposite sides, not solving either the privacy problem OR the security problem.

On a lighter note, with the suggested increase of cameras in Lynwood, Paris Hilton's crime spree may finally come to a halt.

More Cool Privacy Tech

The new privacy technology we are working on at 3VR does more than simply blur people's faces. Here is an example where full images of surveilled people are actually scrambled and encrypted before they are displayed for monitoring security personnel.

The image on the left is from the original video feed. The second image is from a scrambled and encrypted version of that same feed.

Though with the encrypted feed it's possible to detect loitering, fighting, and many other behaviors of concern, identity information is totally protected...that is, until an authorized user chooses to decrypt the feed. BUT then that action is controlled and logged, and an alert can even be generated, to ensure that the surveillance system is not being abused or misused.

The Costs and Benefits of Face Recognition

Most people think face recognition is great when it's catching baddies and weird or invasive when it's tracking innocent people. The technology has sparked debate after debate over whether privacy or security is more important. Honestly? I think we can have both.

First off, people need to understand what facial recognition can and can't do. This ABC story about a face rec system analyzing someone's face and telling a clerk that they're underage is bogus. If the kid had been in the store previously and been busted for buying liquor then yes, the system could alert the clerk, but no one is suggesting that face rec can tell the difference between 17 and 18 any more than a human can.

One thing face rec can do, which hasn't gotten a ton of press, is include simple privacy measures. We have been working on this at 3VR and I wrote recently about a team in Canada that's working on a similar project. Basically, software engineers can write a password-protected program that blurs faces and when an incident occurs, an investigator can unblur faces in particular pieces of video. This way, while people may still feel uncomfortable about being on camera, at least they will not really be watched unless they happen to be present during a robbery or some other incident, in which case they'll typically be glad the cameras were there to help catch the bad guys. Also, because this type of application can also have auditing capabilities written into it, it provides a crucial and often overlooked capability: a way to "watch the watchers," if you will.

The same could feasibly work in the grocery store situation described in the ABC story - if someone was a match with a suspect in the database, then the system could alert the clerk. For everyone else, faces could be blurred, and if someone is caught buying liquor or cigarettes underage, then the store manager could unblur the face and save it to the suspect list.

There has also been a lot of press recently about the rise of surveillance that risks privacy without actually improving security. I agree. Thing is, the bulk of new cameras installed are meant to catch traffic violators and raise money for municipal governments, not improve security. These cameras misfire fairly often, sometimes costing a city more than they're worth, and invading citizens' privacy for no good reason.

During the bombings in London, however, investigators were able to use video footage to find their suspect. If they had had face recognition and video search capabilities, that investigation would have been far shorter. And as banks have begun installing surveillance systems, they have seen a marked increase in the number of fraud cases they're able to solve.

Surveillance clearly has a place in modern society, but I do think that the industry needs to continue to work towards securing both people and their privacy.

Should Red Light Cameras Be Stopped?

Theoretically red light cameras are great - who wants people running red lights? The problem comes when people fail to differentiate between red light cameras and security cameras and wind up throwing the baby out with the bath water. While they appear to be associated with safety, red light cameras in fact exist to generate revenue for cities, and they're largely ineffective at actually reducing accidents. Security cameras, on the other hand, exist to protect cities and people. To lose the latter because of the annoyance and poor performance of the former would be silly in the best case scenario, tragic in the worst. Unfortunately, that is exactly what may happen in cities throughout the country as citizens protest against red light cameras and the trade-off of privacy for city funds that they require.

Privacy-Enhancing Video Technology

It's great to see people trying to address the privacy vs. security question with new solutions instead of complaints on either side. Today, video surveillance is not particularly private nor effective, and we really need both. A new technology developed by researchers at the University of Toronto adds a sort of face blur filter to security video that can be reversed with a decryption key. So, the faces of the innocent remain fuzzy and out-of-focus, while investigators zero in on suspects.

The University of Toronto tech was most likely at least partially motivated by changes in Canadian privacy laws. Even without such laws, technology like this is important and certainly helps to address the privacy issue. At 3VR we are currently working on our own privacy enhancing technologies that also incorporate advanced facial recognition techniques. The technology helps to hide the faces of the innocent while at the same time identifying bad guys, providing a solution that gives equal weight to privacy and security, which is something I think should be at the center of any city surveillance deployment.

Rethinking Surveillance

Video surveillance has become a fact of everyday life. Each time you withdraw cash from the corner ATM, travel through an airport or visit a national monument, your image is probably being recorded. But you may be surprised to learn that there are no federal laws governing how these images can be used, where they should be stored, with whom they may be shared and when they must be destroyed. In this age of YouTube, TMZ and "Cops," it's hard to know where your image might reappear.

The laissez-faire approach of our national legislators is no longer an option. As an increasingly sophisticated surveillance blanket covers more of the United States, we need federal laws to preserve an individual's right to privacy while setting principles governing the use of closed circuit television and other surveillance technologies for bona fide security purposes.

Surveillance technologies will continue to gain in capability -- and become more intrusive. Issues of privacy and public surveillance may appear vexing, but the United States must move forward with laws to effectively adapt to the inevitable spread of this technology. If the public is to trust business and government to watch over us, we need to follow the lessons of Britain and protect video images as we do other private data.

FBI Wants Palm Prints, Eye Scans, Tattoo Mapping

CLARKSBURG, West Virginia (CNN) -- The FBI is gearing up to create a massive computer database of people's physical characteristics, all part of an effort the bureau says to better identify criminals and terrorists. But it's an issue that raises major privacy concerns -- what one civil liberties expert says should concern all Americans. The bureau is expected to announce in coming days the awarding of a $1 billion, 10 year contract to help create the database that will compile an array of biometric information -- from palm prints to eye scans.

Given the recent request for a $3 trillion budget, $1 billion doesn’t seem like all that much anymore.