Computer Vision Resarch Goes Virtual

Sometimes the real world just isn’t real enough. That’s often the case in computer vision application development where super smart PhDs seek to create algorithms and technologies to track and classify people or objects within a video stream. Believe it or not, some of the same neural networks that catch bad guys today got their start by tracking frantic scientists running around their labs, offices, and dorm rooms.

But ObjectVideo thinks there is a better way…at least to start. Using technology from the videogame Half Life 2 , they have built a Virtual Video Tool that can be used to create “virtual surveillance” cameras.

The ObjectVideo Virtual Video (OVVV) Tool generates realistic video from simulated cameras in an interactive virtual world. This tool is free and is based on a modification (aka 'mod') of Half-Life 2, a commercially available game from Valve Software. Our hope in distributing this tool is to stimulate computer vision research in areas that cannot rely on canned video (eg. active tracking) or when large quantities of ground truthed video is unavailable or impractical (multi-camera installations, public spaces, the list goes on!).

The fact that virtual cameras are generally thought to lack the video noise and other artifacts found in real-world cameras, doesn’t prevent this tool from providing real benefits to students and researchers. Today gaming engines are so realistic and of such high quality that the line between real and virtual is being blurred. And, as OV points out, virtual cameras provide another benefit that’s impossible to achieve with real world footage: ground truth data that can be incorporated into the training process. Because virtual cameras are built on models of scenes where ever person and object and color and angle are actually known, a researcher always knows, without guess or estimation, just how well their computer vision algorithms are deciphering a particular video stream.

Beyond that, ObjectVideo has created most of the environments, models, and camera option necessary to test every conceivable surveillance variation during the testing process. Even blur, noise, and even lens and PTZ effects can be simulated with relative ease.

Virtual surveillance video is not just a great tool for computer vision researchers, it’s also an incredibly interesting area of research in itself. The folks at Valve Software have my appreciation for opening their platform enough to enable this kind of work. But believe it or not, Valve’s Half Life 2 is already almost 4 years old. Maybe ObjectVideo’s next endeavor can be a Crysis mod. That would be something. And next year there will be something else…even better.