Researchers from Stanford University have developed new artificial intelligence software called ZunaVision, which can place a still image or video on almost any surface inside a video clip. For example, in a video displaying a painting of the sea hanging on the wall in a living room, the picture on the wall can be edited and replaced by a video of Barak Obama giving a speech.
 |
ZunaVision – easily place any video
or image inside your home made
video movie (Credit: ZunaVision) |
|
The user selects a polygon in the video where the new object will be inserted. Then the intelligent software analyzes the videos to match color, texture and lighting differences. Once the video or image to be inserted is cleverly modified to blend into its surroundings, the final product will look like a composite part of the original video. The researches even took into consideration shadows that might fall on the inserted image, making the results as authentic as possible.
With the aim of producing realistic results, the algorithm named “3D Surface Tracker Technology” is able to deal with people (or other “occluding objects”) passing in front of the inserted object. The software separates objects in the foreground and background by keeping track of which pixels belong to the original video and the inserted object. Therefore, as a person walks in front of an inserted video, this video is able to disappear and then re-appear as it is passed by.
The research team, consisting of computer science graduate students Ashutosh Saxena and Siddharth Batra, and Assistant Professor Andrew Ng,
also overcame the challenging issue of camera movement. As the camera pans and zooms, the area containing the inserted object should move and change in accordance to perception. The algorithm can transform the embedded image using shape shifting geometry and by building a model, pixel by pixel, of the area of interest in the video. “If the lighting begins to change with the motion of the video or the sun or the shadows, we keep a belief of what it will look like in the next frame. This is how we track with very high sub-pixel accuracy,” said Batra. Basically, the embedded image formulates an educated guess of the next movement of the wall and tries to move accordingly.
 |
ZunaVision simple and easy
(Credit: ZunaVision) |
|
ZumaVision branched out from an earlier project named Make3D, which was a website that converted a single still photograph into a brief 3D video. This was done by locating planes in the image and computing the relative distances between the camera and the planes. Using this procedure as a starting point, the researchers implemented it into ZumaVision. Saxena says, “That means, given a single image, our algorithm can figure out which parts are in the front and which parts are in the background, and this capability has now been extended to videos”.
The new technology is also able to support unobtrusive marketing by inserting corporate logos or advertisements onto home videos posted online. There are already other such softwares available in the market that have been applied to special effects movies, but most are expensive, time consuming and demand considerable expertise. The ZumaVision, however, is fast and easy to use.
Batra supplied an interesting example of how Zunavision can be put into use: “Suppose you’re a student living in a dorm and suppose you want to show it to your parents [in a video]. You can put a nice poster there of Albert Einstein,” Batra said. “But if you want to show it to your friends, you can have a Playboy poster there.”
TFOT has previously covered
PlayAnywhere – Microsoft’s Interactive Playground, a front-projected computer vision-based interactive table system, which the company says demonstrates the latest improvements in projection technology, and
PICO Enables Two-Way Man-Machine Interaction, which is a field of industrial design that has advanced the idea of interactive displays to a new level by developing a system that allows a two-way interaction between man and machine. You can also check out our article on
Microsoft Surface Computing which enables several people to digitally interact and can perform a variety of tasks using the display.
A hands-on demonstration of the technology can be seen on the
ZunaVision website.
