Google's DeepStereo makes Street View 'tourism' more realistic

by Mariella Moon | @mariella_moon

Who here uses Street View to do some virtual traveling? It's not a shabby option if you want to look at places you might never visit within your lifetime, but Google thinks there's still a better way to simulate real-world tours. A team of the company's researchers has developed a technique that uses Google's experimental machine vision algorithm called DeepStereo to transform Street View panoramas into seamless virtual tours. See, Street View photos don't usually capture every detail of a location -- there are always a few frames missing in order to construct convincing digital reproductions of places like museums and houses. DeepStereo can synthesize those missing frames based on the ones that go before and after them, giving Google the ability to create realistic virtual tours.

In order to "train" DeepStero, the team had to feed the algorithm with images captured out of a moving vehicle; after that, it was able to start recreating images. It's still not perfect at this point -- some objects like trees or grass are hard to synthesize and details vanish from the recreated frame if the machine lacks pertinent details. In addition, it takes as long as 12 minutes on a powerful workstation to build a single synthetic frame. DeepStereo's obviously still a young technology, but the team believes it could be used not just to create virtual tours, but to generate environments for movies and virtual reality content in the future.