VideoSurf: New, Genuinely Radical Video Search
I’m generally not given to hyperbole or breathless enthusiasm when writing about a new search service. Quite the opposite, in fact—over the decade I’ve covered search I’ve seen so many so-called “radically new” approaches that are simply updates or spins on existing technologies that I’ve grown a bit jaded.
But not after seeing a demo of VideoSurf. I’ll echo the headline for this post: VideoSurf is one of the most innovative, radically different approaches to video search (or any kind of search for that matter) that I’ve ever seen. Even better: It delivers highly relevant search results—something most video search services fall sadly short on for many types of searches.
What’s different about VideoSurf, and why am I so excited by it? Read on.
The problem with video search
Search engines generally do a good job indexing text, but face greater obstacles and challenges in properly “understanding” and indexing video (or any kind of multimedia content, for that matter). Unlike text, which is made up of consistent patterns of letters, words and sentences that are relatively easy to understand, images are essentially random collections of different colored pixels that form a gestalt that our eye processes and sees as a meaningful whole, but are difficult for a computer to “understand.” People are working on computer based image retrieval systems, but they’re still relatively primitive and unreliable, as I’ve written before.
Video poses even more challenges than still images, because videos are made up of a series of rapidly changing images usually combined with some kind of audio soundtrack.
Most video search services rely on metadata—titles, tags, filenames, descriptions associated with videos, and so on, to determine the contents of a video and compute its relevance. The problem with this secondary data is that it’s not always accurate (and it’s very easy for spammers to manipulate). Some services, like blinkx and EveryZing go beyond this, doing speech to text conversion to help improve relevancy. This works well for some kinds of content (particularly news or documentaries) but can have problems with other topics (there aren’t any spoken words in many of those Mentos and Coke videos, for example).
How VideoSurf is different
VideoSurf is a computer vision search engine that processes all of the kinds of information most video search services do, but then goes a step further, applying a proprietary process using “multigrid fast computation” and some heavy-duty computer processing power to analyze videos, identify people, and extract all kinds of additional information directly from the video itself. Until I saw the demo, I thought this type of technology was still years away.
VideoSurf was founded by the folks who built one of the first real-time travel information sites, Farechase, which was acquired by Yahoo in 2004. The VideoSurf team applied many of the lessons they learned in working with real-time info in the development of VideoSurf.
According to Lior Delgo, CEO of VideoSurf, the company focused on three areas of innovation:
Visual search. You can use keywords to search VideoSurf, but this is really just a starting point. VideoSurf results provide a rich set of thumbnails that go beyond what you usually see in video search results. Typical video search results have been created using techniques like scene detection, key frames and so on. Instead of those, VideoSurf analyzes video to try to identify the most “unique moments” in a video and shows you 10 or so of those scenes for each result. Click on one of the scenes and the video starts playing from that point—but even better, you can use a thumbnail to launch an entirely new search.
See the top row of thumbnails? Those are people that VideoSurf has identified as being the most relevant (meaning they appear in more videos than others) for your query. Click on any of those images and your results will change to feature those people. You can also restrict results to just faces in the results for individual videos, which appear below the top row of thumbnails.
Visual discovery. Because VideoSurf “understands” who people are and what they look like, you’ll often see images of different people when you limit your search to an individual. Why? Well, actors and performers work with one another all the time on different projects. This takes Kevin Bacon’s “six degrees of separation” idea and lets you discover those connections in an incredibly easy, intuitive way.
Visual navigation. Because VideoSurf shows you a set of the most important moments in a video, you can quickly skip to the point in the video that you’d like to see, rather than having to either watch the video or use the slider to jump ahead to what you hope is the most relevant part.
VideoSurf also gives you a set of tools that let you limit your search by category (content type, music, movies, funny or TV), or to your favorite video sources, including YouTube and Hulu, as well as major TV networks like Comedy Central and ESPN.
VideoSurf appears to have taken video search to the next level. I asked Delgo about the company’s monetization plans, and he told me that the company had a well developed model, but that they weren’t ready to talk about it yet. I wouldn’t be surprised if the company wasn’t purchased—soon—by one of the bigger players wanting to leapfrog the competition.
If you like to search for videos online, VideoSurf almost certainly will become an essential part of your searching arsenal. And good luck breaking away from it once you start exploring…