An Unprecedented Video Glimpse Into How Google Crafts Its Search Results

How does Google create its search algorithms, its recipes for making the listings and search results you get? For the first time, the general public gets a glimpse into the decision making process, through a video that Google has posted of one of its search quality meetings.

The Video

I was fortunate enough to witness one of these meetings myself recently, and I’ll share some thoughts about that and further context about what happens in them below. But first, the video, for those who may wish to dive right in:

Search Quality Meetings

Every Thursday, members of Google’s search quality team review proposed changes to how Google search operates. New features and tweaks are presented, along with data on how they’ve performed in real-life “sandbox” testing. After discussion, good changes win approval and go live for everyone. Changes with poor performance are held back, perhaps to be reworked or abandoned.

Until now, few outsiders have ever seen one of these meetings, among them three journalists: Saul Hansell, of the New York Times, who described one in a 2007 article; Steven Levy, who touches on them in his excellent book “In The Plex,” and myself.

I got to attend one last October, and it was one of the most fascinating, interesting things I’ve witnessed in my nearly 16 years of covering search. I watched many different features get evaluated, with discussion and debate flying back-and-forth among those gathered in the crowded room as well as those interacting through video conference, or “VC” at Googlers call it internally.

I’ve been meaning to make time to write-up what I’d witnessed, but the video saves me from trying to describe what I saw. Anyone can see it for themselves.

As well-versed as I am in search, understanding some of what was discussed was difficult because Google’s search team almost speaks a different language loaded with their own custom jargon.

Fortunately, Google has taken care to annotate the video to make some of the lingo that flies about more understandable to the layperson, as well as explain more about the particular feature that was being discussed, a change to how it handles spelling corrections.

Think VH-1’s Pop Up Video meets Search Quality Meeting:

The video you’ll see is true to the type of meeting I witnessed, which lasted about an hour and covered several features.

Focus On Users, Detail

Three things struck me most about having watched one of these meetings.

First, there was never any discussion about how any particular change might favor a Google property or revenues. The cynical might say that of course an outsider would never see that type of thing discussed. I’m more optimistic. Nothing felt stage-managed or contrived about the meeting I viewed. Rather, it seemed natural and that, as a routine thing, the focus is on how Google search can work better for the user.

Second, it was amazing to see the level of detail Google considers with search. Running a search engine to just get relevant results is hard, when you’re dealing with sorting through billions of pages, some of which are crafted purposely in an attempt to game your relevancy system. But Google ponders search features that go to incredibly micro-level of detail with search. It’s this type of attention that has created its powerful lead in search.

Third, I was stuck by the similarity between this meeting and another type of meeting that few in the general public have ever witnessed: a newspaper budget meeting.

Like A Newspaper Budget Meeting

I came out of daily newspaper journalism. Each morning, most papers will hold a budget meeting to review upcoming stories. What looks good? How’s the progress on a particular article? Should something new be covered or not?

What appears in a newspaper, online or in print, doesn’t just magically emerge because some impartial “news algorithm” comes along and determines what are the best stories that should be covered. Rather, there’s a human judgement involved, where the human editors hope they’re making decisions in the best interests of their readers.

I’ve long written that search engines are similar to newspapers in that they have editorial results, where they are trying to decide what “stories” make the most sense to list in response to any search.

In fact, when the New York Times suggested that maybe Google’s ranking systems should be subject to government review, I wrote The New York Times Algorithm & Why It Needs Government Regulation, to flip things around and provide some perspective at the First Amendment issues this would raise.

Hopefully, the video will give both the editorial writers at the New York Times and people in general a better idea of how Google works. That’s part of Google’s goal here, a reaction to the accusations that its ranking system is too much a closed box. The Search Quality Highlights series, a monthly recap of search changes launched last December, is another example of this.

The openness won’t be open enough for some, but it’s definitely a positive move in my books.

Bonus Features

Also referenced by the video is a more formal look that Google produced last year about how search features are created and evaluated:

[youtube width=”560″ height=”315″]https://www.youtube.com/watch?v=J5RZOU6vK4Q[/youtube]

If you’ve got an hour-and-half to invest, you might also find an interview I did last year with Google’s Amit Singhal, Ben Gomes and Matt Cutts of interest. The three oversee different aspects of Google search. This was the first time all three had ever talked together about their work on stage:

[youtube width=”560″ height=”315″]https://www.youtube.com/watch?v=qlAydU6vBZo[/youtube]

Google also posted more about today’s video on its Inside Search Blog. For further background about how Google and search engines in general create search results, see some of our past articles below.

Related Articles

• The Periodic Table Of SEO Ranking Factors
• Google 2.0: Google Universal Search
• How Google Instant’s Autocomplete Suggestions Work
• Google Search Algorithm Change For Freshness To Impact 35% Of Searches; Twitter Firehose Remains Off
• Improved Snippets, Rank Boost For “Official” Pages Among 10 New Google Algorithm Changes
• Google’s January Search Update: Panda In The Pipelines, Fresher Results, Date Detection & More
• Google Confirms Panda 3.3 Update, Plus Changes To How It Evaluates Links, Local Search Rankings & Much More
• Infographic: The Google Panda Update, One Year Later
• Keeping Up With Google: Bing Launches New “Search Quality Insights” Series
• An Interview With A Google Search Quality Rater
• Google’s Results Get More Personal With “Search Plus Your World”
• Two Weeks In, Google Says “Search Plus Your World” Going Well, Critics Should Give It Time