An Unprecedented Video Glimpse Into How Google Crafts Its Search Results

How does Google create its search algorithms, its recipes for making the listings and search results you get? For the first time, the general public gets a glimpse into the decision making process, through a video that Google has posted of one of its search quality meetings.

The Video

I was fortunate enough to witness one of these meetings myself recently, and I’ll share some thoughts about that and further context about what happens in them below. But first, the video, for those who may wish to dive right in:

Search Quality Meetings

Every Thursday, members of Google’s search quality team review proposed changes to how Google search operates. New features and tweaks are presented, along with data on how they’ve performed in real-life “sandbox” testing. After discussion, good changes win approval and go live for everyone. Changes with poor performance are held back, perhaps to be reworked or abandoned.

Until now, few outsiders have ever seen one of these meetings, among them three journalists: Saul Hansell, of the New York Times, who described one in a 2007 article; Steven Levy, who touches on them in his excellent book “In The Plex,” and myself.

I got to attend one last October, and it was one of the most fascinating, interesting things I’ve witnessed in my nearly 16 years of covering search. I watched many different features get evaluated, with discussion and debate flying back-and-forth among those gathered in the crowded room as well as those interacting through video conference, or “VC” at Googlers call it internally.

I’ve been meaning to make time to write-up what I’d witnessed, but the video saves me from trying to describe what I saw. Anyone can see it for themselves.

As well-versed as I am in search, understanding some of what was discussed was difficult because Google’s search team almost speaks a different language loaded with their own custom jargon.

Fortunately, Google has taken care to annotate the video to make some of the lingo that flies about more understandable to the layperson, as well as explain more about the particular feature that was being discussed, a change to how it handles spelling corrections.

Think VH-1′s Pop Up Video meets Search Quality Meeting:

The video you’ll see is true to the type of meeting I witnessed, which lasted about an hour and covered several features.

Focus On Users, Detail

Three things struck me most about having watched one of these meetings.

First, there was never any discussion about how any particular change might favor a Google property or revenues. The cynical might say that of course an outsider would never see that type of thing discussed. I’m more optimistic. Nothing felt stage-managed or contrived about the meeting I viewed. Rather, it seemed natural and that, as a routine thing, the focus is on how Google search can work better for the user.

Second, it was amazing to see the level of detail Google considers with search. Running a search engine to just get relevant results is hard, when you’re dealing with sorting through billions of pages, some of which are crafted purposely in an attempt to game your relevancy system. But Google ponders search features that go to incredibly micro-level of detail with search. It’s this type of attention that has created its powerful lead in search.

Third, I was stuck by the similarity between this meeting and another type of meeting that few in the general public have ever witnessed: a newspaper budget meeting.

Like A Newspaper Budget Meeting

I came out of daily newspaper journalism. Each morning, most papers will hold a budget meeting to review upcoming stories. What looks good? How’s the progress on a particular article? Should something new be covered or not?

What appears in a newspaper, online or in print, doesn’t just magically emerge because some impartial “news algorithm” comes along and determines what are the best stories that should be covered. Rather, there’s a human judgement involved, where the human editors hope they’re making decisions in the best interests of their readers.

I’ve long written that search engines are similar to newspapers in that they have editorial results, where they are trying to decide what “stories” make the most sense to list in response to any search.

In fact, when the New York Times suggested that maybe Google’s ranking systems should be subject to government review, I wrote The New York Times Algorithm & Why It Needs Government Regulation, to flip things around and provide some perspective at the First Amendment issues this would raise.

Hopefully, the video will give both the editorial writers at the New York Times and people in general a better idea of how Google works. That’s part of Google’s goal here, a reaction to the accusations that its ranking system is too much a closed box. The Search Quality Highlights series, a monthly recap of search changes launched last December, is another example of this.

The openness won’t be open enough for some, but it’s definitely a positive move in my books.

Bonus Features

Also referenced by the video is a more formal look that Google produced last year about how search features are created and evaluated:

YouTube Preview Image

If you’ve got an hour-and-half to invest, you might also find an interview I did last year with Google’s Amit Singhal, Ben Gomes and Matt Cutts of interest. The three oversee different aspects of Google search. This was the first time all three had ever talked together about their work on stage:

YouTube Preview Image

Google also posted more about today’s video on its Inside Search Blog. For further background about how Google and search engines in general create search results, see some of our past articles below.

Related Articles

Related Topics: Channel: SEO | Features: Analysis | Google: SEO | Google: Web Search | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://www.audettemedia.com Adam Audette

    I love that Google released this and invited you, Danny. A great step towards more transparency, and it’s super interesting getting a glimpse into their collaboration. Now if I could only be a fly on the wall when they rolled out Panda…

  • http://www.robhof.com Robert Hof

    Very cool, Danny. One slight correction: I’m a fourth journalist who has attended one of these meetings: http://www.businessweek.com/magazine/content/09_41/b4150044749206.htm

  • Anthony Long

    Great to see a glimpse of Google’s internal algorithm update process, but it would be more useful to get actionable information to improve site quality in the wake of Panda. Something like the adwords quality scores for SEOs & webmasters would go a long way toward producing higher quality results.

  • Tomas Van den Berckt

    It would also be interesting to see a similar meeting from the Adwords team, as i’m sure impact on revenue would be a dominant feature, rather than user experience.

  • http://searchenginereports.net Garry Egan

    I’ve just checked “Be a fly-on-the-wall at a Google Search meeting” off of my life’s bucket list. What an amazing experience. Thanks, Danny & Google Search. This really means a lot to a lot of us. I can’t really put into words my gratitude.

  • http://www.conductor.com Nathan Safran

    This is really a neat view into how Google makes decisions about algo changes. Based on this little snippet I’m inclined to agree with Danny Sullivan’s read on the decision making process being a relatively impartial discussion of what is best for the user (not that there aren’t plenty of other legitimate issues with some of Google’s other decisions).

  • mesel

    Danny, Can you ask for the “launch report” of SPYW? Or the promotion of it’s own travel and shopping products? This is just a PR move, and I very much doubt you or anyone else from PR would be given an inside look to the harder and controversial launches that inevitably get fast-tracked due to exec demands.

  • http://twitter.com/JMGordonLaw TheBusinessProfessor

    Thank you for posting this.  This is incredibly helpful and supports the development of a plan.

    JM Gordon http://www.TheBusinessProfessor.com

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide