Google Caffeine: Google’s New Search Engine Index

Google has just unveiled a “secret project” of “next-generation architecture for Google’s web search“. This new architecture appears to include crawling, indexing, and ranking changes. For the first time, Google isn’t simply incorporating these changes into their existing infrastructure or replacing it. Instead, they’re providing a developer preview and are asking webmasters and power searchers to try it out and give them feedback. Unlike Google’s now-defunct SearchMash, which was intended for search experiments that wouldn’t necessarily be incorporated into Google’s main web search, the caffeine index seems to be an entirely new search infrastructure that will repace what exists now.

Based on the blog post, we can guess that this new infrastructure may include ways of crawling the web more comprehensively, determining reputation and authority (possibly beyond the link graph and what’s typically thought of as PageRank), and returning more relevant results more quickly, although Google’s Matt Cutts told me that the changes are “primarily in how we index”.

While the biggest visible changes in Microsoft’s relaunched search engine, Bing, are user-interface related, Google’s new search is only infrastructure related and includes no UI changes. On first glance, however, the underlying infrastructure changes do seem to have impacted user interface as it relates to universal search (likely because universal results are influenced by ranking and relevance signals). For the sample searches I did, the first ten results were fairly similar, but the existence and location of images, video, news, and blog posts was notably different.

A search for [buffy the vampire slayer] on the new infrastructure, for instance, returns video and news results midway down the page.

Buffy Results - Google Caffeine

A search on the existing infrastructure, however, returns news at the top, video in the middle, and images at the bottom of the page.

Buffy Results - Google

Buffy Results - Google

Undoubtedly, Google Caffeine will cause quite a kerfluffle in the web developer and search engine optimization world and many will dive in to try and figure out the changes. We’ll likely see many a speculative blog post about how best to optimize for the new infrastructure, but my guess is that it likely does what Google search does, but better. And the foundational elements of having a crawlable infrastructure and compelling content remain.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: SEO | Google: Search Customization | Google: SEO | Top News


About The Author: is a Contributing Editor at Search Engine Land. She built Google Webmaster Central and went on to found software and consulting company Nine By Blue and create Blueprint Search Analytics< which she later sold. Her book, Marketing in the Age of Google, (updated edition, May 2012) provides a foundation for incorporating search strategy into organizations of all levels. Follow her on Twitter at @vanessafox.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • johnsantangelo

    Any idea if the search query will remain behind the hash instead of the “old” search?q= once live, and if that will bring back the concerns over referral and query tracking that came up with the recent “ajax” test?

  • Leah Waddill

    It will be exciting to see the changes in indexing!

  • softwaresoda168

    I just wait for the good search.

  • Errioxa

    I don’t see big changes…

  • jacktrombones

    The new search engine ranked my agency’s website better than with the old version. We also beefed up our internet presence management efforts, which probably also contributed to the better ranking. More at

  • Carter Cole

    i wrote a page to compare the two its here

    it helps to see the differences with a side by side comparison

  • Oliver

    I’m just wondering if this will benefit any of the sites I optimise or make them appear lower in google. I will have to wait and keep checking the ranking of the sites. Thanks for the information.

  • AlbertF

    I really wonder what has changed. I really doubt it is that much different except a bit of tweaks here and there that will greater the relevancy of the results. All in all, I believe this was needed because it seemed as if the search results were full of irrelevant topics when doing a search query before then. Good to see Google trying to improve their algo.

  • nickstamoulis

    It should be interesting to see the major changes withing the Google “caffeine” update…overall I agree with you what has always worked will for on site and off site optimization should remain consistent, but certain aspects shifting…time will really tell what (if any) the major changes will be…

  • tag44

    Thanks Vanessa for the post and for sharing the same in this blog.

  • Nickler Odoya

    Caffeine, like other Google updates scared the hell out of webmasters
    who were dancing close to the line. Many went on to run spam free sites
    while others are lost.

    SEO USA Websites – SEO Company

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide