I can still remember when my when I first switched over to Google on the recommendation of my brother’s girlfriend. She’s literally a rocket scientist and carries a lot of intellectual weight with me; her endorsement was fairly simple – “it returns great results”.
To understand what “great results” means, transport yourself back 10+ years ago and try to remember the search user experience – one dominated by short tailed queries and multiple paginated results until you could find what you were really looking for. We didn’t complain because it was free and much better than that old dewey decimal approach we learned in middle school. In fact, it was magical.
Enter Google and the experience and expectations changed significantly. On the strength of superior results, Google’s market share skyrocketed. Over the past decade we’ve grown accustomed to results so accurate that we rarely peek beyond the top three. Yet, I have trouble believing that Google’s dominance continues to be based on superior search results, especially given the financial and human resources thrown at improving search across multiple competitors.
Danny Sullivan questioned Google search relevancy last year: How the “Focus on First” Helps Hide Google’s Relevancy Problems. And as far back as 2007, Chris Sherman wrote a review of PC World’s search engine shoot out called Stop the Presses! Google Bested in Search Shoot-out!
My personal search approach uses Google as the default while using other sites for specialty searches. On Bing, image search is far superior and Wikipedia for 101 style information. Is this a factor of inertia or am I really getting a better experience with Google? I set up an objective small sample size evaluation of search quality between Google and Microsoft to see for myself.
Let’s Test The Theory
Methodology: I evaluated 20 different searches split evenly between Transactional and Informational queries and the search engines’ ability to deliver quality results, admitting that quality is a very subjective term, but includes things like timeliness, 1 click access to info, volume of content and lack of spam.
To provide a useful analysis, I’ve upped the degree of difficulty, deliberately testing long tail queries that are fairly specific and potentially confusing to a computer. I avoided easy navigational searches in favor of things that could trip up a computer’s ability to understand the user’s intent with queries like “Attorney Tom Brady”.
Results, including the snippet, needed to convince me of a likelihood of finding what I was looking for by clicking through in order to be considered.
To rank the results, 5 points were awarded for a good quality result ranking first, 3 for second and 1 for third. Two bonus points were added for top 3 results being on a highly authoritative site (as determined solely by yours truly). Five points were subtracted if the entire first page didn’t contain any good results. I only considered non-traditional results (like local, news, or Answers) if it appeared above the fold and I logged out of my gmail account when running queries.
Not too surprisingly, there was not a massive disparity in the results of my little test. In fact, Bing came out on top. Some queries performed very differently than others, for example, Bing was able to tell that my query for “Attorney Tom Brady”, was looking for an attorney and not the pictures of the hunky Patriots quarterback served up by Google.
Bing also did well with date nuances, unlike Google, sending me to the 2011 page for “SMX West Agenda” and a future calendar of concert appearances for “When is Trans Siberian orchestra playing next in Seattle” instead of past shows. Google won points by sending me directly to the IMDB page for filming locations for “What town was Beautiful Girls filmed in?” All in all, the results aren’t too surprising.
Of course, this begs the question why has Google been so successful? Are they still riding their brand laurels? Has Microsoft’s brandings and rebrandings of search hurt them among consumers?
Last week, a CNN article suggested that Google’s stock potential upside is probably no more than 20% – are we seeing the end of the growth in search? While these questions are the subject of intense debate in board rooms and VC water coolers, the conversation no longer centers on superior search results.
While I spent most of this post in retrospective navel contemplation about search, what does this mean tactically for the future of the in-house SEO? Despite the fact that we live in an industry of perpetual seismic change, Google has been a constant dominant player for most SEO’s entire career. Is Facebook the new SEO (as someone at work keeps insisting)?
Is Quora, the newest shiny toy whom the technology press dumped Twitter for this year, going to change everything? Is there a revolution in Local or location based search brewing? Or does consumer behavior change so much more slowly than technology (watch Quora trip on results for “Who is Lindsay Lohan” for example) – providing Google with a long lasting competitive advantage.
Your answers to these questions should shape your long term online marketing strategy.
Postscript: See also 89% Find Search Engines Do Good Job Finding Information, But “Noise” Is Issue
Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.