Why The Wikipedia/Google Search Results Study Is Flawed

wikipedia-logoMany SEOs have been chatting this week about a recent study by Intelligent Positioning (first reported by Search Engine Watch) that showed Wikipedia ranks on Google UK for 99 percent of searches.

Yikes, right?

Wikipedia — more specifically, Google’s apparent love of Wikipedia — has long been a sore spot in the SEO industry, so seeing a statistic like that is a big pile of salt in the wounds at this point.

But it’s really not a statistic to get worked up about because, in my opinion, the study itself was flawed.

Study Methodology

As Intelligent Position explains, the company used a couple random noun generators to come up with a list of 1,000 nouns — words like “ashtray” and “volcano,” “snowflake” and “melody.” It then did 1,000 unique searches on Google UK and charted if and where Wikipedia showed up in the first page of results.

google-wikipediaThe results? Wikipedia was on page one for 99 percent of those searches, was the top-ranked result for 56 percent and was in the first five results for 96 percent of those noun searches.

Over on eConsultancy today, Kevin Gibbons makes the point that this shouldn’t be too surprising because Wikipedia does a lot of things right where SEO is concerned: usually very rich/deep content, highly targeted web pages, strong domain authority, loads of inbound links and more.

I don’t argue at all with those points, but I’d add something that seems just as obvious to me: the study only used one-word nouns. Wikipedia is an online encyclopedia — in large part, it’s a repository of information about nouns.

With all of that solid SEO in its favor, chances are great that searches for things like “tortoise,” “asphalt” and “liquid” are going to have Wikipedia very high in Google’s results. The Wikipedia page about tortoises is nothing short of amazing. Ditto the pages about asphalt and liquid.

A Suggestion For Different Methodology

Most search queries are longer than one word nouns. Chitika recently pegged queries at between 4.07 and 4.81 words on average, depending on the search engine. A couple months ago, Hitwise reported that 27 percent of searches that produced clicks were one word — leaving 73 percent of searches not represented in this study.

What I’d love to see someone do is this: Do a thousand searches (or more) that represent actual search engine behavior. Make 27 percent of those random searches be a single word (like “tortoise” or “liquid”); make 24 percent be two words (like “buy laptop” or “ankle pain”); make 19 percent be three words (like “u2 song lyrics” or “funny Valentine’s cards”), and so forth up to seven or eight words.

And then, using a variety of search terms that mimics actual search behavior, show how often Wikipedia appears in the first page of results. I’m pretty sure it’ll still be very high, but it won’t be 99 percent of the results.

Related Topics: Channel: SEO | Features: Analysis | Google: SEO | Google: Web Search | Search Engines: Wikipedia | Stats: General | Top News

Sponsored


About The Author: is Editor-In-Chief of Search Engine Land. His news career includes time spent in TV, radio, and print journalism. His web career continues to include a small number of SEO and social media consulting clients, as well as regular speaking engagements at marketing events around the U.S. He recently launched a site dedicated to Google Glass called Glass Almanac and also blogs at Small Business Search Marketing. Matt can be found on Twitter at @MattMcGee and/or on Google Plus. You can read Matt's disclosures on his personal blog.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://kevgibbo kevgibbo

    Hey Matt – completely agree, Wikipedia are always going to rank highly for informational queries – which is why I wasn’t surprised to see their 99% dominance of page one rankings.

    A study on more transactional queries would definitely be very insightful!

  • Aaron
  • Matt McGee

    Thx Kevin & Aaron. I’m aware there have been studies like this done in the past — Google even cites the one(s) that say Bing favors Wikipedia more. But we need something more current, I think.

  • http://kevgibbo kevgibbo

    It would be great to get a wider study on this, I’ve just asked Sam from Intelligent Positioning about extending the original and it is something they are looking at.

    Steve Plunkett made a valid point on Twitter, that 84% of the webs queries contain more than one word – so while the sample we have is interesting to see (and at the moment all we’ve got, at least recently), it would be great to see the bigger picture on this.

    My post was more about the SEO principles and strategy that Wikipedia do such a great job on, I think those concepts apply across different types and lengths of queries. But agree that they’re never going to dominate search to the same extent for longer-tail, transactional queries.

  • samcope

    Hi Matt – nice post.

    Firstly, I totally agree, the Tortoise page on wikipedia is excellent. Plus i don’t believe for a second that Wiki shouldn’t be page one (or position one) for that search and multitudes of others.

    People have got quite het-up about this. Some say there should have been a million samples. Others say i should have picked brand terms (?), others have told me how great Wikipedia is (i know), others have made the single-word point.

    The study is what it is: to see how Wikipedia performs for random single-word searches.

    As you point out, its on its way to fulfilling the 27% area of searches. But this study never set out to prove anything, just to open debate and ask questions. Not to slate the world’s best site (wiki that is).

    I wanted it to be random thus having a controlling factor, where the searches weren’t determined by popularity, usage, location or whatever, hence tortoise, snowflake and liquid etc were there. But there were also big terms such as loans, television and computer.

    If i had chosen words (based on my opinion, the most competitive search terms, the world’s biggest brands) then that would be flawed too.

    We’ll be doing more searches in our other European offices in different languages. Plus your methodology seems sound. It’s just a matter of picking those appropriate terms – fairly.

    Thanks a lot and thanks to Kevin too.
    Cheers
    Sam
    Intelligent Positioning

  • http://NetArgument.com Ramesh

    Google favors wikipedia. This can be demonstrated very easily by searching for keyword – Scholarpedia quantum mechanics. Google having understood very well that the person is seeking result from scholarpedia, would return results from wikipedia on top. http://www.google.co.in/search?q=quantum+scholarpedia&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a#hl=en&client=firefox-a&hs=NEe&rls=org.mozilla:en-US%3Aofficial&sclient=psy-ab&q=scholarpedia+quantum+mechanics&pbx=1&oq=scholarpedia+quantum+&aq=2v&aqi=g-v3&aql=&gs_sm=1&gs_upl=0l0l1l7750l0l0l0l0l0l0l0l0ll0l0&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&fp=192c583e64b4624f&biw=1280&bih=697

    One may wonder, who is the greater evil, Google or Wikipedia. Shame on you Google.

  • http://www.conductor.com Nathan Safran

    Hey Matt,

    I think you make some valid points about this study. I think we have the data at Conductor to complete the study you are looking for. I’ll take a closer look once we are on the other side of SMX West and will update you then :-)

    -Nathan Safran
    Conductor, Inc

  • http://www.search-usability.com/ Shari Thurow

    Hi all-

    Wow…I’d say the study is flawed. Searcher goals are currently classified as informational, navigational, and transactional. Did the study just throw words out (as indicated here) or did they take those words and use them in context of searcher goals?

    If web searchers do not provide context, then they leave it up to the search engines to determine the context. A one-word query (unless it is a navigational query) generally doesn’t provide context, which makes informational websites, like Wikipedia, a reasonable search listing to display. In fact, the appearance of a Wikipedia listing on the first page is a strong indication that the keyword(s) show informational intent.

    Data without context is just data. Wrong context? Probably wrong analysis and conclusions, or at least inaccurate.

  • http://www.textilesinfomediary.com T.I.

    Wikipedia have earned its reputation as neutral online accessible authentic resource for people by the people.
    Other people can open there account with wiki and express there concern with particular issue.
    Which may take first position in google result. So no argument rest against it.

  • http://www.NetArgument.com ramesh

    “… Neutral, Online, accessible authentic….” Oh really?
    Surely, it is accessible and online, courtesy Google, but that doesn’t make it authentic and neutral.
    Also, you cannot ask others to express their concern there, because that site operates within close loop of admins and their cabal. Unfortunately you also can’t close the arguments against it, because it is not Wikipedia Administrator’s Notice Board.
    One more piece is here http://www.netargument.com/2012/01/wikipedia-new-age-porn-magazine.html

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide