• http://www.tylerherrick.com Tyler Herrick

    I hope Bing dies.

  • John O’Connor

    I would like to point out that the best result on fitbit (ads included) is for fitbit.com official on the google side of the equation. So the best result has the same result for a Google fitbit search.

  • Durant Imboden

    Susan Athey’s use of the term “best search result” strikes me as being naive. If she’d said “algorithmic search result” or “organic search result” or something similar, I might cut her some slack, but “best search result?” What’s best depends on who’s searching and on the intent behind the search.

  • Fili Wiese

    Did anyone submit this post to the EU, out of concern of Microsoft’s sad behaviour? Just wondering :-)

  • http://searchengineland.com/ Danny Sullivan

    Exactly. That’s another flaw in this research, the assumption that the top web result is necessarily the best search result, when a vertical listing might be better.

    Worse, my assumption is that this test was all about a case of “10 blue links” rather than a page with ads, vertical boxes and so on — yet it’s applied to a case involving vertical search results.

    It’s hard to say, however, because the actual research wasn’t shown.

  • http://searchengineland.com/ Danny Sullivan

    I think Bing actually does better here, getting Fitbit up higher. But ads are very dynamic. Keep reloading, you might find a display where Google doesn’t have them, or Bing doesn’t have them.

  • http://www.seo-theory.com/ Michael Martinez

    Good for you, Bing! Let’s see more research come out into the field! Obviously from the reactions in the SEO community you have a lot of ground to cover.

  • http://reelseo.com/about/grant Grant Crowell

    The FTC recently put out an update to Disclosures guide for digital/online advertising, which has significance to the search industry (both search engines and search marketers). You can download a free copy of the report at http://www.ftc.gov/opa/2013/03/dotcom.shtm

  • Andy

    Bing for the win…g

  • http://www.v2interactive.net/ Josh

    Why? Because you, as a business owner would sit back and allow your baby to be crushed by competition? This is business. Dog eat dog. I give props to M$ for having the guts to invest themselves into this.

  • Spann

    I haven’t read all the prior studies but I believe most (and certainly the Chitika study) show that higher rank is associated with more clicks, but do not control for “quality” – thus these studies do not foreclose the possibility that lower ranked sites may get few clicks because they are less relevant. This study appears to control for quality and shows that rank by itself, even controlling for responsiveness or “quality” of the site for the particular query, results in fewer clicks. From what I’ve seen, this is in fact a new result and it dispells an argument that I’ve heard when discussing studies like Chitika’s. I don’t think it’s groundbreaking research, but from an experimental research perspective it’s important, and in any event it’s not deserving of the mean spirited sarcastic tone of this post.

  • Spann

    From an experimental design perspective the assumption identified above actually isn’t a problem. The goal presumably was to show that rank affects clicks setting aside quality (or in other words that the rank-click correlation isn’t confounded by quality). By moving top ranked sites down, this is tested either way. In any event, presumably the search engines have some basis to say that top results are “better” on average for the relevant query (generate more clicks when rank is averaged, for example), the fact that it may not be best for a particular user is simply averaged out – that’s the basis of statistics.

  • http://searchengineland.com/ Danny Sullivan

    This report doesn’t appear to control for quality. It makes an assumption that whatever the first web result is, that’s the best result. It might not be. And that’s the heart of any report trying to make assumptions about whether searchers are harmed or not. Of course, that’s a further assumption, since Microsoft hasn’t shared the actual report.

  • http://searchengineland.com/ Danny Sullivan

    So let’s say you have a site that’s hacked with malware, and it starts ranking tops for “payday loan.” You can average the clicks out all you want — it’s not the best site for that search. There’s nothing in this summary of the research that actually addresses the quality (or not) of whatever was moved above the supposed “best” result. The report is really about predicted clickthrough rates on search listings — which, as I’ve explained, has long been reported on already.

  • Spann

    In that situation you wouldn’t get a rank effect at all – if you search for “Bruce Springsteen” and the first result is a payday loan site and the second is “Springsteen lyrics”, then the top rank wouldn’t get a large percent of the clicks – the rank effect would disappear. Since that didn’t happen here, it’s likely the result set was reasonable. Within the pool of reasonable results, if you have sites X, Y and Z and you show them to enough users who search for “Springsteen” while altering the display order, you’ll find that say 50% click on X, 30% on Y and 20% on Z. Because the order has been randomized across many searches and users, this reflects a “true” quality order for the average search for “Springsteen”. If you then put X in the first rank and it gets 65% of clicks and in the 3rd spot and it gets 35% of clicks, then that tells you that there is a rank effect that is separate from quality of the result. It sounds like that’s what this study did.
    That is different than the other studies on CTR by rank. Surprising? No. Different and an addition to collective knowledge? Yes.

  • Spann

    I agree that it’s hard to know without seeing the actual study. But by moving the results around, they control for quality. If you have one result in spot 1 and a different result in spot 3, one of them is better than the other. What this study seems to show is that no matter which one you put first, the top rank gets more clicks. Unless the results are all the same (just randomly selected), then this controls for quality. I imagine that what actually happened is what I describe in the comment below, so they have some basis to say that a link is “better” than another. But unless the result set they tested is just way off, it doesn’t matter, either way it would show that rank affects clicks regardless of quality. Of course that is different than saying searchers are harmed. I don’t know enough to have an opinion on that, I’m just talking about the “science”.
    BTW, it is not the case that people inherently select rank 1 more than other positions in any human search endeavor. If you showed a vertical list of pictures of 10 people and asked which was most attractive, you would not see a significant rank effect (you’d actually see a small preference for the bottom result). So the search engine rank effect results from a perception of expertise in the ordering, that the ordering does in fact reflect quality.

  • http://twitter.com/MrMazharShah Mazhar Shah


    Good post so for. I think Bing is improving day by day with following Google but its search results is not smooth like Google.

  • http://itnewsweek.com/ Deniel

    Interesting research…

  • Andrew Agnello

    Another example of delusional academia having to conduct research to understand basic common sense. Good thing Susan doesn’t have to function in the real world.

  • http://www.seopro.co.za/ Kim Troskie

    The first search result is the most well optimized website, not the best search result.

  • david mcmillan

    I use each one for different reason. Works out very good. To depend only on one, results could be bad. A few of us found out the hard way. But was corrected

  • http://twitter.com/YoungbloodJoe Joe Youngblood

    Danny, Glad to see you included travel in this. But what about YouTube in video universal (nearly dominates the whole thing), what about Google+ places, etc.. When google first started they used signals from other websites for ranking purposes, now if you use Google+ Author rank on any page it supposedly helps, so now it’s using signals from Google to rank. The garden walls are getting higher.

    I also think Bing would concede their vertical search initiatives if it meant an anti-competition slap on Google.

  • http://searchengineland.com/ Danny Sullivan

    YouTube certainly has grown, in my view. Wrote about that before. Agreed on how authorship turned into Google+.

  • Brenda Thompson

    uptil I saw the bank draft for $6297, I didn’t believe that my mom in-law really bringing in money part-time on there computar.. there best friend has been doing this for less than 20 months and resantly repayed the debts on their mini mansion and purchased a brand new Renault 4. I went here, jump15.comCHECK IT OUT

  • dorabowers

    Sarah. if you think Dennis`s blurb is really great… last thursday I bought a gorgeous Chevrolet Corvette from having made $5864 this – five weeks past and also 10k this past-month. this is certainly the best work I’ve had. I started this 5 months ago and right away began to make minimum $71, per-hour. I went to this web-site, jump15.comCHECK IT OUT

  • Yeye Yates

    Have just recently launched a collectibles online store and all SEO steps
    done focused strongly to appear in the upper tier of Google. Based on
    ColbriTool analysis output, majority of search inbound traffic are coming
    from Google – but still, its better to double the efforts to include Bing and
    Yahoo. Thanks for sharing this article.