• http://azzlsoft.com Rich Miles

    This was a mistake by Google. Sure there will be zealots on both sides of the fence that will never be swayed. There are a lot more people who a.) don’t really care, b.) will see Google’s ruse for what it really is. This gives Bing a lot of publicity. A lot.

    Google came public with lackluster results (90%+ failure rate) and questionable methods.

    There will undoubtedly be many white knights that come to defend Google’s honor, but at the end of the day you can bet there are a lot more people trying Bing than there were two days ago.

  • http://www.alliance-link.com Debra Mastaler

    I keep waiting for one of them to scream “my Dad can beat up your Dad”

  • http://mobilitydigest.com/author/doug-simmons/ Doug Simmons

    It would be lovely if Microsoft could just let one of their engineers respond to Google’s engineers rather than multiple suits.

    “We didn’t do this yada yada period, full stop.” Then STOP, you can stop there, don’t go on about admitting to what Google didn’t accuse you of, accusing them of what you’re accused of and denying what they didn’t accuse you of.

    Just shut up and pass the mic to an engineer, someone who can match the credibility of Google’s Amit Singhal.

    Better yet, just say Google’s right, not cool on our part, we apologize and pledge to knock it off. Then you’d look better than you did before this hit the fan. Damnit.

  • http://www.MoSerious.com Chuck

    I think this hurts Google…
    Publicly engaging in this gives promotion to Bing from them. If Bing is displaying the same results…why not use them?

  • http://www.ideabrightmarketing.com Jared R. Fabac, IMCP

    Incredible debate that’s going here. I just read Yusuf Mehdi’s response. They still haven’t moved away from the “gloves off” approach to this situation. I applaud Mehdi for his direct and straightforwardness on the issue. His immediate reaction will certainly build some credibility for their own case. If his words are true (I’m not saying they’re not, but obviously someone is exaggerating their intentions a bit more than the other. That choice is up to you) than it was a BRAVO statement on his company’s behalf.

    However, back to the last point, It also makes you wonder what is really true? Is Mehdi just sugar coating his comments and using the “clickstream” as a way out of being called a “cheater”? I mean to remain realistic on the situation, “clickstream” would be a justified excuse in 90% of the public eye. I would assume around 10% truly know what “click stream” is (its recording what a user clicks on during a web browsing session).

    Of course, we can anticipate a very cleverly thought out response from the minds at Google, right? Did Google make a judgment too quickly without giving it enough thorough research. Was “they’re doing it, we just don’t know how” perhaps a bit too premature to sustain their case against Bing. Was their manual programming of a search query enough for Bing to get away with saying, “it was manipulated to show the same results”.

    I can see this debate carrying on for a long while. I almost wish Google would have taken the testing to a more complete level so that there was no room for speculation and manipulation on what really happened. If results were being copied, the evidence needed to prove the case, likely, is long gone. If they were not being copied, and we remain optimistic with Bing, then Google’s explosion will come back on them that, perhaps, they did become intimidated by the competitors recent success.

    Nonetheless, sit back, relax, take your phone off the hook, and observe. This should be a fun ride.

  • http://boomient.com/ Jim Magary

    Nerd fight! Get the popcorn!!

    I am loving this story, if only because Microsoft is so obviously on the losing end of it. Google’s sting operation is 100% conclusive… their methodology and results are unimpeachable. They caught Bing with their hand in the cookie jar, and now Bing is blaming them back for… click fraud? Way to change the subject, Bing-lings!

    Google is very smart to make PR hay out of this, to remind people that they are, in fact, still the better “decision engine” vs Bing. But from an ethical standpoint, it’s hard to argue that Bing is doing anything truly wrong. Google’s web results are free to the world, and people use them for all sorts of productive business reasons. Heck, I do it every day.

    This is what you get when you offer a free service, and one could argue that “scraping”, or whatever you might call this version of it, is just a by-product of a free and robust web. As far as I can see, no copyright is being violated here.

    The larger picture here is that both companies need to keep their focus on improving long-tail results and keeping them devoid of spam and low-quality content farm mediocrity. As long as they both do that, their engineers can give each other wedgies all day long for all I care. In fact, keep it up, it’s fun to watch!

  • http://azzlsoft.com Rich Miles

    @Jim Is it 100% conclusive? There are a couple of scenarios that would be interesting. How would the results be different on other sites? Why did only “7-9” of the 100 tests result in “success”? How would the same technique fair without the Bing Bar installed? Without IE? Are they capturing the search information from the user typing in the keywords on Google or parsing the URL string after arriving at the site? Would an artificially high ranking on a mainstream keyword also be elevated in Bing? Could this be correlated with Google’s own clickstream data?

    There are so many variables in play. Their methodology is something you’d expect from an undergraduate research class. Not arguably the top engineering firm in the world.

  • http://boomient.com/ Jim Magary

    Well, Bing is not going to copy every search result, because then they’d have to run every possible query and stash the result. As far as I know they are not being accused of “live” stealing by running the query on Google and reporting the result as their own.

    But the fact that Google used nonsense queries and promoted pages that ought not to rank for them based on any SEO factor, and yet those same pages were popping up to the top on Bing for the same query.

    To me that’s conclusive, even if it happened once. I mean, there’s a trillion-some pages on the web, and both search engines picked the same one, when only one of them had a reason? What other explanation could it be…co-incidence?

  • http://www.beussery.com/blog/ beussery

    Perhaps Mr. Mehdi should “Google” “click fraud” before accusing others of doing it!

    For Mr Mehdi and those who don’t know, “click fraud” is a term that refers to pay per click advertising. It looks like Google’s experiment focused on non-paid search results and NOT pay per click advertising. So, Mehdi’s story and excuses kind of unravel from there… :(

  • http://dynamicwebsolutions.com rbrianf

    I mean, a 7-9% overlap in nonsense terms? Really? I’m kind of surprised that Google didn’t take the high road here. I don’t think there’s enough data to start finger pointing, nor do I think Google, with the lion’s share of search, should start accusing the a competitor with such flimsy data. I love Google but this is just lame.

  • messagelost

    @Rich and @Jared – it is conclusive. If you read their statements, both Google and Microsoft aren’t disputing the facts; Bing used clickstream data gathered from google’s sites (via IE or a browser plugin) to rank it’s results. The difference in opinion is the morality of doing that.

    Google argues that taking an observation “user searches for X on Google and gets Y”, using them to power a differently branded search engine that returns Y for X is morally wrong, as piggybacking on years of hard work of Google’s engineers to optimize long tail queries without so much as attribution (and infact, outright denials of attribution).

    Bing’s argument is: We aren’t copying Google’s results. Period. We are copying results that we got from users that got them from Google. Which is, like, totally different.

    I await a follow up article that explains why adding a layer of indirection somehow clears up any moral ambiguities.

  • http://www.linkedin.com/in/amandaxu Amanda Xu

    From a user’s perspective, the ‘nerd fight’ is childish. For the greater benefit of the mass, it might be useful if the two nerds can form alliance, get the result sets virtualized out of two search engines, and reuse the results creatively. There are just too many uncharted territories still waiting for SEO …

  • http://azzlsoft.com Rich Miles

    @Jim It’s not coincidence at all. It was intentional manipulation of the Bing Bar usage data. The fact that it didn’t happen 100% of the time is more indicative that they aren’t copying results. Personally, I am surprised that their technique didn’t yield a higher success rate given the nature of their test.

    Clickstreams are extremely valuable resources in all software. They let you know the users intent and how many clicks it takes to get there. In most cases you want to reduce the clicks. For instance, if you do a search for YouTube on Bing or Google you will not only see the link for YouTube, but also a list of links below the main site like (Music, Movies, My Videos, Sports, etc.) These are all places that people go when they get to YouTube so why not put that information on the frontpage? This can be easily deduced using clickstream data.

    Google spoofed actual users and were able to manipulate Bing’s clickstream data because of the fabrication of keywords. I don’t know if the keywords were extracted from the target URL (which would be the most obvious way to collect it) or through some other method.

    Regardless, Google calling it copying is extremely misleading. Google used to have a monopoly on their clickstream data. This helped them immensely with the relevancy of their search results. Microsoft has take a step in leveling that playing field and now they are starting to collect enough data for their results to have relevance. This should have Google scared. It appears that it does.

  • http://www.ideabrightmarketing.com Jared R. Fabac, IMCP

    @messagelost Great feedback. I completely agree with you that its conclusive that they’re using clickstream data to generate some levels of results. My question is this: If the clickstream data is one of 1,000’s of variables that determine bing’s algorithm, as stated by Mehdi, for search, than wouldn’t Google’s manipulation of a search phrase by moving a specific result to rank one for an irrelevant word eliminate the majority of verifiable algorithm variables and basically leave the engine to almost solely rely on clickstream data, creating the image of copying?

    It seems in the event that a traditional search query is ran, the clickstream data would only have a small portion of relevance on the result, likely explaining the low percentage of identical results during the 100 tests.

    It also seem to me (which doesn’t mean much, I’m a moron sometimes) that more data could have been brought to the forefront if a traditional search query script was pinged off the search engines and results measured, and then manipulation would occur on the same search query to see if the pinged script returned the same results from both search engines before and after manipulation. I think in the event that the search query “xyzblablabla” with no relevance is created and you manipulate a result to be #1, than due to the irrelevance of the search query, clickstream data would be used much more highly weighted for the ranking hence presenting a duplicate result. Am I nuts?

    Either way, I agree with you that its still an ethical question. But a fun debate.


  • http://azzlsoft.com Rich Miles

    @messagelost and @jared

    If people want to debate the ethics of clickstream data, that’s fine but then Google and Bing are on the same side of that argument. They both use clickstream data. They both track lots of things. Some of those things are to make search results more relevant and some of those things are to better target ads.

  • http://www.know-what-im-sayin.com/ eric garrison

    The whole argument seems ridiculous. All of these guys copy each other. I hope Bing continues to refine their results, and can someday be a real competitor to google.

  • http://markk-b.sitesled.com/ MarkKB

    “Bing’s argument is: We aren’t copying Google’s results. Period. We are copying results that we got from users that got them from Google. Which is, like, totally different.”

    Funny, but wrong. Bing is saying that they analyse what links people click on with regards to what they search for. That means if most of the users* click on the third, or fifth, or ninth result on Google, that result will gain in rank on Bing, rather than the first.

    As you can infer, that’s a pretty terrible way to ‘copy’ Google’s resutls, so I highly doubt that was the intent (especially since Bing could have just pharsed the web page itself and directly copied the results.) Since they seem to do this with *every* website, the intent seems clear: they’re learning for what links humans associate keywords with, and factoring that in to the results for Bing. The only thing ‘bad’ about this is how Google’s reacting about it (which also comes as a surprise, since they’ve been public about this since at least 2009.)

    * ‘users’ meaning ‘everyone who has the Bing toolbar enabled and opted in to data collection’, of course.

    @beussery: funnily enough, I Googled ‘click fraud’. One of the definitions (from watchdog.com) was “an online crime that involves automating the act of clicking on a web link to perpetrate a fraud.” This seems to cover Mehid’s usage, if not literally, at least metaphorically..

  • http://markk-b.sitesled.com/ MarkKB

    In the previous post I meant to say ‘watchguard.com’, not ‘watchdog.com’. Sorry! ^^;

  • http://taranfx.com Tarandeep Singh

    >> Google has “copied” a few of these.
    REALLY ? Didn’t Bing copy these features from Ask.com.. How shameless can Microsoft get by accusing someone of copy when they copied it first!

    Google copied Ask.com not Bing!
    MS did agree that they watch click actiivty, that says it all, they see Google search behavior..

  • messagelost

    @Jared – While those queries were synthetic, the motivating example from the original article was a real query – “torsoraphy”. These “long tail” queries individually have very few searches, but taken in sum can be a huge volume. Signals break down on these types of queries because each is very specialized. Given the result of the synthetic queries (that Bing will republish Google results when it can’t find other signals), and that the Bing “torsoraphy” search in Danny’s screenshot (which showed 3 of 4 results, 2 are irrelevant and 1 identical to the first Google result), it seems likely that a number of the long tail queries on Bing are being served by duplicated Google results after Microsoft’s signals break down. Maybe it’s a small number, or maybe it’s a large number; only the Bing engineers could find out.

    As to the second part, the synthetic queries did rank higher because there was no other signals for them, but that doesn’t mean real queries aren’t also signal-less.

    @Rich – But google isn’t saying all uses clickstream data are immoral, but rather using clickstream results to extract a competitor’s rankings and display them as your own. The interview in the original SEL article had this quote:

    “Absolutely not. The PageRank feature sends back URLs, but we’ve never used those URLs or data to put any results on Google’s results page. We do not do that, and we will not do that,” said Singhal.

    (Danny then points out that they use site speed which is derived via the clickstream to effect ranking, but that wouldn’t have competitor’s rankings in it)

    Just having the same data isn’t the same as using it. And personally, I think there are plenty of moral uses for such data, like examining how a queries on a competitors site and trying to reverse engineer their process. But this isn’t reverse engineering; Bing has no idea why it ranked things for “hiybbprqag” or “torsoraphy” highly, other than these are the results users got when they did the same query on Google.

    I can’t really say I understand your argument. How is it a “pretty terrible way to ‘copy’ Google’s results”? They get every link on a Google search that a user with their software clicked clicked on – users click on relevant results. That’s really the only ones they need to copy.

    Additionally, copying the ninth result is still copying a Google result. Copying the 5000th result is still copying Google’s result if they get from a websearch at Google.com even if they get it through a layer of indirection.

    As to intent, it may indeed be unintentional. It may be intentional. I would find it strange they’ve never thought about the clickstream containing Google results or how they rank queries using them, but it is certainly within the realm of reasonable possibilities that this was overlooked. But intentional or not, copying and rebranding results is still copying results. And finding out your search engine is even unintentionally copying and rebranding results should be time for an “Oops, our bad”, not “What we were obviously doing we weren’t doing. Period.”

  • NOT RahmEmanuel

    Google: You copied us!

    Microsoft: Nothing of the kind. We asked some friends to send us a real-time feed that showed exactly what you were doing. We just happened to rely upon that info to inform our own actions.

  • M Live 74

    Google, How do you call this ?
    Making money off of others hard work at You Tube by showing ads ? Oh yea, you will take it

    off if the content owner writes to you!!! oh sure,… you are a big brother you can display

    my contents without MY permission and I will have to come to you to complain about MY


    Same goes for Google books (sure your books:)… you copy millions of books without

    authors/publishers permission and ME COME TO U TO TAKE IT OFF FROM YOUR BIG BROTHER SIDE!!!@

  • http://jalaj.net/ jalaj

    By revealing this thing, Google fulfilled another intention of his… pollute the Bing’s result by black-hat SEOs who are on his back now. Have posted a full blog-post at http://jalaj.net/2011/02/03/hiybbprqag–the-search-engine-game/

  • http://azzlsoft.com Rich Miles


    Microsoft does not query Google. Microsoft does not even know what rank a site has. All they know is that from a page on Google a user went to another page… just like every other site on the web.

    You seem to take issue with the fact that Google did some sort of organizing of those links to help put the relevant links near the top. Taking issue with this is the height of hypocrisy. The Google ranking is valueless without the hardwork of reputable sites and user feedback. Experts in all fields spend countless hours reading through tomes of technical journals and wading through a sea of spam sites selecting only the best to link to from their sites. Google is piggybacking off of all of that hardwork. I’m not saying it’s not brilliant, but they couldn’t exist if there weren’t millions of people culling the bad data.

    Microsoft shouldn’t — and probably doesn’t — care how a page ranks in Google. They are piggybacking off of the hard work of the USER who is trying to find the relevant results. The user is piggybacking off of Google. Google is piggybacking off the experts.

    At this point, I am uncertain how someone could possibly connect the dots in a rational way to say that Microsoft is “copying” Google. Google injected results into Bing through an obscure and relatively ineffective method and then claimed it was “copied”.

    If you want to hate on Microsoft there are plenty of valid reasons: letting their mobile division languish, their ridiculous approach to tablets, online services losing billions, killing Netscape with monopolistic practices, IE6, Microsoft Bob, Windows ME, Kin, this list goes on for a while. However, there is no validity to the claim that Microsoft is copying Google.

  • AlanLindsay

    A huge amount of nonsense. Anyone who actually looks in real detail at what happened – not the surface level of Google’s deliberate misdirection – but what they actually did and the steps along the way can see that this is actually about Amit Singhal and Matt Cutts actually lying.

    They statement is clear – Bing copied Google’s search results. Even their own detailed statement shows that not to be the case. What happened is that Google engineers opted IN and CHOSE to SEND Google results to Bing as a possible indicator that a page is a good result.

    And then for a small percentage of cases of those results, where there was NO OTHER indicator of ANY good result, the Bing algorithm used that data to surface the page.

    That isn’t copying – that’s believing someone who tells you something in good faith.

    As several people have pointed out – the only way copying would have been proved is if they had explicitly NOT ever opted in to having their search results added to Bing.

    If anything what this shows it that the Google team involved are actually extremely poor logical thinkers and extremely poor scientists who can’t design a good experiment or test. It does make them very good manipulators of the truth and very good slimy PR people though.

  • http://www.beussery.com/blog/ beussery

    @MarkKB As you yourself pointed out in the comments above, Mehid’s comments are “literally” incorrect. This isn’t a metaphorical debate. Bing was caught red handed scraping Google. I guess the world now knows who the real “Decision Engine” really is now? :)

  • http://www.michael-martinez.com/ Michael Martinez

    @beussery “Bing was caught red handed scraping Google.”

    If THAT were true, someone would have published evidence of that by now. I haven’t seen anything other than Google’s ridiculous attempts to take an unscientific sampling of comparisons with a less than 10% hit ratio and blow it out of proportion.

    Bing published a long time ago that it was using user click data in its discovery process. Google and Amit Singhal have lost all credibility with their nonsense.

    And the SEO community should move on because this is really getting silly. We don’t owe either Google or Bing anything. If they want to embarrass themselves further with more name-calling, they’re big companies. They can do whatever they like.

    Google probably has more interesting plans for Egypt than it does for Bing anyway. I suppose it’s only a matter of time before the U.S. Congress passes an anti-corporate submersive activities act based on Google’s behavior.

    There’s no room for science in this discussion: the truth was thrown out the door days ago.

  • Bhekuzulu

    This is obviously costing Google a great deal of money, one could probably say it is unfair. But Google is just as unethical, it would not have cost Google one cent to say a unit of knowledge was coined by myself, knowl. Instead three years later they called a unit of knowledge knol and said they coined the term. All because they could not give a black man credit for anything as they fight not even an acknowledgment.
    Well Bing is making money of Google, good, evil begets evil

  • Bhekuzulu

    Google forced me to warn the world of their type of racism, hating blacks who have original ideas, http://www.blacknews.com/news/knowl_google_bhekuzulu_khumalo101.shtml

    Google should not complain,

  • messagelost

    @Rich –
    Microsoft doesn’t have to query Google, they get the queries by proxy through a opted-in users. I don’t see how taking Google search results through a layer of indirection somehow makes it OK to display them as Bing search results. The fact they don’t know if it was the first or tenth Google result is irrelevant; they clearly know that Google is displaying this result and will redisplay Google’s result as their own. The fact that they are doing it to every site on the web is ALSO not an excuse.

    Google does not do “some sort of organizing”, rather, the entire point of a search engine is you put a text string in and get URLs back; it’s Google’s mapping they are copying.

    You then go onto talk about it being hypocrisy. This is misdirection and a clear logical fallacy. Even if Google is “wrong” to work off experts, two wrongs don’t make a right. Bing is wrong to display Google results as their own regardless of actions Google may have taken.

    You then say that Bing is piggybacking off the User who is piggybacking off Google; again, I fail to see how a layer of indirection somehow makes it not copying, but we’ll agree to disagree on that point.

  • bntabor

    Google is scared and crying like a baby, Bing is smarter by copying and maybe improving on Google’s results. What would you do, not pay attention to your competition, or analyze your competition to death and beat them at their own game. All’s fair in love and war.

  • http://www.beussery.com/blog/ beussery

    @Michael Martinez “If THAT were true, someone would have published evidence of that by now….”

    I hear what you are saying but, one wouldn’t make accusations if they didn’t believe the other had evidence. Potential legal actions from either party aside, unfortunately I don’t think it’s possible for outsiders to come forward with additional data or testing because of the way this internal test was conducted.

    Most folks are aware that search engines use user click data for discovery. I don’t think that should be interpreted as a license to copy and republish the content discovered as their own.

    Such precedent is a reeeeealy slippery slope…