• http://www.michael-martinez.com/ Michael Martinez

    Chris wrote: “‘Eight in ten Americans.’ Not survey respondents. That’s a rather over-reaching statement.”

    You just set off my BS meter, Chris. So you’re implying that every time some industry organization or government agency claims that “4 out of 5 Americans” eat hamburger, that they have actually asked 80% of all Americans?

    You’re creating a double standard here for the sake of criticizing the survey.

    I refuse to read the rest of your article. You’ve done far better journalism than this. As a loyal reader I feel I deserve better than that from you.

  • http://jamesbutler.net James

    @Michael: The opposite is true. Chris is objecting to the use of the “Americans” group, rather than the real group … “respondents”. Chris would prefer it if FairSearch had written: “Eight in ten respondents”, because “eight in ten Americans” is misleading and indicates bias.

    A double standard would be created if Chris had written other articles about other polls/surveys that did not include a complaint about this common irritant … but he hasn’t, has he? Therefore by objecting to this single group’s use of that word, the is no double standard … just an objection.

    And I concur with you both. Frankly, whenever ANY poll reporter refers to “Americans” (i.e. “Most Americans do not like broccoli” because a majority of the 1000 survey respondents didn’t like it) it tells me that whoever is doing the reporting has a bias that needs to be accounted for.

    Perhaps Chris should have noted that he is irritated by ANY poll/survey that uses such manipulations, just to avoid your double standard accusation? He’s complaining about FairSearch’s bias … and that mechanism is one piece of evidence toward that bias.

  • http://searchengineland.com Chris Sherman

    @Michael, in the age of Twitter it’s not uncommon for people to quote/tweet results such as these verbatim. Without the context of adding “respondents” or some other qualifier, this kind of statement can/will likely be used as representative of the entire country. I’ve seen this happen time after time. I stand by my objection, and my assertion that this is a highly biased survey that’s being used to promote an anti-Google agenda.

  • TimmyTime

    “Fairsearch.org attributed the stat it used in its survey to Search Engine Land, which was a report/analysis of the data, not original the research. This source misattribution further undermines the credibility of the survey—sloppy methodology can never help and can only harm anyone undertaking supposedly “objective” surveys.”

    So, is the data wrong or you just didn’t like their foootnote style ? You used the same argument a few times. Is the data wrong and how wrong is it?

  • Chas

    Please tell me who has a search engine ad service with the reach of Google and Microsoft? I am dying to know. Not much of a competition when there are only two players, unless it’s Chess, Tennis, Pool, or Ping-Pong.

  • Winooski

    TimmyTime, the answer is “Neither.” I agree with Chris’ criticism because it’s important for a publication that makes use of qualitative research to cite the *originator* of the research instead of citing another reporter of the research. Here are at least three good reasons why:

    (1) It’s more efficient for (and therefore more respectful of) the reader: If there’s a question about the way the research was conducted, the reader can more quickly get it from “the horse’s mouth” instead of first going through a third party.

    (2) It reduces the likelihood of a factual error: There’s a chance the source reporting on the research got something wrong, and, by citing them instead of the originator, the publication may be inadvertently engaging in a version of The Telephone Game ( http://en.wikipedia.org/wiki/Telephone_(game) ).

    (3) It doesn’t piggyback on the good reputation of the reporter: Most important (and, I suspect, the reason Fairsearch.org attributed the news to Search Engine Land), by citing a reporter of the research rather than the originator, the publication may gin up the credibility of the research.

    In the context of writing about Google, stating that the research came from “Macquarie Group, using Efficient Frontier” may not have much of an emotional impact for the reader, but stating that it came from “Search Engine Land” immediately adds value to the research. “Ooh, this data is about search behavior, and these guys are called ‘Search Engine Land’. They *must* know what they’re talking about right?”

  • TimmyTime

    Winooski,
    He should have attacked their tactics and words used (push polling) not get into 1% difference or who cited who.

    —Google does not “raise prices for advertising.”
    Yes they do. Low Quality Score is one but the idea that less competition causes a price increase is valid too.