In the world of competition between and among websites, search engines and online publishers there is also the “behind the scenes” competition between Nielsen and comScore. Both want to be the source of record for traffic data and both are occasionally the subject of controversy.
There was the incident early in 2008 where comScore put out a report that Google’s paid clicks were down, sending Google shares down. But when Google announced a very strong Q1, comScore shares then took a hit and the company had to go into damage control mode. More recently Nielsen reported that despite Twitter’s massive growth it was only “retaining” 40% of users. There was an immediate and vocal backlash, putting Nielsen on the defensive. However the company stood by its numbers after a second look.
Despite outward confidence and appearances of objectivity, these measurement services are subject to human error (interpreting the data) and occasionally just screw up. However, advertisers, banks, the press and others with an interest in tracking uniques and traffic growth don’t trust the numbers put out by the publishers themselves. So third party measurement and validation is a high stakes game.
The newest metrics controversy, this time pitting comScore against Nielsen, involves Hulu. As the New York Times reports, comScore recently reported almost 42 million unique visitors for Hulu compared with only 8.9 million according to Nielsen. However both data providers show high percentage growth at Hulu. But Hulu was frustrated and upset with the Nielsen figures that show unique visitors declining from March to April.
Quantcast and Compete, imperfect and in disagreement themselves, show Hulu’s numbers closer to those put out by Nielsen than comScore’s estimates:
Google Trends data sits between the Nielsen and comScore numbers:
The obvious conclusion is that one has to look broadly at all the numbers and use the consensus and directional trends. This is something of a mantra for Danny in reviewing the search market share data.
I’ve asked Hitwise to weigh in and will update this story if they do.
Related Topics: Search Engines: Video Search Engines | Stats: comScore | Stats: General | Stats: NetRatings | Stats: Size
Would be very interesting to see an analysis of the sites that have the greatest discrepancy between nielsen and comscore… it may reveal biases one way or the other (does one favor older audiences? at work vs at home usage? etc)
Great article, especially since you also included Quantcast and Compete. Kudos
The difference between 42 million unique visitors and about 9 million unique visitors is quite a gap, even if we are looking at things broadly. I would have suspected large companies like Hulu would have accurate third-party measurement but apparently that’s not the case.
If we can’t trust the numbers put out by the publishers, and we can’t trust the numbers put out by third-party measurement, who can we trust?