Diego Basch, a former Senior Software Engineer at Inktomi, who is now Director of Engineering at LinkedIn explained why on his personal blog.

He outlines the major signs of the fall of Inktomi when he noticed most of the engineers at Inktomi using Google to find content, instead of Inktomi. But why? Why did Inktomi fail? He feels there are many reasons but the core reasons include:

• Inktomi didn’t control the front-end, they feed results to their partners such as Yahoo who controlled that search results experience.
• Inktomi didn’t have snippets or caching despite his efforts to do so, he was shot down by the executives.

Basch explains:

In short, Google had realized that a search engine wasn’t about finding ten links for you to click on. It was about satisfying a need for information.

This additional seed of Series D financing brings the total money raised for Trada to $17 million.

Trada basically lets you offload your PPC management onto a community of SEM experts, who spend the budgets you set up as they see best. The money is spent on Google, Bing, Facebook and other PPC platforms and Trada handles the payment exchange between the SEMs and the company advertising, while taking a small percentage of that spend.

Trada’s CEO told TechCrunch that they hope to be “profitable by the end of 2012.”

In January 2011, the highest earning Optimizer (paid search or Facebook expert) in the Trada marketplace earned $4,500 for the month. Last month, the top earner made $14,500, or $175,000 annualized earnings. And at the beginning of 2011, the startup’s largest advertiser had a monthly budget of about $35,000 per month. Today, Trada’s largest advertiser budget is $500,000.

Tim Mayer, former Yahoo executive, formerly worked at Trada and recently left last month to found a new company called “Heddle.”

Related Stories:

• Crowdsourced SEM “Marketplace” Trada Raises $5.75M From Google Ventures & Others
• Tim Mayer: Search Vet & Ex-Yahoo Lands At Trada

Acquisio’s existing clients will gain access to the Channel Intelligence Shopping Engines and Channel Intelligence Product Search solutions. ClickEquations clients will now be managed and supported by Acquisio and will have access to all Acquisio services.

More information about the acquisition and partnership is available in this press release.

Adobe has agreed to acquire digital marketing agency Efficient Frontier, which has a strong search marketing practice and technology to manage Facebook advertising. The companies didn’t disclose the terms of the deal.

The companies said Efficient Frontier would become a part of Adobe’s Digital Marketing suite, which includes technologies and services it acquired with web analytics and optimization firm Omniture in 2009.

Adobe has long been building up its capabilities in the digital marketing space, largely through acquisition. Earlier this month, it bought Auditude, a video ad management and monetization technology firm. In January, 2011, it purchased Demdex, a data management and audience optimization company. In July of 2010 the company got into web content management with the acquisition of Day Software.

Efficient Frontier began operations as a search marketing company helping large enterprise clients utilize algorithmic technology — using financial trading as a model — to maximize their investments in the space. Since its start, the company has expanded into other digital marketing arenas, notably including Facebook and display advertising.

Adobe says the Facebook and Search-related capabilities will be integrated into its existing Adobe SearchCenter+ offering, and the related analytics will be built into Adobe SocialAnalytics. Efficient Frontier also uses search-like optimization technologies in the display arena via ad exchanges. Adobe says these capabilities will work with the Digital Marketing Suite and its Dynamic Ad Targeting solution.

Efficient Frontier itself last month acquired Australia-based Downstream Marketing, which it called the largest provider of digital marketing technology and services in Australia. Efficient Frontier’s clients include BabyCenter, The Motley Fool, Bankrate, Ask.com, and Home Away/Holiday Rentals.

The acquisition will have to go through the usual government and board-of-director approvals, but Adobe expects it to close early in 2012.

Why comScore Numbers Matter So Much

In today’s world of digital content powered by online advertising, advertisers and publishers alike keep a close eye on the numbers. Web measurement services such as comScore, Hitwise, Compete, and Nielson track the number of visitors to web sites.

In turn, these numbers are used in board presentations to demonstrate growth, by sales teams working to convince advertisers to make big campaign spends, and by advertisers deciding where to spend ad dollars. Venture capitalists make investment decisions based on traffic numbers related by these services. Google might choose to include particular sites to power its “OneBox” results based on this data.

We use the numbers to monitor industries and consumer behavior. Is Bing starting to gain on Google in search engine market share? Are consumers increasingly interested in watching video? What social networks are losing ground?

The stats on these questions can change how and where businesses spend their advertising budgets and where they spend time engaging customers.

Are the numbers accurate?  Well, no. They’re not accurate, but they generally are useful for looking at trends over time.

As these numbers gain value, it becomes more enticing to game them. And if the traffic numbers are being gamed for an unrelated reason (someone needs to hit particular numbers to get a good bonus, advertisers won’t buy unless the site gets more traffic, someone decides to boost ad revenue by injecting the site with more page views), the numbers reported by these measurements services can end up inflated as a potentially unintended side effect.

So should we trust the numbers? Mostly, yes. But it’s important to look at traffic sources, and not just traffic alone.

The Stats on Food

For over two years, allrecipes.com held a fairly comfortable lead in food-related web traffic according to comScore. As you might imagine, this gave them a powerful story to tell advertisers. It might have gone something like, “we’re the number one food destination on the web. Advertise with us for the best visibility to your target audience.”

“Target audience” is key. It’s the reason online advertising is so powerful. It’s why a company like Dole or Kraft would advertise on a food site rather than a car site or a general news site. Advertisers don’t want just any views of their ads; they want views by potential customers who are highly motivated to buy their products. And how much more targeted can you get if you sell food than to someone who’s making a grocery list of what to make for dinner?

When the comScore numbers for May 2011 came out earlier this week, Food Network had gained 3.4 million unique visitors month over month (an increase of 26%) and had taken over the number one spot.

Comscore: Allrecipes.com vs. Foodnetwork.com Traffic (May 2010 – May 2011)

The other measurement services had All Recipes still at number one and didn’t show the traffic spike that comScore reported. So what happened?

Hitwise: Allrecipes.com vs. Foodnetwork.com Traffic (May 2010 – May 2011)

Looking more closely at the data, nearly all 3.4 million new visitors to foodnetwork.com landed on a single page – a video page that autoplays continuous video and video ads.

Land on the page and the videos start playing. And keep playing. The intent behind this ad buy seems to be to increase views of their video ads, which have become big business.

Nearly all of the incremental traffic to this video page came from sites owned by a single network – sites that drove no traffic the previous month. In fact, comScore showed 92 sites driving traffic to foodnetwork.com in April and 196 sites driving traffic in May. (In contrast, All Recipes went from 93 sites to 91).

At least 77 of those new sites were affiliated with a single network: AdOn.

Buying Traffic: How the AdOn Network Works

The AdOn network works like many other ad networks do. They match ad inventory to publisher demand. AdOn (like many others) generally don’t work with publishers directly. Rather, they work with aggregators, who have publisher space available and are looking for ads. AdOn delivers the ads to the aggregators, who in turn display them on publisher sites.

The AdOn network has a click counter showing “clicks delivered today”. They’re dealing in high volumes of clicks.

They pitch their services as:

“We produce high traffic volume via a broad range of categories including cost per click (CPC) search, domain/toolbar, email, pop-under, TQ and conversion. Our advertisers get immediate access to our global partner network consisting of more than 1000 publisher sources, 1 billion search queries and millions of visitors. Our scale will help you satisfy your volume requirements.”

1 billion search queries? Wow. That’s nearly half of Bing’s search volume. Who are all these searchers and where are they searching from?

I talked to AdOn to find out. Robert McDaniel, VP of Product for AdOn, told me that they don’t own any of the sources of traffic. They work with aggregators who have pooled the “more than 1000 publisher sources” who are looking for ads (this would include parked domains, toolbars, popups, and popunders). When AdOn get a request for an ad, they call this a “query” (hence the 1 billion search query number mentioned on their site).

(Update: since this story was published, the AdOn website has been edited to say “over 1 billion queries for ads per day, and millions of visitors and ads served per day”. And AdOn provided us an additional comment to clarify that some of their traffic originates from search.)

Rather than send ad-generated clicks directly to the advertiser, McDaniel says AdOn redirects ad clicks through one of their owned search engine sites and then to the final advertiser destination. McDaniel told me that these owned sites have the look and feel of a search site, but are really just intended to process the traffic, not originate it. This implementation causes the traffic referrer to be one of these search sites, rather than the actual origination of the click (such as that parked domain).

In essence, it works like this:

1. A person clicks on an ad (which typically might be on a parked domain or on a popup that was spawned from a toolbar).
2. The click leads to one of the search engines in the AdOn network, which redirects to the advertiser site.

So while it looks to a measurement service like comScore that all of those clicks are coming from an AdOn search engine, they really aren’t. And those 1 billion search queries are really just non-search clicks redirected through one of AdOn’s search engine sites for processing.

Which makes more sense than actual searchers using these search engines, which have names like chillcow.com and happythat.com and all look suspiciously similar.

If you try to do an actual search on any of them, you’ll find that they return a very similar set of sponsored search results no matter what you search for.

Looking more closely at other Scripps-owned sites in Hitwise, it’s clear that HGTV, for instance, is also newly making use the AdOn network to generate additional traffic. The HGTV traffic also seems to be funneled to a single video page that autoplays videos and video ads.

Below you can see May 2011 Hitwise data for HGTV. The circled sites are AdOn search engine sites and the arrowed sites are parked domains; you can see they sent no traffic the previous month.

This traffic is boosting HGTV’s comScore numbers as well.

But Traffic is Traffic, Right?

Buying traffic is how advertising works. There’s nothing inherently nefarious in it. If you want a lot of new visitors to your site, you might buy ads to get those visitors. But the key again is “targeted” traffic. Traffic based on misleading people into clicking, from untargeted audiences who aren’t actually interested in what you’re providing, or that is automated doesn’t help anyone except someone looking solely for a graph to move up and to the right.

And buying untargeted traffic to view ads on your site (when the advertiser is paying for your targeted audience) might make you money, but the advertiser isn’t really getting what they paid for.

If Kraft is buying ads on the Food Network, they want to advertise to the Food Network audience, not to a random audience brought in from another source just to view those ads.

And if the ad network that the Food Network is buying traffic from is funneling the traffic through intermediate sites, such as these search engines, then the Food Network has no visibility into where that traffic is actually coming from and if the audience is really even interested in food.

(After this story was published, AdOn provided an additional statement that “we make the originating HTTP-Referer that we see as well as the keyword that generated the click available to our advertiser if they would like it.” They also stressed that they believe their traffic is targeted, telling us “We do geo and keyword based targeting like most other CPC/Text Ad networks.   Some of our advertisers buy ‘run of network’… [but], even with the run of network ads, there is some level of user intent when they chose to click on an ad.”)

The Impact on the Online Publishing Industry

When you get right down to it, we have access to so much free content online because online advertising works. Publisher sites can afford to stay in business because of the money that advertising brings in. And advertisers keep spending ad dollars on online advertising because it’s effective. Randall Rothenberg, President and CEO of the Interactive Advertising Bureau recently noted:

“As Americans spend more time online for information and entertainment purposes, digital advertising and marketing has emerged as one of the most effective tools businesses have to attract and retain customers.”

Quarterly Online Advertising Revenue (1999 – 2011): Interactive Advertising Bureau (IAB)

If online advertisers don’t see a return on investment for ad dollars spent on visitors viewing or clicking on their ads, they may stop advertising online, which would hurt all online publishers. The IAB’s has published Networks and Exchanges Quality and Assurance Guidelines that Yahoo’s VP of Marketplace, Ramsey McGrory noted are important for sellers to adhere to because otherwise “the buyers… aren’t going to have a level of comfort about that … ad network’s inventory.”

Their Audience Reach Measurement Guidelines note that “filtration procedures are necessary to ensure that non-human activities (for example, known or suspected robot/spider originating transactions) are excluded from measurement counts.”

These guidelines are intended to ensure that the traffic is both real and targeted.

The Quality of Traffic From Ad Networks

The AdOn network web site notes that their solutions provide “high-quality traffic”, so I asked AdOn’s McDaniel about the quality of the traffic generated by their network. He told me that while quality is a subjective measure, they do work with their advertisers to ensure the traffic performs. He acknowledged that the traffic “isn’t Google traffic. That’s why [advertisers] aren’t paying Google prices”, but said that “performance is a great measure of quality” and that they “want to deliver a quality product that is successful for advertisers.” (And he was adamant that all of their traffic was real, not automated.) He noted that they use Adometry to score the quality of the clicks on their network and “maintain a very low amount of clicks with scores of less than 100“.

These goals sound a lot like the goals of any ad network. But it’s important for advertisers to really dig into the details of what an ad network means by quality traffic.

Specific Media gathers data about audiences to deliver “relevant messages to specific audiences” to generate “maximum brand impact”.

“We’ve developed a sophisticated data mining system that allows us to use the billions of data points we gather across our network to enable advertisers to address the specific consumers they want to reach with relevant advertising based on demographics, interests, contextual relevance and geography.”

Google’s own Double Click also talks about using targeting to developing a relevant message:

“The targeting options in DFA help you deliver a more relevant message when advertising online. Choose from a variety of targeting options, including audience segment, geo, daypart, browser type, OS, keywords, and user lists for remarketing.”

The bottom line is that this issue isn’t about the AdOn network. It’s about publishers potentially buying unqualified traffic to deliver ad views to their advertisers when those advertisers are specifically buying access to that publisher’s audience.

AllRecipes.com president Lisa Sharples told me:

“Allrecipes has been the #1 food site according to comScore for the past 26 months so of course we were surprised to see that our position had changed. As brands shift more of their marketing dollars from traditional to digital formats such as video, mobile and rich media, it’s essential decision makers have access to accurate, credible audience analytics. Allrecipes is committed to developing impactful advertising formats that provide strong results for top brands. Advertisers can trust that when they buy ads from us, they are connecting with Allrecipes’ vibrant, active and engaged community. We are head down focused on integrating online video as part of a positive user experience within our existing community of home cooks, delivering a sound product to them and our advertisers. We applaud Nielsen, Hitwise, comScore and the IAB for their vigilance in ensuring questionable tactics are not tolerated.”

comScore Revises Their Food Network Numbers

comScore initially looked into the Food Network spike and found that since the traffic was mostly coming from searches, Food Network may have invested in search marketing that drove the numbers up. But as you saw above, those searches weren’t actually searches at all.

The original May 2011 comScore numbers show Food Network at 16,828,000 million unique visitors (and foodnetwork.com specifically at 16,760,000 unique visitors). Revised numbers out today include the note: “Food Network and Foodnetwork.com May 2011 Unique Visitors and Page Views were overstated due to inclusion of invalid traffic. Revised data for Food Network is 14,352,000 Total Unique Visitors… Revised data for Foodnetwork.com is 14,294,000 total unique visitors.”

I asked comScore for more information about why they determined this was invalid traffic, but they haven’t yet provided public details.

Hitwise they told me that they discounted most of this traffic during data analysis. They said they look at clickstream data to make sure the traffic is valid and for instance, discount traffic from pop ups, from bots, or is otherwise automated.

Ultimately then, both comScore and Hitwise have deemed the AdOn traffic invalid. AdOn says their traffic is valid. Where’s Food Network in all of this?

Did Food Network knowingly direct a bunch of invalid traffic to a page that plays video ads in order to get more video ad dollars or to appear to advertisers to have a larger audience? Or were they simply using an ad network as many publishers do in order to gain audience?

I was hoping to get more insight on this from Food Network themselves, but I haven’t yet gotten a reply to my emails and voice mails. Without their side of the story, it’s impossible to say for sure. (But I’ll update the story if I hear from them.)

How Often Are Stats Adjusted?

We’ve seen adjustments like this before. In 2007, Microsoft gained 3.1% share by way of its Club.Live.com game that had people searching as part of the game. Compete, which had initially reported the numbers later revised them to exclude Club.Live.com after an investigation. Hitwise explicitly exclude that traffic when calculating search share numbers.

In 2010, comScore devised a new metric called “explicit core search” when looking at search engine market share specifically in order to separate actual searches from “contextual search” (slideshows and other activities that trigger searches that the visitor didn’t explicitly perform).

How Adjustments Are Reported

Looking at the restated comScore report, you may notice that the original numbers are still listed and the new numbers are only included as a note. This is the standard way comScore reports adjustments, as Yahoo found out last year.

When comScore underreported Yahoo page views by more than a billion in June 2010, Yahoo posted the correction themselves, noting “Due to the size of the error, Yahoo! is making the announcement today as comScore does not generally issue restatements of its published reports.”

comScore said that they’ve had the same correction policy for ten years: they include the correction as a note in its client notification center.

“When a client data error is discovered after the data for the month is officially published, we investigate the root cause, recalculate the affected metrics and report our findings to the client for review. Once we are satisfied that the revised metrics are correct, we post them in the Client Notification Center, an equivalent of software ‘release notes’ that compile any known issues for the month. Clients can use the postings to report the corrected metrics in internal and external communications.”

The Bottom Line?

The measurement services all want to report accurate numbers as they have to remain credible sources for this data as advertisers rely on it more and more. Understanding traffic sources is important because online advertising only provides return on investment if the ads are being viewed by real people who are legitimately interested in the types of products and services provided by the advertisers. We all need to get past the narrow focus on page views (and even total visitors). We all have been saying for years and years (and years) that page views is a terrible metric, but we don’t really believe it.

Those organizations that make money from impression-based advertising understandably look at page views and number of unique visitors as a primary metric. More traffic = more ad revenue. But that can be short-sighted thinking. What’s important is increasing traffic of a qualified audience who is interested in what your site offers and what your advertisers offer, and who will return and will engage with both you and your advertisers. Without that, why would advertisers continue to invest ad dollars online?

Li is set to speak at 5:15pm Pacific, and he’ll be interviewed on stage by John Battelle. Live blogging to start shortly. There’s also a live stream here.

Don’t know much about Baidu? China’s largest search engine. BusinessWeek just did a big profile about it, which you can read here or get the highlights from our How Google Could Have Bought Baidu And Other Fascinating Details About China’s Largest Search Engine post.

John: Baidu is largest search company and portal in China, welcomes Robin on. Says it’s his first talk at an industry conference in the US. Mentions Eric Schmidt’s comment in the BusinessWeek profile as a great gift to Baidu.

Robin: 5 years ago, people asked him what would you do to win China. Said he would stay 6 months to a year in China. Eric didn’t take his advice.

John: Your stock is double since January, market cap is pushing $40 billion.

Robin: Puts up roughtly to eBay.

John: Do you feel pressure to continue that rocketship ride?

Robin: When I founded this company 10 years ago, I never knew that search could be so profitable. I really liked to search and thought search could be useful. Used by 100s of millions of people, and I knew I could achieve that. Now, I don’t need more money, but I need to make our product better. That’s the driver behind my daily work. Not the stock price. Maybe some know 5 years ago when public, the Baidu price was $27 and closed at $22 (I think). That was more pressure. The stock has been up or down every year. I’m used to that. What’s important that the users keep growing, other things keep growing. We have plenty of room for growth. In US took about 10 years for internet market to mature. In China, started at roughly same time in 1995, but internet penetration is only about 1/3.

John: But that’s like 400 million people. How big can it get?

Robin: Lots of mobile phones, thinks can get to 800 million (think he said).

John: How many use Baidu?

Robin: about 99%. We have a lot of coverge (laughs from the audience). Answer more queries than any other search engine in their market.

John: Including Google in the US.

Robin: Yes [me: worldwide, Google reported by comScore last year as handling still far more than Baidu worldwide, I'll try to drop a link later).

John: Google got the sense with hacking and govt interference they weren't on a level playing field and China was a favored son.

Robin: That's a common misperceptions. People think there are no choices and Baidu is favored. But the reality is there are more choices in the US than in China. Here you have Google and Bing and what's the number three? In China, named one (didn't catch), and there's Sodu and there are a couple more. Many other Chinese companies are doing web search and places from outside China are offering it.

John: Why else do you think Google wasn't successful in China?

Robin: China is a very different market. It's a large and growing, meaning the market condidtions change every day, and you have to be close to it. Second, lots of VC money, lots of Silicon Valley VCs poured money into it. Third reason is that there are a lot of Chinese engineers here in the US that are very well trained, many of them are willing to go back to China and start their own company or join an existing one. If you're not prepared to compete in this type of market, you're not going to be successful. With Baidu, I think we did try harder. When I moved back to Beijing 10 years ago, I gave up all my stock options here (he was an Infoseek engineer).

John: You wrote a PageRank like algorithm before?

Robin: Before hired, Infoseek liked his research. Had the first patent on it in the US he believes. But he gave all that and focused on Chinese search. We were close to the market, came up with a lot of innovative ideas. For example in 2003, before Web 2.0 idea was coined, they started adding user generated content into their results. A query based online community, they have a bar or message board system where you can ask question, answer those from others or make comments. In early days, there wasn't enough Chinese information online. So we made this product [kind of like how YouTube created video content that in turn helped power a video search revolution]. Chinese users have already answerd 100 million questions on the system. We added a lot of social community so our users feel they really belong here. It’s very hard for them to leave.

John: So do you have Holy Grail of social and search that Google and Facebook seem to be looking at?

Robin: [Sorry, didn't catch first part]. System is designed to make people feel they belong, introduces high switching costs.

John: Would it make sense to have Facebook Connect in Baidu?

Robin: We don’t have a full blown social graph yet. But happy to add that.

John: Mark Zuckerberg will be here tomorrow, so I’m sure you’ll be happy to talk he jokes.

You’ve worked in both markets, what is it like to build a company with an entrepeneurial culture in China, with a very different environment and government. What can we learn?

Robin: China is an emerging market, not everything is developed, so you have to be patient. In US, company can be set up in 1 day, in China, it can take 20. He said it’s OK, hired a consultant, got office space while waiting for license. It doens’t hurt so much, as long as you keep open minded and control what you can. You can’t control censorship or slow internet connections.

John: There’s a perception in the US here that there’s no censorship versus China, that you search the whole web. Is that a frustration for you, along with low bandwidth.

Robin: My first reaction was also let’s move to Hong Kong (over censorship issue, when he first started). But I realized that wouldn’t work. If I were to move to Hong Kong, they’d call me some type of anti-government company, and my life would be ruined. If an American company moves, they’ll still call them “strategic partners.”

John: Found it interesting that the Chinese govt announced it might start a search engine, since Baidu has so much share.

Robin: It’s not a new thing. It’s OK. Providing more choices to the Chinese users is not a bad thing, but we are allowed to compete.

John: It would be unthinkable for the US to do such a thing.

Robin: China has a very strong govt.

John: I think you made an understatement.

Robin: I’m not worried.

John: Let’s talk about mobile. How is that developing in China, and if you ran a mobile app start up, what would be the things you should know.

Robin: There are about 800 million mobile phones but large portion of them aren’t internet enabled, maybe 150 million. But there’s a catch. Most of them use 2G phones to go on the internet, which means it’s slow and expensive, carriers charge by number of bytes. Most high-end people who use them don’t care about the cost. Vast majority of Chinese people are low income, can’t afford a PC, nannies or migrant workers, at night they lie back and go on the internet. It’ll change as the mobile network grows, user behavior will grow, and product will change, but we don’t know yet what the killer app will be. The status is not really stable. We don’t even know what the most poular device to go online.

John: Could Baidu provide it, like Google does with Android.

Robin: Our box idea is that something that boots up within one second and you get a box that lets you do whatever you want to do. It replaces the operating system. you can search, find apps, publish your tweets. The box is intelligent enough to understand.

John: How do you execute that vision?

Robin: One key part is to understand intention. People put all types of queries into their search box, so they know for certain user needs, like weather, there are probably 200 different needs to express the same meaning. So we can better understand intention, and once we have it, it’s easier to deliver it.

John: Lots of entrepeneurs here. One of the great stories is the moment when offered an ungodly amount of money to sell their company and they say no. That narrative is mythic in the valley. That’s happened to you a number of times. $1.5 to 2 billion offered by Google, Yahoo and Microsoft. Why say no?

Robin: Because I knew Baid u had more potential than 1-2 billion. At that time, China’s market was still knew. In 5 years, revs gone up 30 times. Don’t think Google Yahoo Microsoft knew that.

Q&A Time: Why doesn’t India have its own Baidu?

Robin: Most of India speaks English, writes in English, so Google already solved the problem. For China, it was very different. When I went back, people used to wait for 5 seconds to get answer. No space for words in Chinese language, you have to parse it differently. We had to help users create content and the speed of information growth was very high. In the first 3-5 year, we basically tripled our index size. Search engines here think overall growth 30-50 percent, then I’ll grow my index by that, and it’ll be good enough. But in China, we were able to keep up with the growth.

Question: What can the US learn from China?

Robin: Most of the ideas China learned from the US and its early start. But in recent years, there are innovations coming out for search, because we have the largest market. The addition of social layer makes search sticky and hard to switch. For social, Chinese company called 10 cents [I think] with hundreds of millions of users, started with instant messaging and added much to their platform. These kind of things, maybe US can pay more attention. Going forward, you should pay more attention to that market [the Chinese market] because it’s so large, and they’ll encounter new problems, and you’ll start to innovate.

John: Are you coming to the US?

Robin: We do have ambitions to expand outside of China. Been in Japan for two years. But US is probably not a high priority because it’s a large and mature market. We’d have to thnk carefully about it.

And that’s it.

Baidu holds a 73 percent market share in China, the world’s largest internet market. The company is worth $38 billion and is 57 percent larger than Yahoo. Its shares have doubled in value since Google “withdrew” from China to Hong Kong earlier this year.

Baidu CEO Robin Li (41) got a masters at SUNY Buffalo. He once worked at Infoseek and is now the second richest man in China.

The piece contains a number of allegations, which Baidu denies, about unethical practices:

For years legions of advertisers have complained on Chinese Web forums that Baidu secretly penalizes the search rankings of websites that decrease their spending on Baidu . . . In 2008 the company quickly denied Internet chatter claiming it had taken money from Sanlu Group, a dairy producer that had sold milk powder tainted with the toxic chemical melamine, to keep the scandal out of search . . .

Baidu says it is not a kingdom built on Internet piracy, though music companies say its popular MP3 service allows users to download just about any song ever recorded for free. The recording industry sued in 2005, but Chinese district and appeals courts sided with Baidu . . .

Censorship is also willingly practiced by the company:

Like all Chinese Web companies, and Google’s Chinese language site until this year, at the behest of the Chinese government Baidu blocks pornography or references to topics such as Taiwanese independence, the Dalai Lama, and the 1989 Tiananmen Square massacre. The practice is called zi lu, or “self-discipline,” and Baidu does it well. Last year the company accepted one of 20 awards from the Internet Society of China, given for what the group calls “industry self-regulation.”

Baidu was the direct beneficiary of Google’s ouster from the mainland. It’s unlikely that Baidu had anything to do with the GMail hacking incident that caused Google to declare it would no longer cooperate on censorship. But there have been persuasive allegations that the Chinese government or its surrogates were behind the hack attack. If so perhaps the government was trying to indirectly bring about what has in fact happened: the marginalizing of Google in China and the elevation of a cooperative partner, Baidu, to near monopoly status.

Google saw itself as a force for change and helping open up Chinese society by degrees. The Chinese government obviously doesn’t want any part of that.

The story also touches on another success issue, that the Chinese government may have helped Baidu with making Google unreliable:

Google never materialized as a threat, for reasons that no one can agree on. What’s clear is that Google’s China service was mysteriously unreliable, particularly when the Chinese government was angry with the U.S. Google also documented several instances when it was the victim of what’s called domain name system poisoning; users typed in Google.cn and found themselves at Baidu instead.

A New York Times Magazine article from 2006 details this more:

But even after being unblocked, Google still had troubles. The Great Firewall tends to slow down all traffic coming into the country from the world outside. About 15 percent of the time, Google was simply unavailable in China because of data jams. The firewall also began punishing curious minds: whenever someone inside China searched for a banned term, the firewall would often retaliate by sending back a command that tricked the user’s computer into believing Google itself had gone dead. For several minutes, the user would be unable to load Google’s search page — a digital slap on the wrist, as it were. For Google, these delays and shutdowns were a real problem, because search engines like to boast about delivering results in milliseconds. Baidu, Google’s chief Chinese-language rival, had no such problem, because its servers were located on Chinese soil and thus inside the Great Firewall. Worse, Chinese universities had virtually no access to foreign Web sites, which meant that impressionable college students — in other countries, Google’s most ardent fans — were flocking instead to Baidu.

Still Baidu must constantly “curry favor” with the communist regime. According to the article, government displeasure can result in investigations or critical profiles in state-run media:

In November 2008, government-operated China Central Television aired several in-depth investigations about Baidu surrounding allegations that the company earned millions on ads from unlicensed medical providers, and prominently displaying these ads when users typed health-related queries. The first story ran on Li’s 40th birthday, timing that no one at Baidu dismissed as coincidental. The following quarter, Baidu boosted its ad spending by 41 percent, with the bulk of the increase going to CCTV. Few Chinese Internet watchers thought it was a coincidence when the negative coverage ceased.

Baidu now wants to expand horizontally and globally, “into games, e-commerce, and online payments, establishing a Hulu-like video site called Qiyi, and exporting the Baidu brand abroad.” Li is quoted in the article saying he wants Baidu to be a global internet power:

I hope in ten years, Baidu will become a household name in 50 percent of the world. Sooner or later you will see a China-based company that really has a global impact and I think Baidu has a chance to become one of those companies. We should be able to compete on a global basis.

Generally Li comes off relatively well in the article, as a humble Chinese engineer living in the US who was inspired by Yahoo’s co-Founder Jerry Yang. Li had come up with a “link analysis” system similar to Page Rank. And there are other interesting parallels between Baidu and Google, not all of them coincidental.

Though Baidu is Chinese through and through, Silicon Valley money started the company. And early investors made a huge killing, comparable to what happened with Google, when it went public. Interestingly Baidu initially modeled itself on inktomi, which powered search at Yahoo before Google came along.

Li and Xu were much more interested in being the next Inktomi . . . Baidu licensed its search index to Sina and Sohu.com, then the dominant portals in China, and charged them each time a user conducted a search. Busy copying Yahoo’s portal business model, those companies didn’t realize search’s potential, concedes Charles Zhang, chairman and CEO of Sohu. “

Later the company developed its own site when its relationships with the Chinese portals went sour. It was at this point that the company was almost acquired for $40 million by Yahoo:

Finian Tan, then a Hong Kong-based partner at ePlanet, says he was so worried about the company’s prospects that he contacted Jerry Yang, an ePlanet investor, and offered to sell Baidu to Yahoo for $40 million . . . [Yang] referred the matter to a Yahoo colleague, who did not respond. (Yang declined to comment for this story.) Baidu had no choice but to devote its energies to its own website. “We became Baidu.com because we were forced into it,” Tang says.

That wasn’t the only acquisition attempt. There were several near acquisitions of Baidu before it went public, which sound very similar to the early history of Google.

In 2004, “Google contributed $5 million to Baidu’s $15 million third round of financing . . . [the] investment was meant to lay the groundwork for a possible acquisition and ward off the possibility that rivals could buy it.” Later, SoftBank founder Masayoshi Son “wanted to keep Baidu private and offered to invest $100 million at a $1 billion valuation. Yahoo and Microsoft  made acquisition offers at a little over $1 billion . . . Google … ultimately made a $1.6 billion bid.”

One of the early investors and Baidu board member, Asad Jamal, said that “a $2 billion bid would have cemented an acquisition. ‘I personally think that was a missed opportunity for Google.’”

Missed opportunity? Clearly.

Or maybe not.

The bold move? It issued its first quarter 2010 financial results on investor.google.com and said that future financial performance numbers would be announced on the web site. Shocking for a web company in this day and age of the web, no? Right. No. And the mainstream media and investor community that was so outraged? That would be Reuters. And… well, mostly Reuters. We at Search Engine Land happened upon this story on Friday morning (April 16th) and we had a bit of fun with it amongst ourselves. It read a bit to me like a press release itself for, well, press releases. What kept it from seeming more journalistic was a striking lack of sources and an abundance of, to use a technical term, crazy talk.

For instance:

“some worry that this trend [of issuing earnings statements on a company web site rather than via a press release] may harm individual and less-sophisticated investors who cannot access the blogs and websites as quickly as professionals. Others worry that not everyone will get the information.”

Questions: who are “some?” Who are “others?” Are “individual and less sophisticated investors” really less able to access blogs and websites than “professionals?” Seriously? Part of an investor’s professional training is to learn how to click a link or type a URL into a browser?

This same sourceless yet all-knowing “some” apparently worry that this move could therefore “disadvantage some investors”. And as Dominic Jones pointed out, saying that “the statement posed a brief obstacle for the media, analysts and others hungry for Google’s numbers” is like saying air poses a brief obstacle to forward movement. OK, he didn’t say that exactly. He called it “ridiculous”. And unless the faceless “some” has some inside knowledge that all of the media, analysts and others hungry for Google’s numbers have spent their entire lives in caves, devoid of electricity and light and therefore have no knowledge of the web, it’s unlikely many of them would call clicking a link an “obstacle”.

But Jones’s most telling point is around the core assertion of the Reuters article: “It [the move] may also suggest the company is headed down a road that could hurt companies that distribute press releases.” Jones points out that Reuters itself recently bought a company that distributes press releases (yet failed to disclose that in the reporting of the story).

So my initial read on the Reuters story wasn’t so far off. It was a bit of a press releases for press releases.

I’m not even going to get into the notion that Google is now headed down a road that might impact another industry. Surely that’s something they would never do.

Here are some verbatim excerpts from the SEC filing:

Negotiation and Execution of the Definitive Agreements

Pursuant to the terms of the Letter Agreement, the parties will negotiate and execute the Definitive Agreements as soon as practicable but in any event by October 27, 2009 (the “Negotiation Period”). If the Definitive Agreements are not executed during the Negotiation Period, the parties will submit any disputes regarding the final terms of the Definitive Agreements to an arbitration panel.

Conditions to Commencement and Termination Prior to Commencement

Prior to the Commencement Date, the Letter Agreement and Definitive Agreements may be terminated only by (a) mutual consent, (b) if a breach renders a condition incapable of being satisfied by the Termination Date (as defined below), or (c) if the conditions to commencement have not been satisfied by July 29, 2010 (the “Termination Date”); provided that Yahoo!, in its sole discretion, has the right to extend the Termination Date by six (6) months if the required antitrust approvals have not yet been obtained.

Search and Advertising Services and Sales Agreement

For a period of ten (10) years beginning on the Commencement Date (the “Term”), Microsoft will be Yahoo!’s exclusive technology provider for algorithmic and paid search services and Microsoft will provide contextual advertising to Yahoo! on a non-exclusive basis. Yahoo! will be the exclusive worldwide relationship sales force for Yahoo!’s and Microsoft’s premium search advertisers.

The services provided by Microsoft under the Search Agreement will be provided on all web sites, applications and other online digital properties owned or operated by or on behalf of (a) Yahoo!, Yahoo! subsidiaries and Yahoo! joint venture relationships, as well as on software applications developed or distributed by Yahoo! or Yahoo! subsidiaries that provide access to or enable algorithmic search services or paid search services (“Yahoo! Properties”) and (b) Yahoo! Syndication Partners (as defined below), as well as software applications developed or distributed by Yahoo!’s Syndication Partners that provide access to or enable algorithmic search services or paid search services from Yahoo! (“Syndication Properties”). “Syndication Partner” means a third party with whom Yahoo! has contracted to provide algorithmic search services or paid search services.

Subject to certain specified restrictions, Yahoo! will have full flexibility with respect to the user experience, content and look and feel on all of its web pages, and will also be entitled to use the paid search services and algorithmic search services for non-internet search queries with minimal restriction . . .

Microsoft’s mapping services and mobile search services. Yahoo! may implement each of the mapping services and the mobile search services on a non-exclusive or an exclusive basis. Yahoo! also has the option to work with Microsoft to implement the services on other platforms. If Yahoo! elects to receive services for other platforms, it must receive such services on an exclusive basis.

Revenue Share Payments and Other Payments

During the first five years of the Term, Yahoo! will be entitled to receive 88% of the net revenues generated from Microsoft’s services on Yahoo! Properties (the “Revenue Share Rate”). Yahoo! will also be entitled to receive its share (at the Revenue Share Rate) of the net revenues generated on Syndication Properties after the Syndication Partner’s share of net revenues is deducted. For new Syndication Properties during the Term, and for all Syndication Properties after the first five years of the Term, Yahoo! will receive its share (at the Revenue Share Rate) of the net revenues generated from Microsoft’s services on Syndication Properties after the Syndication Partner’s share of net revenues and certain Microsoft costs are deducted.

On the fifth anniversary of the Commencement Date, Microsoft will have the option to terminate Yahoo!’s sales exclusivity for premium search advertisers. If Microsoft exercises its option, the Revenue Share Rate will increase to 93% for the remainder of the Term, unless Yahoo! exercises its option to retain its sales exclusivity, in which case the Revenue Share Rate would be reduced to 83% for the remainder of the Term. If Microsoft does not exercise such option, the Revenue Share Rate will be 90% for the remainder of the Term.

Microsoft will also pay Yahoo! a payment of $50 million annually during the first three (3) years of the Search Agreement. Yahoo! may use these payments to partially cover transition and implementation costs not otherwise covered under the Search Agreement.

Termination Provisions

Yahoo! may terminate the Search Agreement if the trailing 12-month average of the RPS in the United States (the “U.S. RPS”) of Yahoo! and Microsoft’s combined queries falls below a specified percentage of Google Inc.’s (“Google”) estimated RPS measured on a comparable basis or if the combined Yahoo! and Microsoft query market share in the United States falls below a specified percentage; (d) on the fifth anniversary of the Search Agreement, and any time thereafter, Yahoo! has the right to terminate the Search Agreement if the trailing 12-month average of Yahoo!’s U.S. RPS is less than a specified percentage of Google’s estimated RPS; or (e) subject to exceptions, either party may terminate if a law, regulation or order would have a significant, adverse impact on a primary aspect of such party’s intended benefit of the Search Agreement.

Transition and Implementation Plan

Microsoft will hire not less than 400 Yahoo! employees (the “Transferred Employees”) and will offer the Transferred Employees market competitive compensation packages. In addition, Yahoo! and Microsoft will mutually agree on a retention plan to be paid for by Microsoft to assist in retaining the Transferred Employees and an additional 150 Yahoo! employees to be mutually agreed upon between Microsoft and Yahoo! to assist with providing the transition services.

The deal seems broader than the “web, image and video” search scope we heard before. Yahoo can terminate the deal if certain targets aren’t meet surrounding revenue per search, benchmarked to Google. At the five year mark the parties can change who runs “premium sales.” The rev share percentages change accordingly.

On a related note, there’s an opinion piece in the Wall Street Journal that argues these terms give Redmond a bigger win than Yahoo in Microsoft CEO Steve Ballmer’s “win-win” scenario but that, in the larger scheme of things, Google may have already won the game:

The deal is a clear win for Microsoft and a qualified win for Yahoo. The big question is whether it makes any difference in the only contest that really matters, which is the one with Google. The risk for both Microsoft and Yahoo is that the contest is already over. Second place won’t really matter, especially as the competition shifts to Microsoft’s home turf: operating systems.

Related coverage:

• It’s Finally Official, Microsoft & Yahoo Make A Deal, Yahoo Gives Up On Search
• Live Blogging The MSFT – YHOO Search Press Conference
• Microsoft-Yahoo Deals 2008 & 2009, Side-By-Side
• A Search Eulogy For Yahoo
• Micro-Hoo Details: Q&A With Mehdi & Schneider
• The Microsoft-Yahoo Search Deal, In Simple Terms

Well, in the long run, it’s probably a good thing for most of the players involved, including users. My original concern when rumors of Microhoo started to fly was the distraction that the integration of two very different and somewhat challenging cultures would become. I was afraid that the search user experience would get lost in the process. But the revenue share split does away with that concern, which is good for users. Yahoo can continue to be Yahoo, minus the “distraction” of search. Microsoft can get serious about search with a more viable economic engine to justify their investment. Basically, the deal gives Microsoft a whole lot more eyeballs for Bing, and that’s going to be beneficial for everyone.

But there are some other reasons why the search user could benefit from this deal:

A release of talent

There’s a whole bunch of PO’d, disenfranchised search engineers that have just had a frightening but potentially liberating view of the future: freed from the shackles of the rapidly sinking S.S. Yahoo, they might actually have a chance to do something meaningful in search.

I’ve had the opportunity to talk to various members of the Yahoo! search team over the last several years, under Semel, Yang and Bartz. The underlying current was always one of thinly veiled frustration. They deserved better. They were smart and, once upon a time, passionate about search. But for the past several years they were left hanging out there, blowing in the breeze. They yearned to do something meaningful. Now, with the ladder kicked out from underneath them, they have no alternative. Whoever hasn’t already jumped ship will be forced to find new, and hopefully, more rewarding homes.

Pushing the wolves back from the door

At the risk of sounding like a broken record, or, less anachronistically, a skipping MP3 file, search needs a major shot of innovation. The dysfunctional competitive environment that we’ve had for the last few years put all the major players in a position where innovation wasn’t encourage. Specifically:

Google. As the market leader with a major marketshare, almost completely dependent on the search advertising revenue stream, it’s hard to rock that boat too vigorously. Every minor change in the search UI could have a potentially negative impact on ad clickthroughs. The dynamics of the SERP are subtle and small changes can have big impacts on behavior. Given the obsessive focus on quarterly revenue numbers by Wall Street analysts, and the macro economic impact of the recession, this was not the time to boldly screw around with your golden egg-laying goose. Google always speaks a good game when it comes to hyping their efforts to innovate in search, but come on, how different is the Google SERP now from what it was 4 years ago? How innovative have they been with AdWords?

Microsoft. Microsoft was the one who was really in a position to innovate, but despite repeated commitments to search from everyone from Gates on down for the last several years, the company’s track record has been less than stellar, to be exceedingly kind. Bing is the first sign that they’re motivated to get it right, and that’s really more of a catch up move than anything. But at least they’re in the game now. I think the biggest problem was getting alignment in the gargantuan, lumbering Microsoft corporate structure. There was a lot of left hand/right hand lack of communication from Spanky and the gang in Redmond. It’s hard to innovate when you don’t even know who’s working on what.

With Bing, Microsoft showed they can execute. Now, let’s see if they can innovate.

Yahoo. You don’t innovate on the Titanic: you just try to survive. Nuff said.

What the Microsoft-Yahoo deal does is allow Microsoft’s search champions to better sell the need to innovate within the company. They have a bigger market share, can forecast significant revenue potential and make the case for investing in rapidly upping the search experience bar.  Microsoft can afford the fluctuations that are inevitable in ad revenue as they experiment with the UI, because search is a pretty small drop in the revenue bucket. And it’s certainly not that they don’t have the resources—all they need is the corporate prioritization and alignment. This deal may just do that for them.

A bigger ad inventory

Microsoft CEO Steve Ballmer sent the signal of the true value of a Yahoo partnership when he said last December:

“We’re fully prepared to compete without any partnership with Yahoo. We don’t need to act. Would it be advantageous for both of us to make a deal? Look, the fundamental basis for doing the search deal with Yahoo has to do with critical mass in the advertising marketplace. It doesn’t have to do with technology, or any of these other things, it really is a market phenomenon. Together we would have more advertisers… which means we’d have more relevant ads on our page. We’d have higher monetization levels possible in front of us because there would be more people bidding on more key words. Most importantly, Google would have perhaps a real credible competitor sooner.”

Success in search is all about relevancy. And if ads are going to be part of the results set, those ads better be as relevant as possible. The bigger the ad inventory, the more relevant they will be.  It’s a pretty simple equation.

And ad relevance is massively important to the user experience. At Enquiro we’ve done several studies that either directly or indirectly examined the importance of ad relevance to the overall search experience. The results are abundantly clear: the better the ads, the happier the user. The deal gives Microsoft a massive increase in advertiser inventory, and that’s perhaps the most important asset in the deal. It doesn’t matter that Yahoo gets 88% of the revenue. That’s short term thinking.  This is about running all those ads through the Microsoft platform, giving them the ability to control quality, targeting and relevance. The goal is gaining market share, and you need the highest quality ads possible to do that.

A stronger commitment from advertisers

No longer is an ad buy on Microsoft tossing them a bone out of guilt or a gesture of protest against a Google monopoly. With a combined market share pushing 30 points, that’s a serious slice of available eyeballs. Microsoft just became a mandatory buy. That allows them to build stronger relationships with advertisers, getting a more serious commitment in return. If Microsoft can pull the pieces of AdCenter together like they did with Bing, they’ll have a pretty powerful ad management platform. And Microsoft doesn’t suffer from the same passive-aggressive relationship with marketers that still lingers in the halls of the Googleplex. Advertisers look at Google with a mixture of resignation, fear and resentment. More than a few will willing fall into Microsoft’s embrace, now that there’s a justifiable reason to do so.

How does this help the user? Happy advertisers mean happy corporate execs up in Redmond, which means more of an appetite for innovation. Expect Microsoft to be more innovative in offering sponsored search formats. And happy advertisers also means more relevant ads (see my previous point).

More serious competition for Google

This deal sends a signal Google can’t ignore. They’re still in the search driver’s seat, but at least now they can see someone in their rearview mirror. If Microsoft can adopt a passion for innovation and push the envelope, Google will have to respond in kind. The search experience will evolve more rapidly, hopefully kicked out of the revenue obsessed stasis that it’s currently in. Stagnation benefits no one except the analysts and bean counters who insist that quarter over quarter performance is the only metric that matters. We’re way too early in the game to be that cautious and boring.

Will the Microsoft-Yahoo deal break the Google habit? No. In fact, Google will probably net a couple more percentage points out of this in the short term. But this lays the foundation of a more competitive market place, which can’t help but benefit users.

The deal puts Microsoft in the game. Now, let’s see what they do with the opportunity.