Saying the host site was serving malware to users, Google has removed a controversial photo of First Lady Michelle Obama from Google Image Search. The site itself, however, remains listed in Google web search results without any visible malware warning.

Welcome to the murky world of free speech, politics, and Google.

It began last week, when Search Engine Roundtable pointed out a racist image showing as the number one result in Google Image Search for the term [Michelle Obama]. The image was apparently removed yesterday.

In a Google Web Search Help Forum thread discussing the image, a Google employee named Jem explained yesterday that there are three reasons why Google would remove content from its index:

“… Google views the integrity of our search results as an extremely important priority. Accordingly, we do not remove a page from our search results, or images from our Google Images results, simply because the content is in very poor taste or because we receive complaints concerning it. We will, however, remove pages from our results if we believe the image, page (or its site) violates our Webmaster Guidelines, if we believe we are required to do so by law, or at the request of the webmaster who is responsible for the image.“

(emphasis mine)

An offensive photo of Michelle Obama doesn’t obviously violate any of those three guidelines on its own. Google’s press office has yet to respond to our request for an official statement. But in the comments of today’s Search Engine Roundtable post, Google’s Matt Cutts says the site was violating Google’s webmaster guidelines:

“… that page did violate our webmaster guidelines because it was serving malware to users, which violates the quality guideline that says ‘Don’t create pages with malicious behavior, such as phishing or installing viruses, trojans, or other badware.’ I believe that the Images team did a general anti-malware sweep.”

Sure enough, a [site:] search for the site in Google Images produces no results. But the malware sweep apparently didn’t reach the main web search index. As Michael Gray points out on SER, the site itself is still listed in Google.com search results with no malware warning.

Making matters slightly murkier is that, as you see above, the image was hosted on Google’s own blogging platform.

Google’s critics will no doubt call this favoritism toward the Obama administration and be quick to point out the company’s ties to Washington, DC. Google CEO Eric Schmidt endorsed Obama for president, later campaigned with him, and then turned down an offer to join the administration. The Wall Street Journal has reported that Google was the fourth-largest corporate contributor to Obama’s presidential campaign. And we’ve reported about a few notable Google employees who’ve left to work for the Obama administration.

On the other hand, we’ve also written more than once about potential friction between Google and the Obama administration. Christine Varney, the Assistant Attorney General for Antitrust has been quoted as saying that Google “has acquired a monopoly in internet online advertising.” See our stories Will Obama Be The Downfall Of Google? and Google’s Anti-Trust Problem Appears Very Real for more.

If, in fact, the blog hosting the offensive image of Michelle Obama also hosts malware, Google’s removal of the image seems justified in light of the company’s stated policies. But, in that case, a malware warning should also be placed on the site itself in Google’s main search results. Until that happens — and perhaps even after — Google’s critics are likely to question the decision to remove this image.

While today’s episode may be a case of Google looking for an excuse to remove an image from the index, it should be said that the same exact image of the First Lady can be found on other sites and remains in Google Image Search because those sites apparently don’t meet the criteria for content removal. There are similarly offensive images of the President himself that can be found quite easily in Google Image Search, too.

Backtracking on a controversial decision earlier this year, the National Association of REALTORS® (NAR) has adopted a new policy that allows real estate professionals to have their sites — including home listings that belong to others — indexed by search engines.

The controversy reached a peak this year when the NAR agreed with a local decision in Indianapolis that said real estate agents couldn’t let Google and other search engines index the property listings on their sites if those listings belonged to other brokers/agents. In March, the Indianapolis board sent a letter to some agents that essentially called search engines “scraper” sites:

“…the National Association of REALTORS® (NAR) is in agreement with our interpretation of the policy that the above described practice of ‘indexing your Web site’ as you have called it, is a method of scraping or reproducing the data”

At their meeting this week, the NAR Board of Directors revised its policy on home listings and search engines to say that participants “are not required to prevent indexing of their Web sites by recognized search engines.”

That Realtor.org link is also interesting for its comments about the development of REALTORS® Property Resource, which many say is essentially a national search engine for property listings.

The long-running debate over Google and its impact on newspapers and journalism took another turn today when News Corp founder Rupert Murdoch said his company may makes its sites invisible to Google, and Google fired back by saying, in essence, bring it on.

It began with this interview on Australia’s Sky News (which Murdoch owns), reported by the Australian site mUmBRELLA, in which Murdoch reiterated his belief that Google and other search engines “steal” their stories:

“The people who just simply pick up everything and run with it, and steal our stories. We say they steal our stories — they just take them without payment. There’s Google, there’s Microsoft, Ask.com … there’s a whole lot of people.”

Murdoch agrees that Google sends his news sites a lot of traffic it might not have gotten on its own, but questions the value of that traffic to advertisers. He says, plainly: “We’d rather have fewer people coming to our web sites, but paying.” And when asked why his sites haven’t made themselves invisible to Google and other search engines/news aggregators, Murdoch says: “Well, I think we will. But that’s when we start charging.”

To that, Google fired back today, telling the Telegraph that, essentially, they don’t care if Murdoch wants to block its sites from being found via search and/or Google News.

A spokesman for the search giant said: “Google News and web search are a tremendous source of promotion for news organisations, sending them about 100,000 clicks every minute.

“Publishers put their content on the web because they want it to be found, so very few choose not to include their material in Google News and web search. But if they tell us not to include it, we don’t.”

Google is referring, of course, to using robots.txt or a similar protocol to keep content from being indexed. Danny Sullivan wondered aloud why Google’s critics in journalism weren’t already doing that, especially after the Wall Street Journal recently accused Google of “encouraging promiscuity” online by allowing searchers to bounce from one news site to the next with no loyalty. Danny also sat down recently with Google CEO Eric Schmidt for a lengthy conversation about Google and journalism.

The debate/battle is far from over. The question now, at least where Google and Murdoch are concerned is: Who’ll blink first?

Postscript: Bill Tancer has posted about this on the Hitwise blog today, specifically with a look at the traffic that Google sends to the Wall Street Journal’s web site:

“…on a weekly basis Google and Google news are the top traffic providers for WSJ.com account for over 25% of WSJ.com’s traffic. Even more telling. According to Experian Hitwise data, over 44% of WSJ.com visitors coming from Google are ‘new’ users who haven’t visited the domain in the last 30 days.”

Note From Danny Sullivan: I’ve also posted some related thoughts today in my Why An Exclusive Wall Street Journal Deal Wouldn’t Help Bing article.

Google has launched a new privacy dashboard — technically just called Google Dashboard — that gives users quicker access to, and more control over, the personal information stored in Google’s databases. The dashboard is a one-stop shop for managing this data and the settings that are associated with the Google products you use when signed in to your Google account.

“We recognize how important our users’ trust is, so we’re looking for ways to be more transparent,” says Shuman Ghosemajumder, Google’s Business Product Manager for Trust & Safety. “Over the last 11 years we’ve launched a lot of products, so we wanted to provide more transparency for people using those products.”

How to access Google Dashboard

Google Dashboard can be reached by going to www.google.com/dashboard or by going to your Google Account page. You’ll see a link under Personal Settings that says “View data stored with this account.”

You’ll have to be logged in to your Google account first and, because the information in Google Dashboard is sensitive, Google requires a second login before you can access it.

What’s included in Google Dashboard

At the beginning, not all Google products are included in Google Dashboard. Ghosemajumder says there were some “last-minute technical issues” that kept some products out of Dashboard. Google hopes to add those within the next couple weeks. For now, here are lists of what’s in and what’s not:

Included: Account & Profile, Web history, Gmail, Docs, Calendar, YouTube, Blogger, iGoogle, Latitude, Reader, Talk, Health, Orkut, Picasa, Shopping List, Voice, Contacts, Alerts, Finance, Friend Connect, Tasks, Custom search engines, Mobile Sync

Not included: Checkout, Google Video, Groups, SideWiki, SearchWiki, Analytics, AdWords, AdSense, 3D Warehouse, Book Search, Sites, MyMaps, Base, Code, Moderator, PowerMeter, Feed Burner.

As Google launches new products in the future, Ghosemajumder says they’ll be added to Dashboard, too.

How Google Dashboard works

The dashboard lists all of the Google products that are associated with your account and shows different bits of data related to your use of each product. If you have a Gmail account, the dashboard shows information about your Inbox, Sent mail, Chat history, and more. If you use iGoogle, you’ll see how many gadgets are installed. All of the data in the Dashboard is considered private and only viewable by you, except in cases where you’ve elected to share data with others; a small “friends” icon will appear to indicate that.

More importantly, next to all this data are links to manage it. Gmail users will see links such as “Manage chat history” and “Manage HTTPS settings.” Google Docs users will see a “Manage documents” links. Ghosemajumder says that none of these management links are new. It’s all about organizing the user’s ability to see and control the data that gets shared with Google when using their products. Rather than needing to visit each Google product individually, users can manage everything from this single console.

If you use Google products while not logged in to your Google account, the data associated with those uses won’t show up in Dashboard.

Google Dashboard is currently available in 17 languages, and the company hopes to expand that to 40 languages soon.

Final thoughts

All of the major search engines face privacy issues, but it seems that Google is put under the microscope the most due to its size, success, and perhaps its ambition, too. Google clashes with governments over YouTube, over data retention, and even the idea that Google Maps helps terrorists. It clashes with regular Joes over Street View. It clashes with privacy groups over Google Latitude. The list goes on and on.

In September, Google announced its Data Liberation Front – a team focused on making it easy for users to move data in and out of Google products. The Google Dashboard is a sister and almost a prequel to that effort — one that helps users see and manage the data Google that Google has about them. It represents a step toward appeasing some of its critics and preventing some of these privacy clashes. The question is … will Google’s critics feel that it’s a big enough step?

Note From Danny Sullivan: I’m thrilled to see this step and look forward to seeing how it develops further. My Anonymizing Google’s Server Log Data — How’s It Going? post from last year looks at privacy issues at Google and the difficulty of knowing exactly what they have stored. That was a follow-up to Google Responds To EU: Cutting Raw Log Retention Time; Reconsidering Cookie Expiration, in 2007, which also looked at privacy pain points. In that article, we got our first hint that a dashboard might be coming from Google:

Figuring out where all my data resides and how to kill it is a pain — at Google or Microsoft or Yahoo, for that matter. John Battelle had a good suggestion back in early 2006 for a sort of private data control panel that could show you exactly what was stored where and put the user in control:

“I bet 95% of the public will never edit, or even view the data more than once. But the sense that the control panel is there, just in case, will be invaluable to establishing trust.”

We could use that more than ever. Google especially could use that, if it wants to stop the privacy attacks or at least stem them. How about it? I asked Google’s global privacy counsel Peter Fleischer about this yesterday, when talking to him about the Privacy International survey.

“We’re thinking hard internally along the digital dashboard-type of approach. Is there a way to give users a dashboard and visibility to all these elements and give them control,” he said. “It would be hugely complicated to build, but in terms of that vision, I completely share it, and we’re having deep discussions about it.”

So kudos also to John Battelle. His idea of a control panel dashboard, rather than a million settings and cryptic privacy policies, becomes a reality.

Oh, the irony: The People’s Daily, the main newspaper of China’s Communist Party, says Google censored its web site by adding a malware warning to search result listings for the newspaper’s book section.

The paper says the malware warning appeared after it ran an article about a Chinese group that has suggested Google’s book search settlement might violate the rights of Chinese authors. The paper says the malware warning appeared for three days, and claimed that its book section was “maliciously blocked by Google.”

Google called the claims “absolutely incorrect” and pointed out that its malware warnings are generated automatically and without human involvement.

It’s all ironic, of course, due to the long-running battle between Google and China over censorship issues. Usually, though, it’s Google (and others) accusing the Chinese government of censorship, not the other way around. Or, as the Inquirer calls it, this is a case of “wok calls kettle black.”

Is Google a newspaper killer? Not by a long shot, says Google CEO Eric Schmidt. Nor does he want it to be. In a long interview about his company’s relationship with newspapers and the print journalism industry, Schmidt made it clear he wants established players to survive. In fact, he thinks Google has a “moral responsibility” to help. But help doesn’t mean a handout.

I spoke with Schmidt on the topic about two weeks ago in his office at Google. In summary, he felt that Google takes most of the blame for the internet as a whole, in how it has changed news reading habits that have impacted the newspaper industry. But despite that impact, he felt newspapers would survive in some form.

Schmidt would like Google to help by experimenting with new ways of reading news that might help print institutions make it through the transition they face. That’s especially so in that Google has no plans to produce news content itself. Google’s success, he says, is tied to pointing its visitors to sources of quality content.

Moreover, Schmidt said Google has a responsibility to help, given that part of his company’s vision is to make the world a better place. Without journalistic institutions to do professional investigative articles and other “deep” reporting, democracy would be harmed.

That argument is one many beleaguered newspaper executives themselves have made. If hearing that Schmidt agrees with them is a relief, there’s more goodness flowing their way. Schmidt largely believes that only existing mainstream news institutions have the resources and established trust to do deep journalism. He acknowledges that new online publications have emerged, and that there are journalists working independently of large companies. But his faith is still with the old school, so to speak.

As for the ongoing discussions with the Associated Press, he expects a new deal will be reached. More on that, and the other topics I’ve summarized, below.

Google’s Not A Newspaper Vampire

This year, Google has been blamed by some in the mainstream journalism industry for everything from being a vampire that’s sucked the life out of newspapers to undermining democracy by somehow short-changing publications of ad revenue. How does Schmidt view these accusations? He sees them as Google taking the brunt of disruption caused by the internet itself:

I think in this case Google is a proxy for the internet as a whole. So the people would make the same statements about the Internet as they do about Google. Substitute the internet for Google and you get that idea. And because we play such a central role in information, we’ve become somewhat used to being blamed for everything. In some cases people don’t understand that we’re a conduit to other people doing things. They think Google did it when in fact somebody else did it and made it available.

Rereading Schmidt’s answer when writing up this interview, I was struck how it brought to mind something he started talking about back in 2006, his “don’t bet against the internet” line. That’s the idea that the internet was transforming the world and that only foolish businesses would effectively think they could stick with “old” ways.

Newspapers Will Decline But Won’t Die

So it’s the internet that’s killing papers? Schmidt immediately stopped me from suggesting that he’s saying newspapers will die. He thinks they will survive in some form:

Killing newspapers, that’s your words, not mine…

The number of readers for newspapers is declining. The market is becoming more specialized. There will always be a market for people who read the newspaper on a train going into New York City. There will always be a market for people who sit in in the afternoon in a cafe in the city and read the newspaper in the sunshine. The term “killing” is a bit over[blown]. Newspapers face a long-term secular decline because of the shift in user habits due to the Internet.

So again, if you take the criticism as a statement about the Internet, how will Google fix that? I think that’s just politically a better answer from our perspective. Let me put it this way: Imagine if Google didn’t exist. Would the same criticism still exist? You betcha. See my point?

Online Solutions To Newspaper Woes & Google Wants To Help

As for newspapers specifically, Schmidt feels they have three major problems: physical production costs, loss of classified revenue and loss of print ad revenue. Google’s role is to help with online fixes for these, Schmidt said:

In the case of the newspapers, they have multiple problems which are hard to solve. If you think about it there are three fundamental problems. One is that the physical cost of things is going up, physical newsprint. Another one has been the loss of classifieds. And a third one has been essentially the difficulty in selling traditional print ads. So, all of them have online solutions. And we’ve come to the conclusion that the right thing to do is to help them with the online.

One Solution: New Ways To Read News Online

In terms of the physical production issue, Google’s contribution seems to be experimenting with new ways of reading journalism online. Said Schmidt:

We think that over a long enough period of time, most people will have personalized news-reading experiences on mobile-type devices that will largely replace their traditional reading of newspapers. Over a decade or something. And that that kind of news consumption will be very personal, very targeted. It will remember what you know. It will suggest things that you might want to know. It will have advertising. Right? And it will be as convenient and fun as reading a traditional newspaper or magazine.

So one way one to think about it is that the newspaper or magazine industry do a great job of the convenience of scanning and looking and understanding. And we have to get the web to that point, or whatever the web becomes. So we just announced, the official name is Google Fast Flip. And that’s an example of the kind of thing we’re doing. And we have a lot more coming.

Google Fast Flip is out there now for anyone to use. As for the intriguing idea of a personalized news reader, Google’s Marissa Mayer hinted at experiments with this in August (see Of Living URLs, Newspaper Rankings & California Fires). Schmidt also talked again about the concept yesterday. Stay tuned.

New Ads For News Will Come

What about those lost revenues? Schmidt didn’t address the classified revenue loss, perhaps because Craigslist is the poster child for blame there. As for print display ad decline, Schmidt suggested new ads will follow through into the new reading models:

On the business side, which is what people are really talking about, it seems to me that we should be able to get very powerful advertising in display formats that people will like in this new model, invented, built and sold. Now I don’t know how much revenue that is, but it’s a lot more than they’re getting now.

Speaking of revenue sharing, some noted that Google’s Fast Flip seemed to mark the first time Google has shared revenue with news sites. When I asked Schmidt about this, he disagreed, noting that Google has ad deals with a variety of newspapers where revenue is shared.

However, those deals are for ads delivered on the news sites themselves. Publications like USA Today or the Washington Post carry Google search boxes and share in revenues generated by search ads. Other sites also carry display ads through AdSense. How about sharing revenue with news sites for content hosted on Google itself, as Fast Flip does. Isn’t that new?

Google’s Not A Content Company

Yes, that’s “probably true,” Schmidt said, though he stressed the goal is not for Google to be a content company but rather to help those with content thrive:

We need these content partners to survive. We need their content. We are not in the content business. So, you could decide that we’re just evil businessmen trying to give money to the newspapers [through the Fast Flip revenue sharing], or you could decide that we’re altruistic and trying to save an important Fourth Estate of American political discourse. Whichever one leads to the same outcome. I hope you believe the second. But even if you believe the first, it’s still good business. We need their content.

It should be noted that Google has worked to help newspapers with offline newspaper ad sales, but after trying for two years, it shuttered its program this past January. Meanwhile, Google competitor Yahoo continues with its own two-year-old Yahoo Newspaper Consortium that allows nearly 1,000 papers to sell online ads at their own sites and through Yahoo. The consortium has gotten a lot of positive reviews through it is far from a short-term solution, as even Yahoo admits.

Google Has A “Moral Responsibility” To Help The Press

Moving on, I asked Schmidt if Google felt any obligation to help the newspaper industry. Definitely, he agreed, saying:

Google sees itself as trying to make the world a better place. And our values are that more information is positive – transparency. And the historic role of the press was to provide transparency, from Watergate on and so forth. So we really do have a moral responsibility to help solve this problem.

If You Teach A Newspaper To Fish, They Don’t Need A Short Term Bailout

Sort of like the adage about teaching someone to fish, rather than giving them a fish, Schmidt sees Google’s responsibility as helping the press get into a healthier position in the long-term, not by providing subsidies that don’t solve their current problems:

The next question that the journalists who inevitably ask these questions say is, OK then why don’t you just write us a large check? Let me just posit that that’s a question that people might ask, because I know I’ve had it before. And the problem is that just transferring money from an area where we’re making a lot of money to an area where we’re making little money does not solve the problem for the long term. You’re fundamentally better off building the new product that is profitable and growing – again with the news, with magazines and so forth. It’s better for everyone. Because ultimately a subsidy model is a temporary solution. It’s not a long-term solution.

Google Wants “Well Funded” & “Professional” Investigative Journalism

So far during the interview, I’d largely used newspapers as being synonymous with journalism. But they’re not the same. Journalists don’t all work for newspapers; some publish through blogs. So I wondered, when Schmidt talked about feeling an obligation to support the press, did he mean large press organizations?

I specifically am talking about investigative journalism when I talk about this. There’s no lack of bloggers and people who publish their opinions and faux editorial writers and people with an opinion. And I think that one of the great things about the internet is that we can hear them. We can also choose to ignore them. So it’s not correct to say that the internet is decreasing conversation. The internet is clearly increasing conversation at an incredibly rapid pace. The cacophony of voices is overwhelming as you know.

Well-funded, targeted professionally managed investigative journalism is a necessary precondition in my view to a functioning democracy. And so that’s what we worry about. And as you know, that was always subsidized in the newspaper model by the other things that they did. You know, the story about the scandal in Iraq or Afghanistan was difficult to advertise against. But there was enough revenue that it allowed the newspaper to fulfill its mission.

Few Bloggers Can Do What The New York Times Can Do

But what about people who go out and do professional journalism on their own, who don’t turn around and complain they’re unable to succeed because Google’s hurting them? Said Schmidt:

Let’s talk about Afghanistan. How many free bloggers are there that are in a safe-house in Afghanistan with the necessary support structure to do the kind of deep investigative reporting on what’s really going on in the war? I’m not talking about the ones that are embedded in the government. That’s an example. The kind of articles about the scandals in the various government bureaucracies. All of those kinds of things. There are very few bloggers, to use the term broadly, who have the time and the resources – I mean these are stories that take months to develop, they take confidential sources.

Another example that people in our world often miss: Let’s assume you’re a mid-level government executive, not necessarily in the United States, and it’s a crime to leak information for purposes of discussion. Are you willing to leak to a blogger who has no track record of protecting his or her own sources, versus the New York Times, which routinely sends its people to jail over this question of a shield law.

So again, it’s facile in my view to say that the two functions are similar. There’s no question that a large part of the function of newspapers and magazines is broad communication that’s not particularly controversial, and helpful and it’s great. But whatever percentage that is that requires the protection of sources, deep investigative journalism, is very important in a democracy. You would be crazy to not understand the history of that.

People don’t like it, by the way, because it’s very controversial. The Pentagon Papers is a classic example. It was incredibly controversial: Was Daniel Ellsberg a patriot or was he a criminal? He was actually adjudicated and was not a criminal because the government was doing something inappropriate. People disagree over these things. But the point is that that’s the kind of stuff I’m talking about.

Hearing Schmidt talk of this, I could only think that some newspaper executives who have attacked Google ought to be lining him up as a chief spokesperson for their industry.

It was also somewhat amazing to hear. Could I imagine someone leaking information to a blogger? Of course, I thought — to me! I was blogger (according to some) sitting right across from him, yet someone who has routinely honored embargoes and confidential information I’ve received from his own company.

To be fair, Schmidt did talk about bloggers with “no track record” (I think I’ve got one) versus the New York Times as an institution that has a well known track record.

But still, when I started as an independent journalist over a decade ago, I had nothing behind me (I’d have been called a blogger, but we didn’t have blogs back then). My site built its own audience because the traditional press was not covering search engines as well as or in as much depth as my publication was. It thrived because of the internet.

I countered. Aren’t there journalists out there who are independent of mainstream publications but who have good track records and relationships? Not for the deep journalism that Schmidt is worried about:

Not at the level I’m talking about. Name a blogger who today has the kind of deep embedded reporting that a traditional newspaper does for this kind of, for scandals. It just doesn’t exist yet. They may develop. It’s perfectly possible that they will develop. It’s a different kind of reporting. The online world is so immediate, it’s so competitive, you know people are like having heart attacks just keeping up with the publication demands in the online world. So there are some attempts at this. For example, ProPublica, which is funded by the folks [the Sandler Foundation] in Berkeley, San Francisco actually, is an attempt to replicate what I’m describing in a nonprofit way. So there is an example. It’s run by journalists, run by professionals.

Name a blogger doing deep investigative reporting? Schmidt’s got me there. I can’t name them off the top of my head. It’s not an area I focus on. I do suspect some are out there, though — if you know of some, drop them in the comments below.

Personally, I feel the big challenge to large, investigative reporting isn’t figuring out how to fund it or how to develope the trust factor needed. It’s dealing with the aftermath, when some large corporation or government body decides to sue you. That’s the chilling effect to me, for independents, especially when there’s still little clarity about how protected they are by various shield laws for journalists.

City Hall & Local Coverage At Risk

Assuming the mainstream journalism outlets did go away, would we lose investigation? Or would something spring up? Schmidt’s response that something might replace coverage on big issues but “city hall” or local deep reporting is at risk:

It’s a speculation. As I said, ProPublica is a good example. There’s a couple of groups that are funded out of political groups. There’s one that’s under Center for American Progress [Media Matters] …. Their basic job is to keep what they claim is the Republican spend machine honest. So that’s sort of an example of this. But it’s not quite. Again, think Iraq, Afghanistan, Defense Department errors, you know, corruption in governments, local governments. It’s fair to say that, though, I think the biggest worry is actually for local reporting.

Media Matters is an example. I think most people believe that in, hopefully, the unlikely scenario of the loss of all of these voices, most people believe that there’s enough emphasis and interest at the national level. But what happened to the guy who’s investigating the misdeeds of the CFO in the mayor’s office? And again, I’m talking about the stuff you can’t do in an hour. The gumshoe kind, walking around talking to people. There are very few of those people.

The loss of local coverage certainly resonated with me, since my roots in journalism started there. Last year, I did a piece talking about how over the years, the Los Angeles Times greatly reduced its local reporting from the heyday of when I worked there. So there are very few of these people? My response was that I know lots of them — they’ve all been laid off. That prompted Schmidt to say:

But they’re not doing it anymore. Or if they do it, they’re doing it on their own time.

It turns out there’s not enough money there — even with the improvement in overhead costs, because you don’t have a lot less overhead. There’s not enough money yet. Although for the most popular blogs you know, it’s the 1% phenomenon, the head of the tail, they do make money. But the vast majority of blogs end up being, it’s a little bit like wine-making. It’s a lifestyle as opposed to a real profitable business.

Last week, New York University professor Clay Shirky also had much to say about the issues of funding journalism, and the impact it might have on regional reporting. His comments are well worth reading for more on this topic. Shirky also has an interesting dissection of a local paper, looking at how few on a large payroll are actually involved in the reporting.

Schmidt: Institutional Brands Over Individual Journalists

Next the interview moved on to Schmidt’s statements about the internet being a “sewer” that brands such as major newspapers can help sort out. Is it just newspapers that have the important brands that people recognize as trusted sources, when it comes to journalism?

There are two different views. There are two different views even within Google. So one view goes like this: The institution becomes less important but the writer remains as important. So that’s sort of the new view.

I don’t happen to agree with this, but I want to make sure I report it accurately. And the rough argument goes like this: Newspapers existed because you needed an aggregation point of great talent. But you really go to a newspaper to read the writers. And because they have so many other outlets, they will become more like freelancers in this model. They will be paid by institutions and they’ll make enough money to get through the day and people will follow them. And some writers will become so famous that they’ll be like basketball stars – they’ll have large salaries and speaking [and] book deals and things like that, although the majority won’t get there.

I disagree with that view, because I believe that there is a value to the brand of the aggregator as well as this trust issue that I was discussing earlier that ultimately a freelance reporter, that ultimately it would be difficult for freelance reporters, as much as we favor them, to operate without at least some institutions of trust. And trust in two ways: trust to the reader, and trust to the sources.

I found his response fascinating, especially the discussion of a split within Google itself. All too often, there’s an assumption that Google has a monolithic view of everything. When it comes to newspapers, I think many newspaper business executives assume Google’s goal is to destroy their brands, to favor the blogs and aggregators, to be a newspaper-killing aggregator itself.

Instead, Schmidt’s not endorsing some massive revolution that will sweep mainstream publications away, with an air of good riddance. He seems to view the institutions that we now have as essential.

A Rise Of New Brands? Some…

Does this mean the institutional journalism brands we have now are locked in stone? Are there new brands that have arisen, new online ones?

Well, the most obvious one is Politico. So there is an example. I think it’s reasonable to say that there will be, in every category of information, there will be a couple of new brands that are Internet-only. An example in our world is TechCrunch….

All Things Digital is another one. So those are some of the brands that didn’t exist 10 years ago. And if you think about it, they’re defined by the personalities of their founders.

I asked if we should mourn some of the mainstream brands that will inevitably disappear.

Well I’ll tell you a story. I’ve been in this industry for 30 years, and during this time there has always been headline conferences that were very exclusive. And when I was a young executive I assumed that they would live forever. So the Agenda Conference was an example. For me, that was the most important professional event of the whole year. I would make sure that if I was invited I would go. I really enjoyed it. It was very, very important. When was the last Agenda conference? A long time ago.

So, do I mourn that? Yeah, I had a really good time. But society moves forward. New brands emerge. How old is the Starbucks brand? What would we do without Starbucks today? So the point about brands is that while it’s true that brands do end, new brands emerge. So it’s possible that the sum of the brands we were just talking about could ultimately… I’m not suggesting it can’t happen, I’m suggesting it’s very hard.

So, San Jose news. What is the brand that I will go to for news about San Jose? Well, I’ve got the San Jose Mercury News. Let’s assume for the purposes of argument that that’s in decline, which I think is without question. What’s the new brand that I’ll go to? I actually don’t know.

Google & The AP

Next I asked about Google’s current negotiations with the Associated Press. The AP ratcheted up suggestions earlier this year that it wasn’t getting a fair deal from Google from its current agreement, which was cut in 2006. The AP has also suggested that Google should be rewarding “recognizable news brands” more in its regular web search results. What’s the beef? Did the AP not get a good enough deal in the first place?

I would rather not discuss a business negotiation. But you’re smart enough to understand that this is a business negotiation. I am sure we will come to a good deal for all parties. How’s that? I was rather humored by the public criticisms because – there was all this criticism – we have a deal with the Associated Press that’s in place today. So, and surely they’re aware of this.

Indeed, I expect a deal will be struck. But my worry is that “must-carry” publications like the AP will get attended to at the expense of online publications that, as even Schmidt says, struggle to build their own revenues. And given how we’ve had suggestions that the health of democracy is at stake, if mainstream publications can’t get deals with Google, shouldn’t the AP terms be public. So that everyone knows what’s being given?

The fact of the matter is, the problems that are occurring in the industry are intrinsic. They need to be addressed. We’re doing what we can think of and we’ve been upfront about working on those. This is ultimately about money and the difficulty people are having of bringing in revenue. Again, I understand that.

So, in the private discussions with the AP, if the AP wants to do everything public then I’m sure we would consider that. But usually business negotiations are done in private for precisely the reason that people think it’s competitive.

OK, AP, so how about it? I sent the AP what Schmidt said and asked if it would be willing to publish the terms of any deal with Google. No luck. I was told:

As a longstanding corporate policy, The Associated Press has refrained from discussing the terms of its business dealings.

I was also given a quote from Sue Cross, the AP’s Senior Vice President, Global New Media & US/Americas Media Markets:

Commercial agreements are crucial to helping the AP offset the costs of its global newsgathering operation and keep member assessments lower. They allow AP to continue providing vital breaking news, including coverage of this week’s deadly earthquakes and tsunami, and to continue reporting from critical war zones, such as Iraq and Afghanistan.

Back to my interview with Schmidt, I asked him how Google may deal with a situation where if the AP gets a new deal, others may feel left out. He said:

Well, the Associated Press is different from other publications, remember, because the Associated Press is really, they really are an aggregator at some basic level. Again, I don’t want to parse the specifics. But the fact that there’s a deal with AP does not mean that you have the same deal with the New York Times. And in fact we do not.

Finally, I was curious if Schmidt actually read a newspaper regularly. Yes, he does. Two, in fact. But the exact two are the only part of the interview he asked remain off the record. And I have a pretty good track record of dealing with that type of material :)

Also from the interview, on other subjects:

• Google’s Schmidt To Book Settlement Critics: What’s Your Solution?
• Google’s Schmidt: Independent Yahoo Still Important To Competition
• Eric Schmidt’s Favorite Google Product? Chrome!

Judge Denny Chin of the US District Court, who is presiding over the Google Book Search Settlement and was prepared to approve it until legions of opponents came forward, summarizes the dilemma facing all interested parties in his recent order (via the NY Times), taking the impending October 7 final settlement hearing off the calendar:

The current settlement agreement raises significant issues, as demonstrated not only by the number of objections, but also by the fact that the objectors include countries, states, nonprofit organizations, and prominent authors and law professors. Clearly, fair concerns have been raised . . .

[Yet] the settlement would offer many benefits to society, as recognized by supporters of the settlement as well as D.O.J.

Even as it was arguing that the settlement — as negotiated — should be disapproved by the court, the US government argued in favor of the concept:

The United States strongly supports a vibrant marketplace for the electronic distribution of copyrighted works, including in-print, out-of-print, and so-called “orphan” works. The Proposed Settlement has the potential to breathe life into millions of works that are now effectively off limits to the public. By allowing users to search the text of millions of books at no cost, the Proposed Settlement would open the door to new research opportunities. Users with print disabilities would also benefit from the accessibility elements of the Proposed Settlement, and, if the Proposed Settlement were approved, full text access to tens of millions of books would be provided through institutional subscriptions. Finally, the creation of an independent, transparently-operated Book Rights Registry (the “Registry”) that would serve to clarify the copyright status and copyright ownership of out-of-print works would be a welcome development.

So what now?

One optimistic scenario sees all the parties quickly modifying the existing agreement and satisfying the various objections raised by the opponents and US DOJ. But that may be too simple and optimistic. As a practical matter, with many more parties watching and wanting input, it will probably more challenging to reach an agreement. Yet some sort of agreement must be reached.

The settlement ended a class action lawsuit that would presumably be revived if there were no settlement reached. The case has been going on for several years and everyone, including the judge, wants to see it over and done with. Something must happen, the parties are compelled to settle or continue to litigate. But it’s not certain how everyone will get to an agreement that literally satisfies the world.

It’s not clear what will happen. What we’ve got here is what you might call a real cliffhanger.

Ambivalent is word that best describes newspaper publishers’ relationship with Google and Google News in particular. Some publishers see Google as a partner or potential partner and some consider Google to be the devil incarnate, responsible for virtually all their woes.

For its part Google sees itself as a supporter of newspapers, helping them by driving traffic to their sites and potentially developing a payments platform that can better help them monetize their content in the coming return to pay walls. Google CEO Eric Schmidt has spoken out publicly many times in support of the role and importance of newspapers and Google’s desire to help the industry.

PaidContent interviews Google News Business Product Manager Josh Cohen about a range of issues concerning the site itself and Google’s relationship with publishers. There are no real zingers or revelations but there is honest discussion from Cohen about some of Google News’ shortcomings and there are some interesting other bits. Here are a few edited excerpts:

PC: How are the search ads that were introduced earlier this year to Google News working out?

Cohen: I’d say it’s comparable in terms of the effectiveness when we are showing results to it for certain categories.

PC: With some other aggregators, like Digg, there seems to be so much more user engagement. You don’t really see that on Google News.

Cohen: There’s a lot more we can do on the social nature of news. I think Digg does a great job of that. There’s such a social nature to how information overall is consumed now. We have some things were working on in that area.

PC: What do these discussions with [newspaper] publishers amount to?

Cohen: Some of those discussions are purely around the user experience: What does it look like on Google, how can we work together to improve the experience of our users who come to Google News to look for content and also think about what are the ways we can work together to help improve the product side of it. Much of the discussion today is about the business model.

Cohen also says in response to a question about why Google News looks the same as it did a couple of years ago that there might be a “backlash” from publishers (PaidContent’s phrase) if the site were improved and enhanced. But as Google News improves usability it would theoretically become a more effective partner and potential revenue generator (as a payment platform) for publishers.

That paradox captures the “damned if you do, damned if you don’t” position Google News is in with the industry. Why aren’t newspapers similarly upset with other online news sites and aggregators? Google isn’t the leading news site/aggregator online:

Google is the just most visible symbol of how the Internet has all-but-capsized the newspaper business. However, as the Trulia blog shows, the decline in classified advertising, a major source of newspaper ad revenue, has been happening for more than a decade and longer than Google has been in existence.

Despite their frosty relationship, Google is proposing a micropayment system that could give the newspaper industry a way to charge for its online content. According to the Nieman Journalism Lab, the micropayment system will be based on Google Checkout and be available within a year “to both Google and non-Google properties.”

The NJL has posted an 8-page PDF that Google wrote in reply to an RFP from the Newspaper Association of America. “Google is uniquely positioned to help publishers create a scalable e-commerce system via our Checkout product,” the document says, “and also enable users to find this content via search — even if it’s behind a paywall.”

Micropayments are just part of Google’s overall plan to help newspapers make money from online content. Google’s proposal — its “vision of a premium content ecosystem” — include five key features that combine the Google’s e-commerce, search, and advertising platforms:

• Single sign-on capability for users to access content and manage subscriptions
• Ability for publishers to combine subscriptions from different titles together for one price
• Ability for publishers to create multiple payment options and easily include/exclude content behind a paywall
• Multiple tiers of access to search including 1) snippets only with “subscription” label, 2) access to preview pages and 3) “first click free” access
• Advertising systems that offer highly relevant ads for users, such as interest-based advertising

While Google’s proposal includes a micropayment system that’s already in development, the company says it does “not believe” micropayments will catch on widely for buying online content. Instead, Google says, “we envision the typical scenario to be where a user pays a monthly fee for access to a wide-ranging package of premium content.”

Google’s relationship with the newspaper industry has been under the microscope this year, but the war of words goes back at least a couple years. Here are links to some of the coverage here on Search Engine Land and on Danny Sullivan’s personal blog:

• Newspaper Editor: Google Is ‘Hugely Dangerous’
• Newspapers, Google And The “Devaluation” Of Content
• Google’s Love For Newspapers & How Little They Appreciate It
• AP Becomes “Bad Cop” To “Protect” Newspaper Content Against “Misappropriation”
• Amid Tensions Google’s Eric Schmidt Addresses Newspaper Conference
• Sorry, Tom Curley: Don’t Expect A Google Ranking Boost For The AP
• Google Testifies Before Senate About State Of Journalism
• Sergey Brin On Newspapers, Breaking “Page’s Law” & Bing As Name Of Microsoft’s New Search Engine
• Google To Newspapers: Robots.Txt You

Google reportedly made a number of concessions to European officials concerned about the European implications of the Google Books Search settlement deal. The company is fighting on both sides of the Atlantic to gain regulatory and judicial acceptance of the deal previously announced, which has been criticized by a number of self-interested parties and legal observers who contend that Google will have a near monopoly of so-called “orphan” books if the deal goes through. Orphan books are those that are still covered by copyright but rights ownership is uncertain or authors cannot be located.

According to a Bloomberg story about the EU concessions:

1. Google will let two representatives from outside the U.S. join a board administering its digital books settlement
2. Books that are commercially available and under copyright in Europe won’t be considered out of print under a proposed settlement with U.S. publishers

At an EU hearing today in Brussels, both Google and its critics were presenting their respective cases before European Commissioners. Google blogged about the hearing and the benefits to readers of digitizing the European collection. Meanwhile opponents, including Microsoft, Amazon and Yahoo (part of the so-called Open Book Alliance), sought to play up the negatives of allowing Google to proceed.

“We are fighting this cartel that is being proposed by the parties to the U.S. settlement and Google,” Peter Brantley of the Open Book Alliance told reporters ahead of the EU hearing.

(Previously, Microsoft abandoned its own book scanning project.)

Two EU Commissioners, Viviane Reding and Charlie McCreevy, issued a joint statement generally in favor of book digitization and a US-style settlement that would be pan-European.

Germany and France have been two of the biggest critics of the US deal, however. In fact France is going to file objections to the book search deal with the US court in New York overseeing the settlement agreement. A hearing to approve the US settlement is scheduled for October 7. According to statements made to Reuters French Culture Minister Nicolas Georges expressed broad concern about Google’s control over “orphaned” material:

France is concerned about European authors’ rights, Georges said.

“There are lots of European works in Google’s database. Google can digitalise these works without the permission of European authors,” he said.

He cited worries over the copyrights of orphan works, which are books or other materials that are still covered by U.S. copyright law, but it is not clear who owns the rights to them.

“Google will have a monopoly digitalising European orphan works without permission,” Georges said.

Germany previously filed similar objections with the US court.

We can probably expect more concessions from Google in the US and abroad before the settlement deal is formally approved (and there can be a similar arrangement in Europe). I would be very surprised however if the court or regulators entirely blocked the deal. It’s pretty clear that lots of stakeholders want book scanning to proceed and that governments lack the will and resources to do it themselves.