In making the changes to this prime real estate on a search results page, Google is lessening somewhat the visibility of Google+. But it’s still using the space in the upper right of its search results to keep searchers on Google — something well within its rights, but something that may attract criticism in the same way that the Google+ box did.

Let’s look at what’s going on now….

Background: People and Pages On Google+

When Google launched “Search Plus Your World” in January, it made content from Google+ much more visible in the search results.

Part of the Google+ push was a content box called “People and Pages on Google+” that would appear adjacent to the top organic results. It didn’t matter if you were logged in to a Google account, or used Google+ — there was the “People and Pages on Google+” content box in the top right of the search results page. It showed primarily for generic search terms like “music,” “cars” or “Facebook.”

Search results like that — especially the one for “Facebook,” which has been the most popular search term in the U.S. for the past three years — brought on a lot of criticism because Google was promoting Google+ ahead of what were often more relevant social profiles. In the third example, Google was leading searchers to Mark Zuckerberg’s inactive Google+ page rather than to his Facebook profile. Danny Sullivan covered that and several other relevance issues in his article, Real-Life Examples Of How Google’s “Search Plus” Pushes Google+ Over Relevancy.

Now: People and Pages on Google+ Is Gone

Now that the dust is settling on last week’s Google Knowledge Graph launch, we’re no longer able to see the “People and Pages on Google+” content box on any searches. It’s gone, replaced instead by different Knowledge Graph-related content boxes.

That search (from above) for “music” now shows a Knowledge Graph box for “People related to music.”

A search for “cars,” which used to include links to the Google+ pages of brands like Toyota, Nissan, Ferrari and others, now shows a Knowledge Graph box that invites searchers to “See results about” the Pixar movie Cars.

Other searches that previously showed the “People and Pages on Google+” content box, like “Facebook” and “seo,” don’t show anything from the Knowledge Graph. Search Engine Land’s editors did a number of searches yesterday and none displayed the old Google+ “People and Pages” content box.

Google: We’re Blending Content

A Google spokesperson explained that Google is blending different content sources, including Knowledge Graph connections and Google+ profiles, to return the most relevant content on the search results page.

And it’s true that searchers are seeing content from different sources; a search for Google CEO Larry Page, for example, shows this in action. The photo is from (and links to) his Google+ profile, and further down the Knowledge Graph box is a clipped version of his most recent post there.

There’s also text from (and a link to) his Wikipedia page, along with several links related to the Knowledge Graph data. And, even though those links don’t promote Google+ like the old “People and Pages on Google+” did, they do keep people on Google and may lead to some of the same criticisms that Google faced before.

Possible Knowledge Graph Criticisms

To be clear: It’s Google’s search engine and I’m a big believer that they can link to their own properties if they want. But, as with the Search Plus Your World examples from January, if they do so in a way that’s not relevant and/or not user-friendly, they leave themselves open to vocal critics. (And if they do it in a way that appears anti-competitive, those critics may include the U.S. government.)

A search for “seattle mariners” offers an example of what I’m talking about:

In this case, like many others, the Knowledge Box is showing a mix of content — text from Wikipedia, latest post from Google+, and other information. The possible issues are:

• the Mariners’ logo and the light text below it send searchers to the Mariners Google+ page, not to the Mariners’ official site, which would seem to be the most authoritative, relevant and user-friendly link for that spot in the Knowledge Graph box (and the fact that the team logo says “Mariners.com” is likely to mean users will think a click there leads to the official site)
• the links on “Eric Wedge,” “Safeco Field” and the others are links to conduct more Google searches; will users expect to get links to search results there? or will they expect to be linked to the official Safeco Field page? or, since Wedge doesn’t have an official site, will they expect to be linked to his Wikipedia page or his bio on the Mariners website?

I don’t know the answers to those questions. But I do know that, when a Knowledge Graph box about the Seattle Mariners doesn’t contain a single link to Mariners.com, and has several internal Google links instead, critics have an opportunity to continue accusing Google of promoting itself at the expense of relevancy.

The Knowledge Graph box has its pros and cons. The data is, in many cases (but not all) excellent — it’s very cool to be able to scroll through the Mariners’ roster right from inside the Knowledge Graph box, for example. But the fact that all of the links keep searchers on Google, either sending them to Google+ or creating another search, probably won’t quiet the criticism about Google promoting itself too heavily in search results.

Bottom line: The “People and Pages on Google+” content box brought Google a fair amount of flak, but what’s replacing it isn’t necessarily going to change that.

Issues like the Stop Online Piracy Act, or SOPA  which received so much attention earlier this year, privacy issues, security issues, patent laws and a myriad of other items keep the Google, Facebook and Microsoft DC teams extremely active on Capitol Hill.

In my home state of Texas, over-eager tax collectors are auditing SEO firms for back sales taxes. Nevermind that for years the law has been interpreted to mean that service firms are exempt from sales tax. Oh yea, and thanks to the Texas Comptroller, Amazon will pay a fine and start collecting sales tax in Texas beginning July 1. I’m going to rent a UPS box in Oklahoma, I think.

But can you guess who isn’t paying attention? Search Engine Marketers. Look at the headlines in any of the leading SEM trade magazines. Go to the forums. Read the Twitter streams of prominent search marketers. Most SEM news is about algorithm chasing or new things that Google or Facebook has done to piss us off. We’ve also got tips and tricks and coverage from conferences.

A random announcement by Matt Cutts about his cats will receive more pixel ink in our trades than any legislation ever has. We don’t really want to talk about the laws that are being created to govern our industry unless they blatantly threaten us, or Google and the other technology companies tell us we should care (and that’s exactly what happened with SOPA).

And you know what? I don’t blame us.

Let’s face it, unless there is drama unfolding or we can a see a direct impact on our lives, most of us find the political process slow and mind-numbingly boring. I tell my clients it’s a full-time job just keeping up with daily changes in the search and social media landscape. And it really is. Watching the political landscape for threats to our industry is also a full-time job – a very necessary job. But the question is, whose job is it?

Who is looking out for the interests of search and social marketers on Capitol Hill? As far as I can see, no one yet.

Increasingly, we need timely insight into what is happening in Washington D.C. and with governments around the world. New legislation, like the Stop Online Piracy Act (SOPA) and now the new Cyber Intelligence and Sharing Act (CISPA), can threaten our very livelihood.

We have no representation in Washington D.C.

We must rely upon other organization, like Google and Facebook, to make our case on Capitol Hill, and hope their interests align with ours. In many cases, they simply don’t.

In fact, most of us aren’t even aware of the issues that we need to be watching. But as I mentioned earlier, you can bet that the search and social players are spending significant lobbying money. I highly recommend reading this Politico article to understand what’s happening on “The Hill.”

Governments aren’t the only entities that we need to watch.

Recently, Mozilla announced that the Firefox browser would utilize a secure method of searching Google by default. For search marketers, that means our jobs just got a little more difficult. We’ve already seen our analytics regress toward 1999 with the number of Google referrals from “Not Provided” skyrocketing faster than prices at the gas pump.

Then there’s the ubiquitous Penguin update, starting with a cryptic announcement by Matt Cutts at South by Southwest and culminating in an algorithm shift that garnered more mainstream ink than any in decades – even though it purportedly only affected 3-4% of searches (really? like “not provided” on affects 10% of searches?).

Are search engine marketers’ views even considered by these companies when they make decisions? Is the search engine marketing community asked what it thinks about these changes? Do we even completely the understand the rules that we are meant to follow? Hardly.

So my question is, in this day and age, can we all collaborate to find solutions that would benefit users, engines and the SEM community? Yes.

So, why isn’t someone asking our legislators, the search engines and other companies tough questions – and pushing to be part of the conversation?

When we think of lobbying, we think of guys in expensive suits taking politicians out to lunch. But, that’s an antiquated way to view the field. Today, lobbying is about understanding the issues, creating a position on each issue, and communicating your position to those with the decision-making power. That’s not always a government. Sometimes it’s an industry leader.

My conclusion: I don’t have the answers. I have a few suspicions about who should be at the forefront of this conversation – but unfortunately, I think significant action will move somewhat slowly.

Without a hot-button topic like SOPA to fuel the fire, the short-term benefits won’t justify participation by most. But the bottom line is, if we want to maintain our momentum, we need to find common ground and pay for our own representation. I feel like search engine marketers have just earned a seat at the big kid marketing table in the last few years. If we want to keep our seat, we need someone watching our backs.

So where are the search engine marketing lobbyists? I don’t know, but I hope they show up soon.

Image used under license, courtesy of Shutterstock.

Josh Davis drew attention to the DBCC situation in his post today, documenting how after receiving three link requests from the same person, he finally followed up, only to be pitched on placing a link from one of his articles to the DBCC site in return for $30 per month.

What’s Dun & Bradstreet — the nearly 200-year-old Fortune 500 company that brokers information about people and companies for business decisions — doing buying links? It’s not. DBCC was spun-off from D&B in 2010 and is a privately-held company, providing credit solutions to small businesses, with a license to use the D&B brand.

That DBCC isn’t a Fortune 500 company takes some of the “wow” factor out of the story. In addition, we’ve already had stories about big companies ranging from JC Penney to Overstock to even Google itself getting caught for paid links. That makes what DBCC was doing seem even less newsworthy to some. After all, doesn’t everyone do this now? What’s really new or unique here?

I supposed there is nothing particularly new, but clearly there’s a bunch of reminders that are useful to have out there.

Client Beware

Judy Hacket, the chief marketing officer of DBCC, sounded pretty horrified when I talked with her today about the situation. Her department was scrambling to discover how exactly it ended up with these links being purchased.

Davis connects the link request in his story back to iAcquire. Hacket wouldn’t say if DBCC is working with that firm, citing possible confidentiality clauses in contacts. She did say, however, “we have absolutely no agreement with iAcquire or anyone else allowing them to use any grey hat or black hat practices.”

Hacket was also adamant that DBCC had no desire to violate any of Google’s guidelines.

“We would never endorse something like this,” she said.

Of course, we’ve heard this type of denial / shock before. Recall what JC Penney said last year, after the New York Times profiled it for using paid links:

J.C. Penney did not authorize, and we were not involved with or aware of, the posting of the links that you sent to us, as it is against our natural search policies.

Curious to learn more, I asked JC Penney what those policies were after the New York Times story came out. A JCP spokesperson emailed me back:

We are not going to provide our policies, but obviously, they would include staying within Google’s guidelines.

Well, obviously! Except they didn’t, otherwise JC Penney wouldn’t have been banned. When JCP said it didn’t authorize or was involved with paid links, it meant that its SEO firm did all that. As I was told further in my email exchange:

SearchDex ran our SEO program. We do not pay for links as they go against Google’s guidelines. SearchDex was terminated because as our SEO provider they should have known. This was a clear failure on their part.

It was also a clear failure on JCP’s part, for not understanding what its SEO company was doing. The same is true for DBCC. Indeed, I’ve been joking that for some time, whenever some large brand gets dinged for paid links, it’s handy to have an SEO firm they can pin the blame on.

The reality is that for the large companies or brand names, this type of behavior seems to get a 90 day slap, then they’re back in Google’s good graces. It’s difficult for Google to permanently remove an important company that people expect to find. That means as a client, or as an important brand, keep these lessons in mind:

• Do you fully understand how your SEO company will obtain links for you?
• If you don’t want paid links, have you made that crystal-clear?
• If you approve of buying paid links, are you prepared for a potential short-term PR black eye?
• If you approve of buying paid links, is that worth a potnetial short-term Google penalty?

If you’re not a major brand or an essential resource that Google has to list, there’s really only one question you need to ask. Are you prepared to lose all your traffic from Google? That’s because for the non-essential people, being caught for paid links can be a death sentence, not a temporarily set-back.

SEO Company Beware

As for the SEO company buying links, you’d better be prepared for your client to toss you to the wolves, if a paid link campaign comes to light. Also do be prepared for that campaign to come to light, unless you’re incredibly careful with whom you are soliciting.

In this case, the SEO company pitched someone whose “About” page explains that he writes about marketing. That should have been a warning that this person is probably somewhat savvy about paid links, so some disguised pitch for one wasn’t wise.

I get these types of pitches myself. So does Matt Cutts, the head of Google’s web spam team. If I’d gotten this type of email sent to me on behalf of a major brand, I might very well have written about it myself and concerns about “outing” be damned. I’d view it as a protective service to the general human population. It’s like watching someone drive backwards on the freeway. They’re a danger to everyone.

As for iAcquire itself, it won’t comment on the case, citing client confidentiality. It won’t acknowledge that it was involved in any way, nor confirm if it has worked with DBCC. But the company did give me this statement:

We work with many of the largest brands in the world. It’s very common that we run into large brands everyday buying links from blog networks and large paid link marketplaces, and our mission as a business is to direct brand strategy towards whiter hat link building approaches.

That movement often takes time and effort – and, in the real world in working with big brands with pre-established objectives it frequently is a multi-step process, and requires a lot of education at both the SEO manager, online marketing marketing manager and CMO-levels. We are literally driving the education process every day.

To be clear, we are not a link network. Every link we build is based on the very same principles touted throughout the industry. Our links are contextual and relevant through outreach performed by 40 actual in-house people that sit in our Arizona office and everything is pushed through strenuous quality assurance.

Our business is to push brands to white hat strategy, but we frequently acquire new customers that are still on that path, and we support these companies toward that white hat direction. We have been investing significantly into our content marketing, social media, and digital PR channels to more rapidly make those changes internally and for these brands.

Regarding the article written about our company, we can’t talk about specific strategy for specific customers or potential customers – due to confidentiality agreements. Financial compensation for links does not represent the strategic direction of our company. iAcquire’s services are holistic and include a great deal of content marketing, digital PR and social media promotion, and on-page SEO consulting.

We’ve never had a problem with Google’s algorithm and our clients – and, we understand that it is important for us to continue to drive the market towards techniques that best represent the guidelines established by search engines. iAcquire continues to evolve its service lines, and recently brought in Mike King to help drive that direction to ensure we are considering search engine guidelines and industry best practices. In addition, he continues to promote these best practices at various conferences worldwide.

Wait, is iAcquire suggesting that DBCC — assuming it eventually emerges as a client — was one of those companies needing to be nudged into the white hat world? Cofounder Joe Griffin effectively said no, when he emailed this follow-up statement:

We aren’t talking about D&BCC (when we mention that we transition people from grey to white in the second sentence) – we can’t talk about specifics of clients or potential clients – we are prevented from doing so.

The enterprise world has a lot of nuances, and we believe we have more than anyone helped to correct SEO brand strategy as it relates to off-page SEO and specifically as it relates to killing black hat link networks.

At the end of the day we run into a lot of different goals, and different approaches, and we try to bring all clients to a fully white hat solution. Our team is heavily focused on high quality editorial content and creative development to attract links. We do a TON of link reclamation as well.

We brought Mike on board specifically to continue to build upon this direction. Mike is one of the best in the business in educating SEOs about how to properly implement off-page SEO strategy – he’s helping us here as well.

We are not a paid link company. We deliver holistic off-page SEO to small and large companies – and are the leading charge in proper off-page SEO education.

By Mike, Griffin is referring to Michael King, who I’d say has built a good reputation for himself in some SEO circles over the past year in his writings and speaking. He’s spoken at our own SMX events and is slated to again next month. He’s sharp, has lots of insight, and he seemed a win for iAcquire when they hired him about two months ago.

Suffice to say, I was pretty surprised that he appeared mixed up with all this. He’s seemed very white hat. I think it’s great if he’s going to help iAcquire and/or its clients move to white hat activities, but I’d say the sooner the better, if iAcquire really doesn’t want to be known as a paid link company.

Right now, however, if that link request is effectively coming out of iAcquire’s work, it might not be a paid link company, but it sure seems as if it has been buying links. That’s tough to square talk of following search engine guidelines.

Google, Oh Google

Meanwhile, there’s Google. This time last year, it was counting the news about JC Penney as a win in the war against paid links. A year later, has anything changed? Was it really that much a deterrent?

I honestly don’t know. I’ve heard some say that many SEOs buy links. That’s it’s just what you have to do. I don’t have any good survey data to back those types of statements up or knock them down.

Fair to say, however, today’s news didn’t surprise many. Even if it it had been D&B itself, I’m not sure if the New York Times or the Wall Street Journal would have cared to run stories, as they did last year with JC Penney and Overstock.

I do know that Google has got to come up with something better than counting links. I keep expecting that social will be a larger signal, and my previous story below explains more about why this makes sense:

• When Everyone Gets The Vote: Social Shares As The New Link Building

In the meantime, we’re stuck with the increasingly creaking, rotting link signal. But for anyone who thinks that’s an excuse for anything goes, look again to what I said the Client Beware section above. Are you really big enough to take a chance on being dropped from Google?

There are plenty of people who bought links who did get caught in the Penguin Update. Some of them are learning to their horror that the only chance of coming back will be to start completely over.

Finally, I did ask Google if it had any comment on the DBBC situation. Nothing specific, just this general warning:

Our guidance on paid links remains the same as ever: paying for links that pass PageRank violates our guidelines, and Google takes appropriate action in response.

If you don’t know what PageRank is, well, read our guide: What Is Google PageRank? A Guide For Searchers & Webmasters.

Publisher Beware; Link Broker For Shame

For those being approached about selling links, this is a reminder that Google really doesn’t like you to do that and has penalized sites for doing so since 2007. If you’re approached out of the blue with a link request, unless you block that link by using something like the nofollow attribute, you’re placing your site at jeopardy.

Don’t expect the link request to alert you to any of these things. The request that went out on behalf of DBCC was a classic example of non-disclosure. It lacks warnings about possible Google penalties. It even required that there be no visible disclosure, which might very well violate US Federal Trade Commission guidelines. From the request:

Link must not be marked as Paid in the visible content or source code (Common designations include: Partner, Links, Paid Links, Ads or Sponsored Links)

The link can’t have any disclosures, we want it to appear natural.

The whole thing reminds me of the type of spammy requests I get all the time. While people in the SEO space may want to debate whether it’s fair or required or commonplace to buy links or not, I don’t see much room for debate that you shouldn’t try to foist a paid link on someone without full disclosure.

As I wrote before, in my Conversation With An Idiot Link Broker article from 2008

There are plenty of people who disagree over the paid link issue, plus whether Google actually penalizes sites that hard for it. That disagreement is no excuse for unethical behavior. And there is unethical behavior in search marketing, and this is a perfect example of it. No risk was disclosed. When asked repeatedly about risk issues, they were denied….

You want to buy links or be a link broker? Then be upfront that this is an activity that Google does not like and that the faint hearted shouldn’t apply. Only after you’ve scared the heck out of them should you start talking about the ways that you’ll try to reduce the risk, if they choose to carry on.

Personally, I’m somewhat amazed, or really, disheartened, over some of the comments Davis is taking over his post. As I said, some dismiss the paid links as old news. Some are angry, viewing his post as some unnecessary “outing” of paid links.

No one seems bothered that some SEO firm was potentially getting a third-party web site into trouble with Google. That’s the most disturbing aspect of all of this. That’s not new, either, but it ought to be stamped out.

Postscript: DBCC has now sent a letter out to its SEO agencies saying in part:

Without our knowledge or approval, certain parties have reached out to other parties to link to our website (the “Unauthorized Links”) for no valid reason….

Please be informed that we are not affiliated with nor do we have any relationship with these companies.

We ask that you remove any Unauthorized Links immediately unless you believe the content is relevant and provides value to your users. Under no circumstances will we authorized payment or pay for any Unauthorized Links.

You can see the full letter below:

DBBC says a copy was also sent to Google, and it’s part of what DBBC is doing to try and rectify the situation. The list of companies it named in the letter are:

• InternetReach.org
• DigitalPros.org
• MediaFinders.net
• iOutReach.org
• LinkBuilder.net
• SolarPros.com
• Conductor

DBCC said it also sent a copy of the letter to the published contact addresses of those listed. The list matches those that Josh Davis listed in his original post on the paid link situation, companies he connects with iAcquire.

iAcquire was not listed in the DBBC letter. However, iAcquire has now been banned from Google since this story came out, probably because Google believes it either works in association with some of the companies named above or that it controls them.

iAcquire Banned From Google After Link Buying Allegations is our story has more about that, plus has a postscript where I explain the connections more.

Related Stories

• Official: Selling Paid Links Can Hurt Your PageRank Or Rankings On Google
• Time For Google To Give Up The Fight Against Paid Links?
• Conversation With An Idiot Link Broker
• New York Times Exposes J.C. Penney Link Scheme That Causes Plummeting Rankings in Google
• 90 Days Later, J.C. Penney Regains Its Google Rankings
• Google’s Action Against Paid Links Continues: Overstock & Forbes Latest Casualties; Conductor Exits Brokering Business
• Google’s Chrome Page No Longer Ranks For “Browser” After Sponsored Post Penalty
• Google Chrome’s Paid Link Penalty Now Lifted
• Home Depot To Correct Misleading Link Request
• Google Launches “Penguin Update” Targeting Webspam In Search Results
• Two Weeks In, Google Talks Penguin Update, Ways To Recover & Negative SEO

Near the end of the talk, someone asked if how much money Google will make is factored into decisions about changes to Google’s (unpaid search) algorithms. Singhal was adamant: “no revenue measurement is included in our evaluation of a rankings change.” Listening to him explain how excites he gets about search improvements and how changes are evaluated, you realize there’s no spin here. He’s absolutely telling the truth. And he would know. Chris Sherman asked if anyone at Google really understands how the whole thing works and he replied that while no one knows how everything works (all of unpaid search, AdWords, Android, etc.), he has a pretty good idea of how all of unpaid search works. Not many can make that claim.

Core to Singhal’s talk was a focus on what Google does look at when improving unpaid search algorithms. The key is always relevance.

Singhal talked about the evolution of Google’s unpaid search algorithms. In 2003, they worked on stemming and synonyms. This meant that those searching for [watch buffy the vampire slayer] [watching buffy the vampire slayer] and [view buffy the vampire slayer] would likely all see the same results. In 2007, came universal search, which was a big step forward in understanding searcher intent. (Searchers typing in [i have a dream] not only are looking for Martin Luther King Jr.’s speech,  but would like to see a video of it.)

Understanding Intent

Ten years ago, search results were keyword-based, but Google is now moving towards understanding the intent behind the words. Singhal talked about Google’s acquisition of the company FreeBase, which has done substantial work on understanding phrases as entities rather than strings. “Mount Everest” isn’t just two words, it’s also a mountain, with a height, in a location, and so on. (Shortly after the talk, Google launched their Knowledge Graph, which is the next step in this understanding.) Combine intent with speech recognition and mobile devices and you almost end up with what Singhal first glimpsed years ago on Star Trek. We do indeed, live in the future (almost).

Personalization

In 2012 took a big step (whether or not that step was forward is up for debate) towards greater personalization with Search Plus Your World, which began incorporating Google+ into search results for those logged in. Singhal explained that Google+ integration was not the point, it was just a proof of concept. The point was a foundation for a wider world of (more secure) searching over everything: both what’s public in the world and what’s private to each searcher. Perhaps one day Google will in fact be able to find your car keys.

Singhal said that searcher click behavior shows that searchers are happy with this integration. But he acknowledged there’s work to be done. When asked when it would launch in Europe, he said that based on feedback, it’s undergoing improvements first.

Relevance and Data: How Changes Are Evaluated

Search Plus Your World is built and evaluated the way all ranking algorithm changes are: build, evaluate, launch, learn, improve, repeat. Relevance is key to every measurement. Singhal stepped through the process:

1. An engineer at Google has an idea of a signal (one of over 200) that might be introduced or tweaked to improve overall relevance.
2. That algorithm change is run on a test set of data and if all looks good, human raters look at before and after results for a wide set of queries (a kind of manual A/B test). The human raters don’t know which is the before and which is the after. The raters report what percentage of queries got better (more relevant) and what percentage got worse (less relevant).
3. This process gets looped several times as the algorithm is tweaked to better serve results for the queries in the “worse” set.
4. Once the overall manual ratings show that the algorithm tweak makes results better overall, it’s all tested again. This time, a data center (one of many that contains Google’s index and serves results to searchers) is loaded with the new algorithm and a very small slice of searchers (typically 1%) see the modified result set. Are those searchers happier than the ones seeing the version of results without the tweak? Singhal says they compare where searchers click. Clicks on higher ranked pages mean results at the top are likely more relevant, and searchers are happier. (He didn’t say so, but they may look at other data, such as click and back behavior.)
5. An independent analyst compiles the results and provides a statistical analysis, which is presented at a search quality meeting, where engineers look at the data and debate the change. If they decide this tweak improves the quality of search results overall (and is good for the web and doesn’t overly tax internal systems), the change goes out.

This process is happening all of the time with lots of different proposed tweaks and tests. 525 algorithm changes were launched in 2011. That may seem like a lot, but earlier this year Singhal noted that many more changes were tested.

“Concurrently we have approximately 100 ideas floating around that people are testing – we test thousands in a year. Last year we ran around 20,000 experiments. Clearly they don’t all make it out there but we run the process very scientifically.”

Aggregated data from millions of searchers typing millions of queries provides clear patterns. Singhal said that not only do those who get better results more quickly click higher in the search results, but they also search more. (We’ve heard this before from Google. Marissa Mayer, for instance has noted that a half a second delay in rendering search results resulted in 20% fewer searches).

Singhal noted that the kind of personalization platform envisioned with Search Plus Your World is harder to test. Human evaluation looks at relevance, but personal relevance is unique for each searcher. All Google really has to go on is click behavior. Singhal talked with Danny Sullivan about this dilemma a few weeks after Search Plus Your World launched:

“Every time a real user is getting those results, they really are delighted. Given how personal this product is, you can only judge it based on personal experiences or by aggregate numbers you can observe through click-through.”

All of this gets complicated by varied screen size. The user interface becomes more important as increased use of mobile devices and tablets shrink screen real estate.

If these changes are all about increased relevance, why is only Google+ represented in Search Plus Your World? Why not Facebook and Twitter? Singhal explained that most personally useful Facebook data is locked behind a login, and Twitter produces content at a rate that is too massive for Google to crawl quickly and comprehensively. Or, they could, but it would probably take down the Twitter servers. Twitter has also had some technical issues that have made crawling difficult, although are being fixed.

What About Panda and Penguin?

Singhal said that Google’s algorithms aren’t perfect (hence the 20,000 experiments a year). He looks at bad queries every day (and encouraged the audience to let him know about them! So, add them to comments to this post and we’ll forward them along). But when asked specifically about Panda and Penguin, two of the latest high profile algorithm changes, he said that data has shown they significantly improved the number of high quality sites being returned in results. They are not only refining what signals they use in ranking, but are improving how they gather and tune the signals themselves (so signal quality is higher). They are  constantly looking for aberrations in signals.

At the end of the day, he said, site owners need to take a hard look at what value their sites are providing. What is the additional value the visitor gets from that site beyond just a skeleton answer? Ultimately, it’s those sites that provide that something extra that Google wants to showcase on the first page of search results.

Knowledge Graph? “Graph” is a technical term used to describe how a set of objects are connected. Google has used a “link graph” to model how pages link to each other, in order to help determine which are popular and relevant for particular searches. Facebook has used a “social graph” understand how people are connected. “Knowledge Graph” is Google’s term for how it is building relationships between different people, places and things and report facts about these entities.

Big Change, Subtle Appearance

Earlier this year, the Wall Street Journal wrote about the coming change. At the time, I felt what was described seemed more an extension of things Google had already been doing rather than a dramatic shift. Now having seen it first-hand, I stand corrected. The WSJ had it right. This is indeed a big change in line with other major launches like Search Plus Your World last January and Universal Search in 2007.

Big change, but I don’t think it’ll be a shocking change to most Google users who will begin seeing it over the coming days on Google.com, if they’re searching in US English.

Google will still look largely the same as it does now. Knowledge Graph information flows into new units — they have no official name (and I did ask), so I’ll call them “knowledge panels.” These panels appear to the right of Google’s regular results, rather than disrupt those familiar links:

Knowledge panels don’t always appear, only showing up only when Google deems them relevant. But when Google does think they’re relevant, they’re a pretty cool search exploration tool. When the head of Google Search, Amit Singhal, let me play with the new system following his keynote talk at our SMX London show yesterday, I couldn’t help but think of it like a form of StumbleUpon or channel surfing for search.

Fact Surfing

A search for Star Trek brought up a panel that included a reference to Star Trek: Voyager, my favorite of all the series. Jumping to explore that, the Voyager box included a reference to Babylon 5, another favorite sci-fi show of mine. Jumping to that box, there was a reference to Claudia Christian, who wonderfully played one of the main characters in Babylon 5, Susan Ivanova. I surfed over for a look.

If you’ve ever started reading a Wikipedia page and then gotten lost jumping from one topic to another, that’s the experience I think many are about to discover with Google. You’ll not only discover answers to factual questions, but you’ll likely quickly explore more than you had planned and have fun doing it.

3.5 Billion Facts About 500 Million Objects

Google says it has compiled over 3.5 billion facts, which include information about and relationships between 500 million objects or “entities,” as it sometimes calls them. In general, entities are persons, places and things. You know, nouns.

In particular, these are just some of the categories of objects Google has facts about:

• Actors, Directors, Movies
• Art Works & Museums
• Cities & Countries
• Islands, Lakes, Lighthouses
• Music Albums & Music Groups
• Planets & Spacecraft
• Roller Coasters & Skyscrapers
• Sports Teams

Again, those are just some of the categories. The relationships are also as important as the facts. The relationships allow the Knowledge Graph to know which actors to list for a particular movie or which spacecraft have visited  a planet.

The Most Popular Facts

How do you keep from getting overwhelmed with useless facts? Google picks out the facts for each object that are most sought in relation to that object.

“We are showing all the things that people look for in a given query,” Singhal told me.

Consider these two knowledge panels, one for Simpson’s creator Matt Groening, the other for architect Frank Lloyd Wright (you can click to enlarge):

For both, you’re told when they were born and where they were educated. After that, the remaining facts shown differ.

Only Groening has facts about his parents and siblings listed. Why? Look closely at the names: Margaret (Marge), Homer, Lisa. Groening named characters after his own family. Looking at searches related to Groening, Google can tell these are commonly sought answers.

For Groening, the books he’s authored are listed. For Wright, his famous buildings are. That makes sense. People are far more interested in structures by Wright than by books by him. Indeed, Google’s autocomplete suggestions — which are based on the most popular terms related to a core search topic — reflect this:

I found it fascinating to see what was shown, as I ran through various classes of searches. For Disneyland, popular rides were shown. For a ride like Space Mountain, the duration was shown (really, only 3 minutes?). For an astronaut, I was shown the missions and overall time they’d spent in space (how cool to have that as a fact about yourself). For Buckingham Palace, the size of floor space was listed. For Larry Page and Mark Zuckerberg, their estimated net worth was shown.

Each knowledge panel has a “People also search for” area at the bottom which lists related people, places or things. Again, the relationships are determined by looking at search data. People who search for Groening, for example, often search for David X. Cohen, who co-created Futurama with Groening.

For search marketers, or anyone interested in how people search, these panels have become another great discovery resource along with keyword research tools like Google Trends, Google Insights, Google Correlate or the AdWords Keyword Tool.

Facts But Not Actions

One thing I found lacking was that the knowledge panels I saw often lacked links to let people take actions related to these objects. For example, one of the popular things people want in relation to Buckingham Palace is to book tickets for tours. However, the panel had no options for this.

In contrast, the new “Snapshots” announced (but still about a week from going live) as part of Bing’s relaunch last week are heavy on trying to help people do things like book tickets or reservations.

Why not have actions?

“We will, of course, explore that, but right now, we just want to take it out and see how it works,” Singhal said.

Occasionally you can take actions via the links to some of the source providers of facts, as with some music searches that might credit Songkick or StubHub.

Which Andromeda Did You Mean?

For some searches, there may be more than one entity that Google has facts for related to a search. In these cases, rather than make the wrong guess, Google will put up a “See results box” as shown below for Andromeda:

Andromeda could mean, in Google’s Knowledge Graph, the galaxy, the TV show or the Swedish band. This box, also known as a disambiguation box, allows people to make the right choice.

Where Do The Facts Come From?

How does Google know any of these facts? Google Squared was an initial attempt in 2009 to extract facts from the web. Google still has that technology, but the service was never that impressive on accuracy and closed as standalone site last year.

Rather, it was Google’s purchase of Metaweb in 2010 that really jump-started the Knowledge Graph. Metaweb was building both the relationships and, though Freebase, a database of facts.

Since that time, Singhal said Google’s massively grown the fact database. Contributions happen with Freebase, but data also comes from publicly-available sources like Wikipedia and The CIA World Factbook and even information out of Google Books. Beyond that, Google also licenses data from others.

“Wherever we can get our hands on structured data, we add it,” Singhal said.

Fixing Bad Data

Drawing from Wikipedia and other public sources means that there’s no guarantee that the facts are accurate. That’s why the knowledge panels on Google all have a “Report a problem” link at the bottom.

If you click on that, you can then indicate if any particular fact is incorrect. Singhal said that Google will use a combination of computer algorithms and human review to decide if a particular fact should be corrected

If Google makes a change, the source provider is told. This mean, in particular, Wikipedia will be informed of any errors. It doesn’t have to change anything, but apparently the service is looking forward to the feedback.

“They really are excited about it. They get to get feedback from a much bigger group of people,” Singhal said.

Will Publisher Traffic Drop?

Search engines have increasingly moved toward showing direct answers in their results over the years. Such efforts have worried some publishers, leaving them wondering if they’ll be left out of receiving search traffic. After all, if search engines provide answers right within their results, why would anyone click away?

Google’s Knowledge Graph is going to massively increase the number of direct answers shown, which will almost certainly renew concerns.

Singhal’s response is that publishers shouldn’t worry. He said that most of these types of queries, Google has found, don’t take traffic away from most sites. Part of this seems to be that the boxes encourage more searching, which in turn still eventually takes people to external sites.

Still, some are going to lose out, he admits. But he sees that as something that was going to happen inevitably, anyway, using a “2+2″ metaphor. If people are searching for 2+2, why shouldn’t Google give a direct answer to that versus sending searchers to a site? By the way, Google does do math like this already and has for years.

Below, you can hear Singhal talk more about this when asked by a member of the audience at SMX London yesterday:

Click here to view the embedded video.

My concern is what happens if publishers have compiled great information that someone at Wikipedia or Freebase harvests into a database. For example, if a Disneyland fan site has organized a list of ride durations by doing original legwork, what credit do they get if that data is used? Facts can’t be trademarked, at least in the US, so anyone can help themselves assuming they don’t duplicate the exact format or presentation.

Google does list credit links to places like Wikipedia. In turn, Wikipedia does give credit (albeit in a way that doesn’t help search rankings) to the sources it draws from. But that puts actual source material two clicks away from the searcher, assuming the searcher wants to go beyond the fact they already received.

This is one that has to be watched closely. As I wrote before, it seems likely the Knowledge Graph will impact a relatively small set of sites that focus on facts, sites that already likely exposing answers in their listing descriptions and so not getting traffic anyway. But we’ll see.

It’s also important to remember that the “main” results aren’t disappearing. Consider again the Frank Lloyd Wright search, this time with the knowledge panel in context with the regular results:

As you can see, links to sites outside of Google remain to the left and in the most viewed area of a search results page.

Being Included

What if you want to be part of the new knowledge panels and Knowledge Graph in general? Singhal said that at the moment, there’s no mechanism designed for sites to do this. IE, if you run a site about Frank Lloyd Wright, there’s no way to be associated as some type of suggested source for the Frank Lloyd Wright panel.

Potentially, you could head over Freebase, open an account and contribute. Of course, I’m pretty sure adding your blog to a horrible list of blogs like this isn’t going to help. Maybe other categories might be more successful, but I’d hold off, for the moment.

Tagging parts of your pages with commonly used schema might be helpful, though I wouldn’t do this solely in hopes of getting your facts into the Knowledge Graph. The articles below have more about using schema:

• Schema.org: Google, Bing & Yahoo Unite To Make Search Listings Richer Through Structured Data
• Google Takes First Big Bite Into Rich Snippet Search With Recipes
• A New Google Rich Snippet For Real Estate, Other Business Listings
• Google Adds Rich Snippets For Application Reviews: iTunes Apps, Android & More
• Google Adds Rich Snippet Support For Music
• How Retailers Can Improve Product Visibility Using Structured Markup
• Concert Rich Snippets: List Your Ticket Sale Site Under Band Web Sites
• MicroData & Retail Products: Not Ready For Primetime?
• How To Use Rich Snippets, Structured Markup For High Powered SEO

Ads, Mobile & Tablet Formats

Anyone familiar with Google’s ads will immediately wonder what happens when the panel shows.

Singhal said that if there are also ads along with a knowledge panel for any search, the ads will still display. Google also has different formats for when a query has a few, many or no ads. I haven’t seen these, but I’ll try to update as they become visible after the launch.

In addition, Google also uses special formats to make the panels work well on tablet and mobile devices, he said. They aren’t restricted to just desktop search, so that’s good news for those of you who want an easier time to cheat at pub and bar quiz nights.

Sadly, there’s no way to just search the Knowledge Graph directly. It only appears with regular Google Search.

The Competition

Google’s not alone in having a knowledge graph, of course. Wolfram Alpha, launched in 2009, has continued to refine its service. It got a big boost being picked as a search partner by Apple to help power Siri (even though that recently embarrassed Apple on a particular search about smart phones).

As for Bing, it has a partnership with Wolfram Alpha plus owns Powerset technology that, somewhat similar to the Knowledge Graph, tries to deeply understand the meanings of words, rather than just really match patterns of letters.

But Bing hasn’t really seemed to capitalize on either its Wolfram partnership nor Powerset. Really, the Knowledge Graph seems to be going more head-to-head with Wolfram Alpha. Does it?

“Wolfram is far more computational,” Singhal said, explaining that Wolfram Alpha’s goal seems to be finding ways that you can effectively use facts in computations.

For example, you can enter cars in california / california population into Wolfram Alpha to have it take those two facts and come up with an average (about 1 car for every two people, by the way, using 2009 data).

Google’s not trying to perform these types of calculations. The focus is instead on providing popular facts.

The Future

The big picture, of course, is that some day the Knowledge Graph won’t just be used for facts. Instead, if Google can better tag actual web pages to entities, then it can better understand what those pages are about and related to, which might increase the relevancy of its regular results.

That’s down the line, as are many other changes to the knowledge panel themselves. Today represents only a start.

“This is just a baby step, in my view, to expose this to our users,” Singhal said.

To learn more about the Google Knowledge Graph, see coverage from others across the web organized here on Techmeme, the official Google blog post, plus the official video, below:

Click here to view the embedded video.

Related Articles

• Google 2.0: Google Universal Search
• Powerset Launches “Understanding Engine” For Wikipedia Content
• Wolfram Alpha Live Review: The Un-Google
• Up Close With Google Squared & Some Wolfram Alpha Thoughts
• Google Buys Metaweb To Bolster Answers, Google Squared & Rich Snippets
• How Google Instant’s Autocomplete Suggestions Work
• WSJ Says Big Google Search Changes Coming? Reality Check Time!
• Google Testing “Sources” Area With Info About Movies, Books, People, Music & More
• Bing Relaunches, Features New Social Sidebar

We’ve been here before, such last December, when Apple was accused of preventing Siri from finding abortion clinics. Actually, that was largely down to abortion clinics themselves not calling themselves that, as my post from last year explains: Why Siri Can’t Find Abortion Clinics & How It’s Not An Apple Conspiracy.

Say What, Siri?

What’s going on now? It seems to have started with WMPoweruser, a Windows Phone site, which published a screenshot showing how Siri, when asked “What’s the best cell phone ever,” listed the Nokia Lumia 900:

WMPoweruser correctly explained that the information was actually coming from Wolfram Alpha, which provides certain types of answers to those who search using Siri. Similarly, Yelp provides some answers, as does Google.

Still, the story went on to hit a number of tech blogs since then. MG Siegler follows it around, and Techmeme has a round-up of stories. Me, I thought I’d trace the data itself back to the source, Wolfram Alpha.

Say What, Wolfram Alpha?

If you ask on Wolfram “What’s the best cell phone ever,” it comes back with the Lumia listed like this:

This is the listing that “powers” what some saw on Siri.

Note the two arrows. The first is Wolfram is deciding to show “best mobile phones” sorted by the highest customer reviews. It picks the top one from a full list shown further down on the page. The second arrow shows how many reviews are apparently getting the Lumia that top rating.

It’s a total of four. Four is a pretty small number. You really want a lot more reviews than that before you go declaring anything to be the best.

Say What, Best Buy?

You can’t actually see these reviews on Wolfram Alpha, but I can show you where they’re almost certainly coming from. Best Buy.

At the bottom of the page is this:

Click on the image if you want to enlarge it. See the arrow? Clicking on the “Source information” link that it’s coming from makes the Source Information box it’s pointing to appear. The source turns out to be Best Buy.

Unfortunately, you can’t click to any actual list that Best Buy might have of “the best cell phones ever.” But I can go to the Best Buy page for the Lumia 900 and see what’s shown:

Here, we can see that the Lumia has a 5 star rating like Wolfram shows. This time, it’s based on six reviews. That means Wolfram is probably a little out-of-date. But then look at the bottom. Actually, this has a 4.4 star rating, based on 11 reviews. What’s happening there? I don’t know. Maybe these are reviews for all colors of the Lumia.

Now, what I really want is the list of top rated phones from Best Buy, to see if they match the list Wolfram has. As it turns out, that’s pretty easy. There’s a mobile phones page, with an option to filter to top-rated phones.

Doing this, it turns out the top rated phone is the Rocketfish Mobile-Micro USB Data Transfer Cable. I know that seems weird, but when you speak into one side of it, the person at the other end can hear really well.

OK, there’s some glitch. But after that, it’s the iPhones coming up:

I kept trying to find a way to get Best Buy to give me a list like Wolfram had. I could never exactly replicate it. But I got close. That screenshot above? By default, top rated phones are also shown weighted by top selling, which makes sense. You’re getting a combination of metrics.

If I sorted by purely highest rated, the HTC Trophy came first, followed by the iPhone. The same thing happened if I kept that sort but narrowed to “Best Buy items.” If I instead narrowed to online-only items, then I could make the Lumia 900 show up.

Of course, Wolfram will use its own mechanism for sorting. The bottom line is that Wolfram has ratings from Best Buy, and it’s not trying to weight those in any particular fashion such as number of reviews or number of purchases.

The Lumia rates tops on Wolfram because four people gave it 5 stars, versus 86 people who give the AT&T 16GB version of the iPhone 4S an overall rating of 4.7. The Lumia is batting 1.000 after being up to bat only 4 times. The iPhone is batting .940 after 47 at bats. Or something like that. I like baseball, but I’m not a baseball stats expert.

Say What, Lumia?

All this got me thinking. What would the Lumia give me if I spoke to it? The Lumia, after all, features voice search like Siri, only it brings back results from Bing.

A search for “What’s the best cell phone ever?” brought back answers that were polluted with news stories about what’s happening with Siri and the Lumia today. So, I tried a search for “best cell phones,” figuring that was fair. A search for the singular “best cell phone” also gave me the same top sites:

These results come from Bing, Microsoft’s search engine, and the top site listed is a review of what CNET considers the top five cell phones. The best phone on the list the Lumia recommends, via Bing’s results, is the Motorola Droid Razr Maxx:

The Lumia is even recommending the iPhone over itself, if you want to attribute this list as a Lumia recommendation. The iPhone ranks third; the Lumia ranks fourth.

Meanwhile, The Next Web did that same thing but with a different search, “What is the best smartphone ever,” bringing up a review from Business Insider that recommends the iPhone.

Say What, Siri?

As for Siri, I can’t get it to do the same thing for me, not as WMPoweruser found nor when trying things like “What is the best smartphone ever,” which Apple Insider used.

Nope, for me, Siri either gives answers that Apple’s Siri team have programmed in as jokes (and which predate what happened today), or occasionally I get a local search:

As I said, some of these jokey responses did exist before. Why some started getting them to bring up Wolfram Alpha instead isn’t clear, but Siri can be strange and mysterious that way.

Postscript, May 14: CNET has a story up suggestion that Siri has suddenly changed to these jokey responses. As I said, I was pretty sure they existed before. I was correct, as you’ll see in these examples from last year.

By the way, I coincidentally wrote a column today over at CNET about what it’s like living with phones for all three major smartphone platforms, the iPhone, Android and Windows Phone.

I like them all in various ways. Windows Phone is nice, in fact nicer than Android, when it comes to doing email (the iPhone is even better). Android has its own strengths over the iPhone, just as sometimes the iPhone is better than Android.

Ultimately, what’s the best smartphone will be down to the person who actually uses it.

Now I’m off to order some tomato soup. Because it’s raining outside, and I don’t want to put on real shoes:

Click here to view the embedded video.

Postscript: See my follow-up story, With Fix In Place, Wolfram Alpha Explains How Siri “Recommended” The Lumia By Mistake

Hey Google, How’s New Girl?

I thought I’d start with a search for New Girl. It’s a great show. I should know. I was a Fan of the Week. What’s Google show me?

The arrows point at things that have come because of social connections, and I’m only going to focus on these things, in this head-to-head. Do these additions seem to help or hurt my results?

Google tells me at the top that someone I know has +1′d the official site. Maybe that calls my attention to it more, but I could take it or leave it.

Further down, my own post on Google+ about being made Fan of the Week appears. I say point to Google. Ego searches are one of the key ways people historically judge search engine relevancy. Can they find your own stuff? Google could.

Further down, the official New Girl Facebook page is listed. It only shows up this high because I’m “friends” with New Girl on Google+. Content that your friends like moves higher. If I turn off the personalization that Search Plus Your World does, that link drops from the top results.

Finally, to the right side is information from the New Girl page on Google+. This is nice to see, helpful.

New Girl On Bing

Over on Bing, there’s less going on. The IMDB listing has a thumbs-up, to alert me that one or more of my friends likes it. I like this a little better than Google flagging pictures. It feels cleaner, less intrusive.

Further down on the page, I’ve shown one of two different links that appear with a trending arrow, to alert me they are currently being frequently shared on Twitter. Again, kind of nice, though also kind of confusing when there’s also a news box.

Bing really falls down in the Sidebar are, where it should really shine. I follow the Facebook New Girl page, but that’s not shown. I do get the Twitter account of Zooey Deschanel, the main character of the series. But why not show me at least the Twitter account for the New Girl show itself?

In the end, I think Google comes out ahead on social integration, in this case.

Google & Penguin

If you’re in the search marketing space, Penguin recently changed from being one of those cute flightless birds to the name of a new Google update aimed at web spam. This seemed a perfect test for the new social features. What would a search for Penguin yield?

Mixed among the regular results, Google gave me three based on my social connections. The top was something I myself had just shared on Google+, a new article I published about Penguin. OK, relevant, but not that helpful. I know I published the article. And really, what I’d rather have is a link to the actual article instead of a link to my post about the article.

After that, more Google+ — this time a post from my news editor Barry Schwartz, sharing a video related to Penguin. Lastly, an article I wrote on Marketing Land.

Overall, these things make sense to appear, but some more variety, some posts on Penguin from others in my social network, those might have been more helfpul.

Bing & Penguin

Next up, Bing:

The Sidebar on this particular query is pretty useless. It’s mostly people who have pictures of penguins. Apparently, none of my Facebook friends have shared or posted anything to do with the Penguin Update.

Over in the main search results, impressive success. Because I and another person have liked my original article on the Penguin Update, it ranks tops in my personalized results.

I call this test about even. I felt Google didn’t give me much variety, but neither did Bing.

X-Games

One of my son’s is big on BMX, so I tried a search for X-Games as a final test. Here’s are the social results I got back, Google on the left, Bing on the right:

Nothing dramatic, really. Wow, I get my own pictures on Google. Might be useful, but since I know X Games itself on Google+, why not at least suggest that? Meanwhile on Bing, I can see that several of my friends like X Games on Facebook, but why not list the page itself?

Social Is Still Developing

In theory, social mixed with search sounds great. In reality, it’s still has a long way to go. For some background on some of the challenges, I really urge you to see my post from earlier this year, When Everyone Gets The Vote: Social Shares As The New Link Building.

Related Articles

• The New Bing: Microsoft Tries Again With Search Meets Social
• FAQ: What’s The Debate About Google’s Search Plus Your World?

]]> http://searchengineland.com/the-new-bing-vs-googles-search-plus-your-world-120817/feed 9 http://searchengineland.com/google-talks-penguin-update-recover-negative-seo-120463 http://searchengineland.com/google-talks-penguin-update-recover-negative-seo-120463#comments Thu, 10 May 2012 13:00:09 +0000 Danny Sullivan http://searchengineland.com/?p=120463 It’s been about two weeks since Google launched its Penguin Update. Google’s happy the new spam-fighting algorithm is improving things as intended. But some hurt by it are still wondering how to recover, and there remain concerns about “negative SEO” as a threat. I caught up with Matt Cutts, the head of Google’s web spam team, on these and some related questions.

Penguin: “A Success”

The goal of any algorithm update is to improve search results. So how’s Penguin been for Google?

“It’s been a success from our standpoint,” Cutts said.

What About Those Weird Results?

Of course, soon after Penguin was released, people quickly started citing examples of odd results. The official Viagra site wasn’t listed, while hacked sites were. An empty web site was listed for “make money online,” and there were reports of other empty sites ranking well. Scraper sites were reported outranking the sites they scraped.

How could Penguin be a success with these types of things happening?

Cutts said that many of these issues existed before Penguin launched and were not caused by the new spam-fighting algorithm.

Indeed, the Viagra issue, which has now been fixed, was a problem before Penguin hit. Penguin didn’t cause it.

False Positives? A Few Cases

How about false positives, people who feel they’ve been unfairly hit by Penguin when they weren’t doing any spam?

“We’ve seen a few cases where we might want to investigate more, but this change hasn’t had the same impact as Panda or Florida,” Cutts said.

The Panda Update was Google’s big update that targeted low-quality spam last year. The Florida Update was a major Google update in 2003 intended to improve its search quality.

I’d agree that both of those seemed to have impacted more sites than Penguin has, based on having watched reactions to all these updates. Not everyone will agree with me, of course. It’s also worth the regular reminder that for any site that “lost” in the rankings, someone gained. You rarely hear from those who gain.

Bottom line, Google seems pretty confident that the Penguin Update is indeed catching people who were spamming, as was intended.

Why Spam Still Gets Through

Certainly when I’ve looked into reports, I’ve often found spam at the core of why someone dropped. But if Penguin is working, why are some sites that are clearly spamming still getting through?

“No algorithm is perfect. While we’d like to achieve perfection, our litmus test is, ‘Do things get better than before?’,” Cutts said.

Cutts also explained that Penguin was designed to be quite precise, to act against pages when there was an extremely high-confidence of spam being involved. The downside is that some spam might get through, but the upside is that you have fewer false positives.

How Can You Recover?

One of the most difficult things with this update is telling people how to recover. Anyone hit by Penguin was deemed to be spamming Google.

In the past, if you spammed Google, you were told to file a reconsideration request. However, Google’s specifically said that reconsideration requests won’t help those hit by Penguin. They’ll recover naturally, Google says, if they clean the spam up.

However, one of the main reasons I’ve seen when looking at sites hit by Penguin seems to be bad linking practices. People have used sponsored WordPress themes, or poor quality reciprocal linking, have purchased links or participated in linking networks, such as those recently targeted by Google.

How do people pull themselves out of these link networks, if perhaps they don’t have control over those links now?

“It is possible to clean things up,” Cutts said, and he suggested people review two videos he’s done on this topic:

Click here to view the embedded video.

Click here to view the embedded video.

“The bottom line is, try to resolve what you can,” Cutts said.

Waiting On Penguin To Update Again

If you do clean things up, how will you know? Ideally, you’ll see your traffic from Google recover, the next time Penguin is updated.

That leads to another important point. Penguin, like Panda, is a filter that gets refreshed from time-to-time. Penguin is not constantly running but rather is used to tag things as spam above-and-beyond Google’s regular spam filtering on a periodic basis.

Is Penguin a site-wide penalty like Panda or page-specific? Cutts wouldn’t say. But given that Panda has site-wide impacts, I think it’s a fair assumption that Penguin works the same.

What that means is that if some of your site is deemed Penguin-like, all of it may suffer. Again, recovery means cleaning up the spam. If you’ve cleaned and still don’t recover, ultimately, you might need to start all over with a fresh site, Cutts said.

New Concerns Over Negative SEO

Before Penguin, talk of “negative SEO” had been ramping up. Since then, it seems to have gotten worse in some places. I’ve seen post-after-post making it sound as if anyone is now in serious danger that some competitor can harm them.

At the core of these fears seems to be a perfect storm of assumptions. Google recently targeted some linking schemes. That caused some people to lose traffic. Google also sent out warnings about sites with “artificial” or “unnatural” links. That generated further concerns in some quarters. Then the Penguin Update hit, which caused more people to lose traffic as they were either hit for link spam or no longer benefited from link spam that was wiped out.

These things made it ripe for people to assume that pointing bad links at a site can hurt it. But as I wrote before, negative SEO concerns aren’t new. They’ve been around for years. Despite this, we’ve not seen it become a major concern.

Google has said it’s difficult for others to harm a site, and that’s indeed seemed to be the case. In particular, pointing bad links at a good site with many other good signals seems to be like trying to infect it with a disease that it has antibodies to. The good stuff outweighs the bad.

Cutts stressed again that negative SEO is rare and hard. “We have done a huge amount of work to try to make sure one person can’t hurt another person,” he said.

Cutts also stressed again what Google said before. Most of the those 700,000 messages to publishers that Google sent out earlier this year were not about bad link networks. Nor were they all suddenly done on the same day. Rather, many sites have had both manual and algorithmic penalties attached to them over time but which were never revealed. Google recently decided to open up about these.

After Negative SEO Campaign, A Link Warning

Of course, new messages do go out, which leads to the case of Dan Thies. His site was targeted by some trying to show that negative SEO works. He received an unnatural link warning after this happened. He also lost some rankings. Is this the proof that negative SEO really works?

Thies told me that his lost rankings were likely due to changes he made himself, when he removed a link across all pages on his site that led back to his home page. After restoring that, he told me, he regained his rankings.

His overall traffic, he said, never got worse. That tends to go against the concerns that negative SEO is a lurking threat, because if it had worked enough to tag his site as part of the Penguin Update, he should have seen a huge drop.

Still, what about link warning? Thies did believe that came because of the negative SEO attempt. That’s scary stuff. He also said he filed three reconsideration requests, which each time returned messages saying that there were no spam actions found. Was he hit with a warning but not one that was also associated with a penalty?

I asked Cutts about the case, but he declined to comment on Thies’s particular situation. He did say that typically a link warning is a precursor to a ranking drop. If the site fixes the problem and does a reconsideration request quickly enough, that might prevent a drop.

Solving The Concerns

I expect we’ll continue to see discussions of negative SEO, with a strong belief by some that it’s a major concern for anyone. I was involved in one discussion over at SEO Book about this that’s well worth a read.

When it’s cheaper to buy links than ever, it’s easy to see why there are concerns. Stories like what happened to Thies or this person, who got a warning after 24,000 links appeared pointing at his site in one day, are worrisome.

Then again, the person’s warning came after he apparently dropped in rankings because of Penguin. So did these negative SEO links actually cause the drop, or was it something else? As is common, it’s hard to tell, because the actual site isn’t provided.

To further confuse matters, some who lost traffic because of Penguin might not be victims of a penalty at all. Rather, Google may have stopped allowing some links to pass credit, if they were deemed to be part of some attempt to just manipulate rankings. If sites were heavily dependent on these artificial links, they’d see a drop just because the link credit was pulled, not because they were hit with a penalty.

I’ve seen a number of people now publicly wishing for a way to “disvow” links pointing at them. Google had no comment about adding such a feature at this time, when I asked about this. I certainly wouldn’t wait around for it now, if you know you were hit by Penguin. I’d do what you can to clean things up.

One good suggestion out of the SEO Book discussion was that Google not penalize sites for bad links pointing at them. Ignore the links, don’t let the links pass credit, but don’t penalize the site. That’s an excellent suggestion for defusing negative SEO concerns, I’d say.

I’d also stress again that from what I’ve seen, negative SEO isn’t really what most hit by Penguin should probably be concerned about. It seems far more likely they were hit by spam they were somehow actively involved in, rather than something a competitor did.

Recovering From Penguin

Our Google Penguin Update Recovery Tips & Advice post from two weeks ago gave some initial advice about dealing with Penguin, and that still holds up. In summary, if you know that you were hit by Penguin (because your traffic dropped on April 24):

• Clean up on-page spam you know you’ve done
• Clean up bad links you know you’re been involved with, as best you can
• Wait for news of a future Penguin Update and see if you recover after it happens
• If it doesn’t, try further cleaning or consider starting over with a fresh site
• If you really believe you were a false positive, file a report as explained here

Just in, by the way, a list of WordPress plug-ins that apparently insert hidden links. If you use some of these, and they have inserted hidden links, that could have caused a penalty.

I’d also say again, take a hard look at your own site. When I’ve looked at sites, it’s painfully easy to find bad link networks they’ve been part of. That doesn’t mean that there’s not spam that’s getting past Penguin. But complaining about what wasn’t caught isn’t a solution to improving your own situation, if you were hit.

Related Articles

• What Is SEO / Search Engine Optimization?
• The Periodic Table Of SEO Ranking Factors
• Search Engine Land’s Guide To SEO
• Google Launches “Penguin Update” Targeting Webspam In Search Results
• Did Penguin Make Google’s Search Results Better Or Worse?
• Penguin’s Reminder: Google Doesn’t Owe You A Living, So Don’t Depend On It
• Penguin Update Peck Your Site By Mistake? Google’s Got A Form For That
• Penguin Update Recovery Tips & Advice

Experiment Confirmed

I noticed these appearing yesterday in my own search results, and Google has confirmed that there’s an experiment happening:

We’re always experimenting with ways to improve search, but we have nothing to announce at this time.

Google does indeed often experiment with new formats, randomly tagging some visitors to see the formats being tested. I apparently was one of those tagged. Most people, however, won’t see some of the examples I’ll share below, as they’re not tagged into the experiment.

I think that will change in the near future, because as I’ll explain at the end, this is one test that I suspect will go fully live soon.

TV Shows

Here’s what I see in a search for Lost:

You can see off to the right-hand side there’s a little summary about the TV show “Lost” that’s drawn from Wikipedia. Cast members are listed, along with a “People also search for” area showing other TV shows that seem related to Lost.

In the previous test last November, this box was called “Sources.” That label is now gone. I’m continuing to call it the “Sources” box for want of a better name.

Actors

Clicking on the links generates a new Google search along with further information. Here’s a close-up of what’s shown for Evangeline Lilly:

Movies

Similar units appear for movies, such as this for in a search for Avengers:

Music & Bands

Musical groups like U2 appear:

People

There’s also extended information about some people, apparently if they are listed in Wikipedia. For example, a search for “dooce” brings up extended information about blogger Heather Armstrong, who is known by that name:

Books

Even books can get a special display:

With Or Without Google+

None of the units are dependent on Google+, nor are they part of Search Plus Your World. They appear whether you’re signed in or not, whether you use Google+ or not. But if you are part of Google+, the units might get additional information.

For example, signed-out, I see this for a search on the TV show New Girl:

Signed-in, since I follow the New Girl page on Google+, information from that appears in a “From your circles” area above the sources box:

New Format For Direct Answers Live For Everyone?

Google is also showing direct answers that come from different sources. Bas van den Beld from State Of Search also appears to be in the test and noted that for a search on “hot fuzz director,” he got a direct answer like this:

This is similar to what you see at Bing for the same search, where the information is disclosed as coming from Freebase, a service that Google owns:

However, these direct answers don’t seem part of the sources experiment. For example, using my Chrome browser in “incognito” mode, which keeps me out of the Google sources test, I can still make these boxes appear, such as this for “avengers cast:”

Is This The “Search Refresh” The WSJ Wrote About?

Direct answers like these have long been part of Google. The formats, however, seem new. The addition of the sources box also suggests that what the Wall Street Journal wrote about earlier this year, about Google offering more direct and semantic answers, is about to happen.

In WSJ Says Big Google Search Changes Coming? Reality Check Time!, I took the WSJ article to task because I thought it overhyped what was likely to happen, which was Google expanding things that it already does.

That still seems to be the case. This will no doubt be a big change to hit Google. But it will also be one of those “evolutionary not revolutionary” type of changes.

If you see these boxes, then you’re one of the few who have also been tagged as part of the experiment. If you don’t, then there’s not much you can do but wait. It seems likely they’ll eventually come for everyone.

Related Articles

• Meet The Google OneBox, Plus Box, Direct Answers & The 10-Pack
• Google Squared Is Now Live
• Up Close With Google Squared & Some Wolfram Alpha Thoughts
• Google Squared Powers Answer Sources & Something Different Refinements
• Google Goes Beyond Answers, Starts Guessing Release Dates
• Google Buys Metaweb To Bolster Answers, Google Squared & Rich Snippets
• Confirmed: Google Testing New “Sources” Display In Search Results
• Google’s Results Get More Personal With “Search Plus Your World”
• Two Weeks In, Google Says “Search Plus Your World” Going Well, Critics Should Give It Time
• WSJ Says Big Google Search Changes Coming? Reality Check Time!

While that may not surprise much of the SEO industry, it contradicts what some Google employees have previously said, as well as another recent study on Wikipedia’s search visibilty.

Last week, Conductor released updated data about Wikipedia’s search visibility, this time including Bing for comparison to its previous study that focused solely on Wikipedia’s Google visibility.

In the previous study, Wikipedia appeared on page one of Google 46 percent of the time, and on page two for 25 percent of the searches/keywords.

In its updated study, Wikipedia appeared on Bing’s page one for 31 percent of searches, and on page two only five percent of the time. The chart below shows Wikipedia’s visibility on Google (blue) and Bing (green) for all 2,000 combined keywords, as well as for the separate informational and transactional keywords.

(Note: The image above is a replacement provided by Conductor because the original image had a typo.)

At various conferences that I’ve attended, Google’s Matt Cutts has mentioned seeing earlier studies that had Bing showing Wikipedia more often in its search results than Google does. He echoed that sentiment in the comments of his Google+ post about Conductor’s earlier study. A recent Search Engine Watch study also suggested that Bing favors Wikipedia more than Google.

Why the discrepancies? It’s likely due to survey methodology, not to mention the rapidly changing search results landscape. (Search results can differ day-to-day, even hour-to-hour, so a study done one month is likely to differ from another study done a month later.)

In its studies, Conductor compared 2,000 informational and transactional keywords of varying word lengths. Navigational keywords were not included because Conductor found very low Wikipedia visibility for those types of keywords.

Bing Still Loves Wikipedia, Just Differently Than Google

Conductor’s recent study did reveal an interesting trend: Even though Wikipedia appears more often on Google’s page one, when Wikipedia does show up on Bing’s page one, it’s at the top of the results more often.

When Wikipedia is one page one, Conductor found it in the top three results 83 percent of the time on Bing, compared to 65 percent on Google.

Conductor also removed one-word queries from its study and found that, overall, Wikipedia appeared about 10 percent more often on Google’s page one than on Bing’s page one.

As I mentioned above, with search results changing so frequently, no study is likely to end the discussion on Wikipedia’s search visibility. But, if nothing else, all of this data related to search visibility — whether it’s Wikipedia or other major sites like Amazon — makes for good industry discussion.