Did Penguin Make Google’s Search Results Better Or Worse?

Google’s latest search algorithm change designed to fight spam and improve its search results went live yesterday. Plenty of people are seeing its impact already. Better or worse? It’s easy to find some examples of things being bad; it’s hard to say overall if there’s been a net improvement or not.

Judging Relevancy Is Tricky

I’ll go through some examples, but let me start with some caveats. If you dive into some forums, you’ll find plenty of people screaming. People tend to scream after any update that things have gotten worse because they’ve lost rankings. Few scream about how things have improved, much less provide examples. In short, relying just on forums can give you a skewed view.

That doesn’t mean you ignore what you might discover through forums, however. Few know the quality of any search engine’s search results as well as SEOs. They might not be happy if something outranks them, but good ones know if something better or worse has moved up. The same is true even for some of the “good” black hat people out there. They might deliberately violate Google’s guidelines, but that doesn’t mean they don’t know search quality.

Time for examples. To get these results, I was used “Incognito Mode” in Google Chrome to help eliminate personalization other than geographic targeting. That can’t be prevented.

Viagra: Missing Official Site & Many Other Problems

One of the first queries I did to check on the new algorithm’s impact was for “viagra,” because that search is a natural magnet for the type of spam that Google seeks to wipeout with this update. If that was the goal, Google had a big miss here:

Most glaringly, Google fails to list the official Viagra site in the top results, something that Bing gets right.

Postscript: John Andrews pointed out after this was posted that the official site has apparently been gone for 10 days or more, so this particular change seems unrelated to the new spam-fighting algorithm. I had asked Google about oddities with this particular search before posting my article but never heard back.

It’s also a good reminder to add that there’s no guarantee that any of the problems I’ll cover with these examples are due to the algorithm change. There’s no easy way to do a “before-and-after” for these searches to know what’s changed. However, you can look at the current state to assess whether they seem to be of good quality overall, which is part of what Google aims for with this current and all algorithm updates.

Almost as alarming, the four sites I’ve pointed at with red arrows have been hacked so that those visiting them get redirected to some place selling Viagra online. Three of these, Google even knows they’ve been hacked, because it shows a warning to searchers. Why it allows things to rank with those type of warnings makes no sense.

The two blue arrows point to two university web sites that have nothing to do with Viagra but which rank. Chances are, they were also hacked recently and fixed their problems. But if so, they’re still not relevant for Viagra and shouldn’t rank.

Bing: Better For Viagra

Here’s Bing for viagra, for comparison’s sake:

As mentioned, Bing gets the official Viagra site listed tops as you’d expect good search results to do. Bing also lists several informational sites, such as Drugs.com, How Stuff Works and Wikipedia. But all of these are also listed by Google, so there’s no big difference there.

Where Bing really pulls ahead is that instead of all those hacked sites, it lists a lot of online stores and affiliate sites. That’s certainly better than rewarding companies that have hacked other web sites, so it had the edge over Google. But whether these are legit online stores that require prescriptions is another matter.

Postscript: As noted in the comments below, one of these sites Bing lists seems likely to have been hacked.

Make Money Online: Google Has Very Slight Edge

One query I’ve seen go around in discussions, and which was tipped to us through email and in a comment, is for “make money online.” Is Google falling down here?

Honestly, it’s pretty tough to know what should rank well for that type of query, given that it’s so broad and so heavily filled with various schemes that purport on how you can earn money online (the answer, of course, is to create an online sharing service that makes pictures look old and has no income, then sell for 1 billion).

Still, let’s make the attempt. In this case, I’m going to put Google and Bing side-by-side, Google on the left, Bing on the right. I’ve removed the related searches from Bing’s listings and slightly spaced out the listings to even them up with Google, but the order is exactly as originally appeared:

The damning thing some point to with Google is a site on Blogger that has no content at all ranking tops. Yes, that’s terrible. It’s also exactly the same thing that happens on Bing, though Bing ranks it lower on the page (sometimes, Google also ranks it lower). Certainly it would be better for Google if its ranking algorithm wasn’t returning a useless site. But pick any search, on either search engine with any update you care to name. There’s always some weird outlier.

I’ve also seen some call out Google for having two older articles ranking. The same thing happens with Bing, for one of these. It’s easier to notice on Google because it lists dates for these articles. Bing has some outdated information but doesn’t show dates.

Google is heavier on listing blogs about making money, though many of these really just seem to be blogs about blogging. Bing is heavier on places that overtly pitch some “make money” type of courses. If I had to choose what’s better, I’d go with listing the blog sites over the course sites.

In the end, I’d give Google the slight edge here over Bing. None of the results are really that great, of course. But I guess that’s what I expect for that type of search, knowing the flood of “get rich online” content that’s out there.

Overdose Advice? Bing Beats Google

Another example from a comment on our original story was about “how many pills does it take to overdose,” where Google is taken to task for featuring Q&A sites, places like Yahoo Answers, where anyone rather than an actual expert can give advice.

Here’s the side-by-side, Google on the left, Bing on the right. Again, I spaced things out a bit, but the ranking order is the same:

As you can see, both Google and Bing favor Q&A sites. I’d agree, that’s not optimal for this type of search. You’d rather get information from respected medical web sites, and not information on how to commit an overdose but what to do if you suspect you’ve accidentally done that. Neither deliver.

One of the biggest reasons for this failure is probably that the medical web sites simple don’t have content written in this type of plain language. If there’s nothing but garbage out there, garbage is what you’re going to get back.

Still, Bing gets the edge because Google has three completely irrelevant answers coming up, pages with nothing to do with the search topic. It was also disappointing not to see Google kick in the special suicide prevention information that it does offer for some searches.

Payday Loans: A Toss-Up

Another example of Google failure that was suggested is for payday loans, with Check N Go — which I gather is a major known provider — not ranking in the top two pages of results. Let’s do the side-by-side. As I said before, I did some spacing to help make it easier to compare, but no order was changed:

Bing detected my location as being in El Segundo, California — actually about 40 miles from where I live, plus it tossed in a bunch of San Diego listings. That’s a strike against it right off the mark, but I decided to manually set my location to Costa Mesa, California for both search engines, to better even the comparison.

Both lead off with the same major provider of PayDay One. Bing does have Check N Go, while Google lacks that. But Google has many local providers, places I could walk in to, displayed. Bing buries these plus gives me lots of irrelevant local providers, including one in the UK.

It’s so bad with Bing that I kind of want to call this test in Google’s favor. But not knowing this space well, I get the impression that Check N Go probably is a provider it should be listing. Plus, listing what appears to be an affiliate site that probably has fake user reviews also was a knock against Google. So, I left the two tied.

Treadmill Reviews: Bing Gets The Edge

Earlier, I said that SEOs are often subject experts in their own areas. Daniel Deceuster, who commented on the changes, seems to be an example of this. He runs a site about treadmill reviews. The comment he left was full of the type of detail that a non-expert might not pick-up:

I monitor these rankings everyday, have for over a year. They have never been as bad as they are. #4 is an article reviewing just 8 machines from 2008, none of which are available for purchase any longer. Good luck finding anything through #3. Then #5 and #6 both link to each other and are obvious members of a huge internal link scheme. #7 doesn’t even have any reviews, just “discount codes” that aren’t codes but affiliate links that don’t get any discounts. #9 is a massive chart of numbers and ratings of random machines, some of which are no longer available. No reviews and their link to more reviews returns a 404 error.

I decided to take a closer look myself. The side-by-side:

I’m not an expert on treadmill reviews, so there are things Deceuster will see that I’m probably missing. But some of the sites he dislikes that Google is listing? Bing’s listing them, too. He focuses on an outdated article. That is annoying, and it doesn’t happen in this case with Bing. But it might happen in other searches with Bing.

Overall, I’d say that Bing does seem to do a better job of getting more sites that seem to offer fairly nice reviews of treadmills to the top. The one I liked the most, with my non-expert eye, was the one that Bing listed tops and that Google didn’t have at all.

Google Sucks For New Shoes? Not So Fast….

Another popular query I’ve been seeing going around for Google’s results apparently having gotten worse is a search for new shoes. The side-by-side is below, Google on the left, Bing on the right. Related searches and the news box from Bing have been removed, but the ranking order is the same:

The criticism is really about the second listing, which looks like some odd simulation thing that is totally irrelevant. It is relevant. It’s a marketing course called “NewShoes.”

Is it the most relevant thing for Google to list? I’d say not. I’d say most people who are searching for “new shoes” probably aren’t thinking about that course but rather actual shoes. Then again, a lot of people might be searching for a song called “New Shoes,” especially given that the official video has over 5 million views on YouTube. Google lists this, as well as how to buy it on Amazon. Bing doesn’t.

Both list a variety of major name-brand online retailers. Google doesn’t have Shoe Buy; Bing doesn’t have Foot Locker. Both have Zappos (which I’d have expected tops at both). Bing’s got some really specialized retailer that feels odd to reward. Then again, it also found what seems to be a gem of a site, one that lists new shoe release dates. Who knew!

I’d just barely hand this one to Bing, mainly because that marketing course, while relevant, does feel out of place to rank so highly. But on the whole, I don’t think you can look at this set of results and condemn Google’s entire algorithm change as flawed. As I said before, it’s easy to find outliers on any set of results with either Google or Bing.

Who Serves Results for “Autism Resources” Better?

Aunesty Reikofski commented about concerns she had with as search for “autism resources.” That resonated with me. Who wouldn’t want the best results coming up for a search on that?

Reikofski wrote:

Coming at this as a ‘mom’ and not an SEO professional….

Try searching for  “autism resources” #1 position is a hyphenated site that hasn’t been changed since 2006.  A total link directory, and all the national organizations rank below

Let’s do the side-by-side, Google on the left, Bing on the right. Bing has been spaced out to make comparing easier, but there’s no change in the order:

As it turns out, what Reikofski is most upset about, that top ranked site that’s apparently years out of date, it ranks exactly the same as Bing — the search engine she also commented that she would switch to. In fact, Bing lists two pages from the site, whereas Google only lists one. More correctly, Google give it one listing and some smaller “sitelinks,” but that doesn’t deprive some other page for the chance to be in the top results, as happens with Bing. Google also ranks the main national society, as best I can tell, higher than Bing does.

Postscript: Reikofski explains more about her reasoning for switching to Bing below in the comments; it’s less about Bing being more relevant than Google and more about Google being less relevant than she expects.

Google has one page that seems last updated in 2010, but Bing also has a page that seems outdated and kind of AdSense-heavy. Google lists two local organizations that might not be useful to many general searches; Bing lists one like this. Google has two decent-looking info sites that Bing lacks; Bing has a non-profit support group and a Wikipedia page that Google doesn’t have.

I don’t know that Google is really that much better or worse than Bing in this case.

For SEO, Google Edges Out Bing

How about an area that I’m a subject expert in, such as SEO? I’ve written before about how disappointing the results for that search can be, when it seems like getting a bunch of client links or inserting some links into a blog template could rank you well. Nor am I the only one. If that’s all it takes to rank for such a competitive area, those talking about doing white hat SEO best practices can be made to seem stupid by Google allowing this.

Let’s do the side-by-side, Google on the left, Bing on the right. Same as before, I’ve spaced things out on the Bing side, removed related links and a newsbox, so you can compare more easily. The order is untouched:

Several sites I’d consider offering good SEO information are listed in both places. For the first time, Search Engine Land finally even ranks for the term. I think we’re a good site, so that’s an improvement in my books.

I don’t know what moved out. I do know there’s still a lot of irrelevant garbage. An SEO firm near my home is listed. So what? Who buys SEO locally? I get some national SEO firm I’ve never heard of. Some Australian SEO firm that few, if anyone doing this search, is going to use. Another national firm that ranks primarily on the strength of its domain name — but hey, that happens for the same company at Bing, too.

I actually give Google the edge here, and not because we did better. It’s mainly because Bing lists some local SEO company from out of my area, as well as a “secrets” site with no information nor any active form to learn more. The latter is the type of thing some are mocking Google’s latest change as rewarding, as in the case of the “make money online” search. Bing deserves the same criticism, but being Bing, no one notices.

Being As “Bad” As Bing Isn’t Good

Of course, this isn’t really about whether Google is as good as Bing, or if Bing also has the same problems. I’m using Bing as a comparison because you need something to compare the Google results to, otherwise you lose all perspective. But ideally, you’d want the perfect set of results to compare with. Someone let me know where those are!

Really, this is about Google being as best as it says it wants to be. Some of the problems such as lacking the official site for Viagra? That’s pretty embarrassing. A site with no content ranking tops for “make money online?” Even if it happens to Bing, it still hits hard against Google that’s supposed by many to be the search leader.

Also remember that our results are increasingly personalized at both Google and Bing these days. The results above don’t show this, but when I’m logged in, some of the results can be radically different. Of course, some of the glaring problems pointed out above may still be reflected.

Some Perspective

I’ll conclude by trying to clear up some misconceptions and give a little more perspective.

As I said at the beginning, we really have no idea if the change has made Google’s results better or worse. There is no lack of people commenting on how terrible things are now, but like I said, that’s what always tends to dominate forums after any change.

Anyone who is an educated, veteran SEO in the space knows this. Susanna Miles, who commented on my original post, seems to be one of these people:

You know what I wish… I wish that people who weren’t hit or who saw positive change would post in these forums. It ends up looking really hopeless and horrible because it’s only the people who’ve been hurt that bother to do this.

Some people are happy, seeing a positive benefit. For anyone complaining, there’s someone who did better with no incentive to speak up.

I also saw this comment from Sean Paul that I thought was noteworthy:

As usual big corps wins, SMB’s lose out

Looking through the examples above, I don’t see that playing out. I see plenty of small sites doing perfectly fine, and I see some big sites who clearly aren’t. This update seems to have hit businesses of all types.

Over this past week, there’s also been a giant rise in the belief that “negative SEO” is somehow the lurking threat that people must run in panic over, and that this latest update makes it even worse. I’ll be coming back to a future article about that, but if you can’t wait, I’d suggest diving in first with this SEO Book post.

The short answer is this. Yes, there are things that people can do to hurt other people’s rankings, something Google has acknowledged for years, such as in this video last year. No, most people don’t have to worry about it, even with the unatural link warnings that went out recently. Again, I promise to come back to this more.

Recently, Rand Fiskin dared people to prove negative SEO doesn’t work by trying it against his personal website. Separately, someone did get his company web site at SEOmoz to rank for “snuggie dog bed.” I’ve seen this confused as proof negative SEO works.

No, the latter test proves you can still Googlebomb pages for a short period of time. We knew that. Or it proves you can redirect one site to another site to help it inherit rankings. We knew that, too. It doesn’t prove negative SEO can work, but we don’t need that proven. As I said, we already knew that could happen in some specific cases.

I think the most important thing for those who are worried about the latest changes to remember is this. Forget the rankings. Look at your traffic. Give it a day or two or three. Is your overall traffic from Google Search much better than it was before yesterday? You’ve probably gained. No change? The update had no net impact on you. A severe drop? Then yes, you got dinged.

What to do if you were dinged? I’ll be coming back with more about that, on whether a reconsideration request will really help or if there are other things you should try. Obviously, if you’ve been knowingly spamming Google, stop and try to correct that. If you haven’t, and you’re seeing really bad stuff showing up in the search results, it never hurts to do a good blog post illustrating why Google’s results aren’t helping users as well as they could.

That’s always the key. Google’s job is providing great results for searchers. Publishers complaining they’ve been hurt, that doesn’t carry much weight. Someone always does get hurt, as I said. Publishers showing that searchers aren’t getting the best stuff, that’s effective pushback.

I do suspect that Google’s twin goals of greatly increasing relevancy and not rewarding spam haven’t been met with this update. There’s just too much weirdness that I’ve been seeing, which does make me suspect that we’ll see the algorithm changed quickly again in the near future. But that’s really just speculation. I’ll try to get an official reaction from Google on this and other matters soon.

Related Articles

Related Topics: Channel: SEO | Features: Analysis | Google: Algorithm Updates | Google: Penguin Update | Google: SEO | Google: Web Search | Stats: Relevancy | Top News


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn


Get all the top search stories emailed daily!  


Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Matt McGee

    William, you have now posted this same exact comment about 5-6 times in the past 24 hours. Please stop. If you continue to repost the same comment on all of our articles, we’ll remove your ability to post here. Please be considerate of other readers and refrain from this kind of activity. Thanks.

  • http://twitter.com/Soccerselection Dogfox

    When tripod sites and yahoo voice sites rank in the top 10 for a specific search and youtube videos and spam sites make up the other 8, Google needs to take notice.  Searchers, webmasters and advertisers are being royally screwed and Google should be better at customer service.  Look after your customers and your customers will look after you.  Forget that basic rule and you lose a lot of friends, customers, webmasters and advertisers, your choice Google!

  • http://doityourselfcomputing.com/ Robbi Drake

    I don’t know that I agree with that comment @John. I do feel that more effort is in order and some level of change or amendment. I lost some ground on some sites, and I gained ground on others. In the beginning I was incensed, but after really thinking about it,  the bottom line is, who do I blame for that?

     At the end of the day, we learn from it or we whine about it. . Granted what was accepted–even encouraged practice a year ago is no longer so.but we learn from it and we move on. Google is fighting for their own life and their business and they have always been the most fickle of friends. I hope none of us is so naive as to believe that it’s not about making money. Improving the search results may be the stated goal, but the reason for that desired improvement is also making money.

     I  take exception to the hypocrisy of it–as I’ve said before, if building links and blog networks are wrong,and get you slapped,  then taking money to advertise those businesses is also wrong..(Google “build links” and see the paid adverts that are being accepted) if you slam it then you can’t really take money for promoting  it. It tends to confuse people. If you’re promoting it then why isn’t it accepted practice etc.

    I  feel that the quality of search in some areas has been strengthened and in some areas has fallen dismally.It’s a balance in my book and it will quite likely get better. I’m seeing a lot of improvements in just the last week
    Insofar as recovery,  and search engine placement, I think It would be the height of foolishness to say that quality content and solid holistic SEO are all that you need. We all know that it doesn’t work that way. I’ve written content online for more than 12 years and I can say definitively that .If you build it they will NOT come, it takes effort :) It takes traffic driving and search engine placement effort and it takes smart thinking and hard work.

    All that being said,  I think that we need to take another lesson away from this that many
    failed to take from Panda. Google does not OWE us traffic.  That’s
    something we’ve lost sight of  and need to take a hard look at. We seem
    to have this incredible “entitlement”  line of thinking about Google.. 
    We as SEO providers and as intelligent people need to take our traffic
    building efforts to other venues and stop putting all of our eggs in the
    Google basket. We secure multiple traffic streams  and stop thinking that because we built it and we seo’d
    it that Google owes us the traffic. They don’t.

    Sorry for writing a book guys.

  • John Ernest Guadalupe

    Do you have any info how Bing ranks its sites? It would be a great help to us who utilize SEO on their websites and blogs. It seems Bing will soon outperform Google in a number of years time.

  • Desperate_searcher

    AAAHHHH!!!  So THAT’S what’s gone wrong with Google Search!  The main, huge, horrible problem is that the search now includes everything on a web page, not just the text and headline of the story. Since many news sites have an index of news stories at the side of the page it means that if there is one story on the site that fits the search criteria, every single story on that site will be returned in the search results.  This is also affected by menus containing the search word and any ads that appear on the page.  Just try searching for “Countryside Alliance” on the Politics site – as the CA has an ad, every single story on that site appears.
    The second horrible problem is – only the last few hours’ worth of stories are returned (relevant or not), then there’s a huge gap between about 3 hours and 13 hours.  After about 13 hours, only a few results per hour appear.  
    Both these problems together mean that (a) you have to wade through masses and masses of junk and (b) if you want a comprehensive search, you have to repeat it every 3 hours otherwise you will miss stuff. 
    SOMEONE PLEASE TELL THEM!!!  I have complained every day since this started to happen and nothing has been done.

  • http://www.facebook.com/chrismarciante Chris Marciante

    THIS GOOGLE UPDATE IS HORRIBLE! I am a Network Consultant in-directly for Apple. I use google to answer little questions I have all day long about tech things. When I search Buffalo giga N450 the rankings are most populated with “BUY Buffalo giga N450, PURCHASE Buffalo giga N450, BUY ONLINE Buffalo giga N450″ etc… 

  • http://profile.yahoo.com/3LNPZVIK2C46GNBYPPNRM5AQP4 Patrick

    Search Engine dynamics keep on changing and within it nothing is perfect.
    Hence, there is no particular SEO technique that worked yesterday and will surely work today as well. SEO is kind of trial and error : sometimes a “shot in the dark”.
    This makes it all the more challenging and interesting.
    Patrick Carter from http://www.computergeek247.com

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest


Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States


Australia & China

Learn more about: SMX | MarTech

Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!



Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide