Did Penguin Make Google’s Search Results Better Or Worse?

Google’s latest search algorithm change designed to fight spam and improve its search results went live yesterday. Plenty of people are seeing its impact already. Better or worse? It’s easy to find some examples of things being bad; it’s hard to say overall if there’s been a net improvement or not. Judging Relevancy Is Tricky […]

Chat with SearchBot

Google’s latest search algorithm change designed to fight spam and improve its search results went live yesterday. Plenty of people are seeing its impact already. Better or worse? It’s easy to find some examples of things being bad; it’s hard to say overall if there’s been a net improvement or not.

Judging Relevancy Is Tricky

I’ll go through some examples, but let me start with some caveats. If you dive into some forums, you’ll find plenty of people screaming. People tend to scream after any update that things have gotten worse because they’ve lost rankings. Few scream about how things have improved, much less provide examples. In short, relying just on forums can give you a skewed view.

That doesn’t mean you ignore what you might discover through forums, however. Few know the quality of any search engine’s search results as well as SEOs. They might not be happy if something outranks them, but good ones know if something better or worse has moved up. The same is true even for some of the “good” black hat people out there. They might deliberately violate Google’s guidelines, but that doesn’t mean they don’t know search quality.

Time for examples. To get these results, I was used “Incognito Mode” in Google Chrome to help eliminate personalization other than geographic targeting. That can’t be prevented.

Viagra: Missing Official Site & Many Other Problems

One of the first queries I did to check on the new algorithm’s impact was for “viagra,” because that search is a natural magnet for the type of spam that Google seeks to wipeout with this update. If that was the goal, Google had a big miss here:

Viagra1

Most glaringly, Google fails to list the official Viagra site in the top results, something that Bing gets right.

Postscript: John Andrews pointed out after this was posted that the official site has apparently been gone for 10 days or more, so this particular change seems unrelated to the new spam-fighting algorithm. I had asked Google about oddities with this particular search before posting my article but never heard back.

It’s also a good reminder to add that there’s no guarantee that any of the problems I’ll cover with these examples are due to the algorithm change. There’s no easy way to do a “before-and-after” for these searches to know what’s changed. However, you can look at the current state to assess whether they seem to be of good quality overall, which is part of what Google aims for with this current and all algorithm updates.

Almost as alarming, the four sites I’ve pointed at with red arrows have been hacked so that those visiting them get redirected to some place selling Viagra online. Three of these, Google even knows they’ve been hacked, because it shows a warning to searchers. Why it allows things to rank with those type of warnings makes no sense.

The two blue arrows point to two university web sites that have nothing to do with Viagra but which rank. Chances are, they were also hacked recently and fixed their problems. But if so, they’re still not relevant for Viagra and shouldn’t rank.

Bing: Better For Viagra

Here’s Bing for viagra, for comparison’s sake:

Viagra Bing

As mentioned, Bing gets the official Viagra site listed tops as you’d expect good search results to do. Bing also lists several informational sites, such as Drugs.com, How Stuff Works and Wikipedia. But all of these are also listed by Google, so there’s no big difference there.

Where Bing really pulls ahead is that instead of all those hacked sites, it lists a lot of online stores and affiliate sites. That’s certainly better than rewarding companies that have hacked other web sites, so it had the edge over Google. But whether these are legit online stores that require prescriptions is another matter.

Postscript: As noted in the comments below, one of these sites Bing lists seems likely to have been hacked.

Make Money Online: Google Has Very Slight Edge

One query I’ve seen go around in discussions, and which was tipped to us through email and in a comment, is for “make money online.” Is Google falling down here?

Honestly, it’s pretty tough to know what should rank well for that type of query, given that it’s so broad and so heavily filled with various schemes that purport on how you can earn money online (the answer, of course, is to create an online sharing service that makes pictures look old and has no income, then sell for 1 billion).

Still, let’s make the attempt. In this case, I’m going to put Google and Bing side-by-side, Google on the left, Bing on the right. I’ve removed the related searches from Bing’s listings and slightly spaced out the listings to even them up with Google, but the order is exactly as originally appeared:

Make Money

The damning thing some point to with Google is a site on Blogger that has no content at all ranking tops. Yes, that’s terrible. It’s also exactly the same thing that happens on Bing, though Bing ranks it lower on the page (sometimes, Google also ranks it lower). Certainly it would be better for Google if its ranking algorithm wasn’t returning a useless site. But pick any search, on either search engine with any update you care to name. There’s always some weird outlier.

I’ve also seen some call out Google for having two older articles ranking. The same thing happens with Bing, for one of these. It’s easier to notice on Google because it lists dates for these articles. Bing has some outdated information but doesn’t show dates.

Google is heavier on listing blogs about making money, though many of these really just seem to be blogs about blogging. Bing is heavier on places that overtly pitch some “make money” type of courses. If I had to choose what’s better, I’d go with listing the blog sites over the course sites.

In the end, I’d give Google the slight edge here over Bing. None of the results are really that great, of course. But I guess that’s what I expect for that type of search, knowing the flood of “get rich online” content that’s out there.

Overdose Advice? Bing Beats Google

Another example from a comment on our original story was about “how many pills does it take to overdose,” where Google is taken to task for featuring Q&A sites, places like Yahoo Answers, where anyone rather than an actual expert can give advice.

Here’s the side-by-side, Google on the left, Bing on the right. Again, I spaced things out a bit, but the ranking order is the same:

PillsAs you can see, both Google and Bing favor Q&A sites. I’d agree, that’s not optimal for this type of search. You’d rather get information from respected medical web sites, and not information on how to commit an overdose but what to do if you suspect you’ve accidentally done that. Neither deliver.

One of the biggest reasons for this failure is probably that the medical web sites simple don’t have content written in this type of plain language. If there’s nothing but garbage out there, garbage is what you’re going to get back.

Still, Bing gets the edge because Google has three completely irrelevant answers coming up, pages with nothing to do with the search topic. It was also disappointing not to see Google kick in the special suicide prevention information that it does offer for some searches.

Payday Loans: A Toss-Up

Another example of Google failure that was suggested is for payday loans, with Check N Go — which I gather is a major known provider — not ranking in the top two pages of results. Let’s do the side-by-side. As I said before, I did some spacing to help make it easier to compare, but no order was changed:

Payday Loans

Bing detected my location as being in El Segundo, California — actually about 40 miles from where I live, plus it tossed in a bunch of San Diego listings. That’s a strike against it right off the mark, but I decided to manually set my location to Costa Mesa, California for both search engines, to better even the comparison.

Both lead off with the same major provider of PayDay One. Bing does have Check N Go, while Google lacks that. But Google has many local providers, places I could walk in to, displayed. Bing buries these plus gives me lots of irrelevant local providers, including one in the UK.

It’s so bad with Bing that I kind of want to call this test in Google’s favor. But not knowing this space well, I get the impression that Check N Go probably is a provider it should be listing. Plus, listing what appears to be an affiliate site that probably has fake user reviews also was a knock against Google. So, I left the two tied.

Treadmill Reviews: Bing Gets The Edge

Earlier, I said that SEOs are often subject experts in their own areas. Daniel Deceuster, who commented on the changes, seems to be an example of this. He runs a site about treadmill reviews. The comment he left was full of the type of detail that a non-expert might not pick-up:

I monitor these rankings everyday, have for over a year. They have never been as bad as they are. #4 is an article reviewing just 8 machines from 2008, none of which are available for purchase any longer. Good luck finding anything through #3. Then #5 and #6 both link to each other and are obvious members of a huge internal link scheme. #7 doesn’t even have any reviews, just “discount codes” that aren’t codes but affiliate links that don’t get any discounts. #9 is a massive chart of numbers and ratings of random machines, some of which are no longer available. No reviews and their link to more reviews returns a 404 error.

I decided to take a closer look myself. The side-by-side:

Treadmill Reviews

I’m not an expert on treadmill reviews, so there are things Deceuster will see that I’m probably missing. But some of the sites he dislikes that Google is listing? Bing’s listing them, too. He focuses on an outdated article. That is annoying, and it doesn’t happen in this case with Bing. But it might happen in other searches with Bing.

Overall, I’d say that Bing does seem to do a better job of getting more sites that seem to offer fairly nice reviews of treadmills to the top. The one I liked the most, with my non-expert eye, was the one that Bing listed tops and that Google didn’t have at all.

Google Sucks For New Shoes? Not So Fast….

Another popular query I’ve been seeing going around for Google’s results apparently having gotten worse is a search for new shoes. The side-by-side is below, Google on the left, Bing on the right. Related searches and the news box from Bing have been removed, but the ranking order is the same:

New Shoes

The criticism is really about the second listing, which looks like some odd simulation thing that is totally irrelevant. It is relevant. It’s a marketing course called “NewShoes.”

Is it the most relevant thing for Google to list? I’d say not. I’d say most people who are searching for “new shoes” probably aren’t thinking about that course but rather actual shoes. Then again, a lot of people might be searching for a song called “New Shoes,” especially given that the official video has over 5 million views on YouTube. Google lists this, as well as how to buy it on Amazon. Bing doesn’t.

Both list a variety of major name-brand online retailers. Google doesn’t have Shoe Buy; Bing doesn’t have Foot Locker. Both have Zappos (which I’d have expected tops at both). Bing’s got some really specialized retailer that feels odd to reward. Then again, it also found what seems to be a gem of a site, one that lists new shoe release dates. Who knew!

I’d just barely hand this one to Bing, mainly because that marketing course, while relevant, does feel out of place to rank so highly. But on the whole, I don’t think you can look at this set of results and condemn Google’s entire algorithm change as flawed. As I said before, it’s easy to find outliers on any set of results with either Google or Bing.

Who Serves Results for “Autism Resources” Better?

Aunesty Reikofski commented about concerns she had with as search for “autism resources.” That resonated with me. Who wouldn’t want the best results coming up for a search on that?

Reikofski wrote:

Coming at this as a ‘mom’ and not an SEO professional….

Try searching for  “autism resources” #1 position is a hyphenated site that hasn’t been changed since 2006.  A total link directory, and all the national organizations rank below

Let’s do the side-by-side, Google on the left, Bing on the right. Bing has been spaced out to make comparing easier, but there’s no change in the order:

Autism

As it turns out, what Reikofski is most upset about, that top ranked site that’s apparently years out of date, it ranks exactly the same as Bing — the search engine she also commented that she would switch to. In fact, Bing lists two pages from the site, whereas Google only lists one. More correctly, Google give it one listing and some smaller “sitelinks,” but that doesn’t deprive some other page for the chance to be in the top results, as happens with Bing. Google also ranks the main national society, as best I can tell, higher than Bing does.

Postscript: Reikofski explains more about her reasoning for switching to Bing below in the comments; it’s less about Bing being more relevant than Google and more about Google being less relevant than she expects.

Google has one page that seems last updated in 2010, but Bing also has a page that seems outdated and kind of AdSense-heavy. Google lists two local organizations that might not be useful to many general searches; Bing lists one like this. Google has two decent-looking info sites that Bing lacks; Bing has a non-profit support group and a Wikipedia page that Google doesn’t have.

I don’t know that Google is really that much better or worse than Bing in this case.

For SEO, Google Edges Out Bing

How about an area that I’m a subject expert in, such as SEO? I’ve written before about how disappointing the results for that search can be, when it seems like getting a bunch of client links or inserting some links into a blog template could rank you well. Nor am I the only one. If that’s all it takes to rank for such a competitive area, those talking about doing white hat SEO best practices can be made to seem stupid by Google allowing this.

Let’s do the side-by-side, Google on the left, Bing on the right. Same as before, I’ve spaced things out on the Bing side, removed related links and a newsbox, so you can compare more easily. The order is untouched:

Seo Rank

Several sites I’d consider offering good SEO information are listed in both places. For the first time, Search Engine Land finally even ranks for the term. I think we’re a good site, so that’s an improvement in my books.

I don’t know what moved out. I do know there’s still a lot of irrelevant garbage. An SEO firm near my home is listed. So what? Who buys SEO locally? I get some national SEO firm I’ve never heard of. Some Australian SEO firm that few, if anyone doing this search, is going to use. Another national firm that ranks primarily on the strength of its domain name — but hey, that happens for the same company at Bing, too.

I actually give Google the edge here, and not because we did better. It’s mainly because Bing lists some local SEO company from out of my area, as well as a “secrets” site with no information nor any active form to learn more. The latter is the type of thing some are mocking Google’s latest change as rewarding, as in the case of the “make money online” search. Bing deserves the same criticism, but being Bing, no one notices.

Being As “Bad” As Bing Isn’t Good

Of course, this isn’t really about whether Google is as good as Bing, or if Bing also has the same problems. I’m using Bing as a comparison because you need something to compare the Google results to, otherwise you lose all perspective. But ideally, you’d want the perfect set of results to compare with. Someone let me know where those are!

Really, this is about Google being as best as it says it wants to be. Some of the problems such as lacking the official site for Viagra? That’s pretty embarrassing. A site with no content ranking tops for “make money online?” Even if it happens to Bing, it still hits hard against Google that’s supposed by many to be the search leader.

Also remember that our results are increasingly personalized at both Google and Bing these days. The results above don’t show this, but when I’m logged in, some of the results can be radically different. Of course, some of the glaring problems pointed out above may still be reflected.

Some Perspective

I’ll conclude by trying to clear up some misconceptions and give a little more perspective.

As I said at the beginning, we really have no idea if the change has made Google’s results better or worse. There is no lack of people commenting on how terrible things are now, but like I said, that’s what always tends to dominate forums after any change.

Anyone who is an educated, veteran SEO in the space knows this. Susanna Miles, who commented on my original post, seems to be one of these people:

You know what I wish… I wish that people who weren’t hit or who saw positive change would post in these forums. It ends up looking really hopeless and horrible because it’s only the people who’ve been hurt that bother to do this.

Some people are happy, seeing a positive benefit. For anyone complaining, there’s someone who did better with no incentive to speak up.

I also saw this comment from Sean Paul that I thought was noteworthy:

As usual big corps wins, SMB’s lose out

Looking through the examples above, I don’t see that playing out. I see plenty of small sites doing perfectly fine, and I see some big sites who clearly aren’t. This update seems to have hit businesses of all types.

Over this past week, there’s also been a giant rise in the belief that “negative SEO” is somehow the lurking threat that people must run in panic over, and that this latest update makes it even worse. I’ll be coming back to a future article about that, but if you can’t wait, I’d suggest diving in first with this SEO Book post.

The short answer is this. Yes, there are things that people can do to hurt other people’s rankings, something Google has acknowledged for years, such as in this video last year. No, most people don’t have to worry about it, even with the unatural link warnings that went out recently. Again, I promise to come back to this more.

Recently, Rand Fiskin dared people to prove negative SEO doesn’t work by trying it against his personal website. Separately, someone did get his company web site at SEOmoz to rank for “snuggie dog bed.” I’ve seen this confused as proof negative SEO works.

No, the latter test proves you can still Googlebomb pages for a short period of time. We knew that. Or it proves you can redirect one site to another site to help it inherit rankings. We knew that, too. It doesn’t prove negative SEO can work, but we don’t need that proven. As I said, we already knew that could happen in some specific cases.

I think the most important thing for those who are worried about the latest changes to remember is this. Forget the rankings. Look at your traffic. Give it a day or two or three. Is your overall traffic from Google Search much better than it was before yesterday? You’ve probably gained. No change? The update had no net impact on you. A severe drop? Then yes, you got dinged.

What to do if you were dinged? I’ll be coming back with more about that, on whether a reconsideration request will really help or if there are other things you should try. Obviously, if you’ve been knowingly spamming Google, stop and try to correct that. If you haven’t, and you’re seeing really bad stuff showing up in the search results, it never hurts to do a good blog post illustrating why Google’s results aren’t helping users as well as they could.

That’s always the key. Google’s job is providing great results for searchers. Publishers complaining they’ve been hurt, that doesn’t carry much weight. Someone always does get hurt, as I said. Publishers showing that searchers aren’t getting the best stuff, that’s effective pushback.

I do suspect that Google’s twin goals of greatly increasing relevancy and not rewarding spam haven’t been met with this update. There’s just too much weirdness that I’ve been seeing, which does make me suspect that we’ll see the algorithm changed quickly again in the near future. But that’s really just speculation. I’ll try to get an official reaction from Google on this and other matters soon.

Related Articles


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.


About the author

Danny Sullivan
Contributor
Danny Sullivan was a journalist and analyst who covered the digital and search marketing space from 1996 through 2017. He was also a cofounder of Third Door Media, which publishes Search Engine Land and MarTech, and produces the SMX: Search Marketing Expo and MarTech events. He retired from journalism and Third Door Media in June 2017. You can learn more about him on his personal site & blog He can also be found on Facebook and Twitter.

Get the must-read newsletter for search marketers.