Worthy Alternatives To The Useless SEO Data Provided By Search Engines

The data classically used in search engine optimization (SEO) to benchmark sites and track performance is all but useless today. Ranking reports, still a necessary evil in SEO reporting, have been unreliable (at best) for at least 3 years. Their validity has been further marginalized over the last 24 months by way of Google’s increased use of personalization, localization and experimentation with blended results, to name just a few.

Today, the convention of using indexed page counts and backlinks to benchmark and report on site performance is facing a similar demise.

Google has always been careful at hiding data from prying eyes, including its supplemental index results (which Google claims do not exist anyway) and backlinks. The link: command contains purposefully obfuscated data. Even for sites that have been validated in Webmaster Tools, the link information we’re given has not evolved much. We get a very simple result set of sites linking to us, but we don’t get to slice, dice or order the data much further than that. It’s pretty much the same tool today that it was when it launched in October of 2007.

Index page counts returned using site: commands have been very inconsistent since at least November 2009, in my experience. Depending on how you search, or when you search, or where you search from, you’re likely to see very different results—sometimes varying by tens or even hundreds of thousands!

Yahoo gave us Site Explorer, the de facto backlink research tool in the SEO industry. Thanks, Yahoo, for that. But what’s happening with that tool? It’s been giving us very inconsistent, downright misleading data for the last few months. It’s no longer reliable.

Google is doing too good a job hiding the kinds of info we need as SEOs (supplemental index, indexed pages, backlinks) while Yahoo has given up completely and isn’t putting any resources into search. What we’re left with is a wasteland.

And the worst part? We keep right on doing what we’ve been doing. We really have no choice—we need data points like this. We’re basically stuck in a catch-22 of needing these metrics, while realizing they’re inconsistent and unreliable. That’s not a good place to be.

So just what is the enterprising enterprise SEO practitioner to do? There are options, but unless and until the search engines themselves give us valid and accurate data, these are workarounds and conduits rather than outright solutions.

What tools to use

To be fair, even with accurate data from the engines we’d still use our own tools and perform our own analyses, so really we’re back where we started from. There are some excellent tools we can use to track backlinks. Indexed pages, not so much, but there are workarounds for that, too.

There are three primary tools that every SEO worth her salt should be using (and probably already is). These are Linkscape,
MajesticSEO and Open Site Explorer (OSE).

Although SEOmoz has gotten a lot of flak for the technology behind its tool Linkscape, it’s actually an awesome piece of kit. (I wrote “kit” so I could sound like a cool Brit. Yes, I failed.) There are many different ways one can use Linkscape to analyze, dissect, reverse engineer and just basically own the link information of competing websites.

Majestic is another great tool and it really shines once a site is authenticated (for sites you don’t own, you can pay Majestic a fee to get the same data). However, I’m anxious to see how their data ages and the accuracy of its trending information.

Both of these tools do a great job analyzing backlink profiles and showing important metrics such as the domain and URL authority, the number of unique domains linking to a page or website, anchor text information and much more. They also make use of proprietary scoring methods: “ACRank” for Majestic and “mozRank” for Linkscape.

OSE is the newcomer, also from SEOmoz and it’s designed to replace the already dying Yahoo Site Explorer (thank god). It’s another good tool and is based on Linkscape.

While we can’t accurately track indexed pages outside of the search engines, we can use a tool like SEMrush to analyze overall keyword visibility in Google. No, this isn’t going to give us N number of URLs in Google’s index, but what it will do is surface a selection (the data can’t be definitive, nor does it need to in order to be actionable) of terms a domain is ranking for. Pretty darn useful, if you ask me.

The best part about all of these tools, including Google’s own Webmaster Tools, is that the data can be exported into Excel for further analysis. That’s where the magic happens.

What metrics to track

The primary metrics we track are the percentage of overall site traffic from free search, the percentage share of each engine, free search traffic at the keyword level (using clustering for related terms) and the delta between branded and non-branded free search traffic. Then, you can slice and dice as deeply as you need to go, looking at bounce rates, conversion data and more. Often, we do specific analyses at the category or product page level, too.

These are really the primary metrics we should track. You can go further, too, to track things like landing page yield and keyword reach, as illustrated by Brian Klais in the Analyze This column (which I highly recommend adding to your feed reader subscription list).

I really want to stop tracking indexed pages and backlinks using Google and Yahoo’s data. I need to stop. But until there’s a better way, until a reliable and accurate set of metrics and tools exists, this is the standard that we have. This is the best we can do. Sure, we can use our own tools like the excellent ones outlined here and perform our own analyses and we can ensure that cutting-edge and competitive SEO strategies are being formulated.

But C-level executives will continue to request things like ranking reports, indexed page counts, backlink counts (from the engines, not from a third-party tool) and even toolbar PageRank. The problem with these “metrics,” if you can even call them that, is they’re only useful in documenting what the engines are saying. They aren’t useful in documenting a site’s health, potential or competitive SEO stature.