Enterprise SEO is all about the data. More accurately, it’s all about data storage.

If you can look back over a year, pull out different metrics and see which ones correlate to success, you’re one step closer to repeating that success. If you realize at the end of the year it would’ve been realllllly nice to know page speed numbers across the whole site, well, you’re sunk.

So why are so many ‘big players’ still using Excel as their main data storage tool?

Portent’s not exactly ‘enterprise’ scale, but we store a lot of information for a lot of different clients. Sometimes, we even need to retrieve it. So here’s the solution I’ve built, in broad terms.

• It cost a total of $5,000 to set up; that includes my time
• It costs $100/month to host
• I only touch it if I need the data
• For the basic stuff, it uses Excel as the front end

If you want me to get more detailed, I can write future columns on each individual component.

The Requirements

I built this tool myself. I know, I’m a CEO; I’m not supposed to do this kind of thing. But I’m also a geek, and this was a hell of a lot of fun. Here were the requirements I wrote down when I started:

• Can store and retrieve hundreds of millions of records, without causing server meltdowns
• Allows easy Excel import
• Built on technology I already know: Python, LINUX, MySQL and/or MongoDB
• Keep different clients’ data separate
• But allow global analysis across all clients, if desired
• Never make me cry

Thus armed, I got to work.

The Servers

If you care about this kind of stuff: I’m running this system on two Ubuntu LINUX servers. One server runs the Python code and MySQL database (which is quite small, see the next section). The other runs MongoDB. If we ever hit the point where we need more oomph, we’ll spread MongoDB across multiple servers. It’s good at that.

The Database(s)

I ended up using a hybrid: MySQL stores the usual client stuff, like account names, site addresses and such. This kind of dataset won’t grow too large, and one client will only have so many websites, and it benefits from using MySQL’s relational structure.

MongoDB stores things like daily/hourly Facebook data, Google Analytics exports and the kind of stuff that just keeps growing. Why? Because MongoDB is a ‘NoSQL’ database. It stores everything in a flattened list-type format. That makes it a bit speedier for straight-up requests like “Show me all of the visitors, by day, for www.mysite.com, since 2011.”

Here’s a really basic look at the structure:

That’s not a database layout. It just shows how the two databases ‘talk’ to each other: MySQL stores the information used to access and retrieve data. MongoDB actually stores the data itself. MongoDB actually has about 30 different collections (in NoSQL, they call ‘tables’ collections, instead, just to be different) at the moment.

The beauty of the NoSQL system is that we can add fields and collections as needed, without screwing up the previous data. I won’t even try to explain why – I’m too much of a noob. Just keep in mind that NoSQL = lots of flexibility. Which can be a good or bad thing.

The Code

Then came the hard part: I had to actually get the data from each source and into my database.

Working with some APIs is a cinch. For example, pulling in data from Twitter was relatively easy. Here’s a quick example in Python. It grabs my account information from Twitter and prints it:

import twitter import time wait_period = 3600

screen_name = ‘portentint’ # that’s me

# you’ll need to get these by creating a Twitter API account

CONSUMER_KEY = ” CONSUMER_SECRET = ” ACCESS_TOKEN = ” ACCESS_SECRET = ”

t = twitter.Api(consumer_key = CONSUMER_KEY, consumer_secret = CONSUMER_SECRET, access_token_key = ACCESS_TOKEN, access_token_secret = ACCESS_SECRET)

response = t.GetUser(screen_name) try: print response # response is in JSON. Since I’m using MongoDB, I can just dump it into the database

except twitter.TwitterError, e:

error = str(e)

if error == ‘Rate limit exceeded. Clients may not make more than 350 requests per hour.’:

print “Rate limit exceeded. Waiting for the next hour.”

time.sleep(wait_period) else: print str(e)

Code like this powers the entire tool. The Facebook code is more complicated. The Google Analytics code is more complicated still. But other tools, like SEMRush, were far easier. And, once they’re built, you set ’em to run and let them do their jobs.

The key is to build the data collection tools you can now, and start collecting that data now. Don’t wait until you have ‘every metric’ collected and stored, because there’s always a new metric, or a new API. You’ll never launch. Only used storage is useful.

Making It All Work

I thought about building a big, fancy-schmancy reporting tool, but realized I’ve already got one: Excel.

Microsoft Excel has a cool but almost undocumented tool called Web Query. With it, you can directly import a comma- or tab-delimited file into your spreadsheet.

1. Create a script that generates and prints a comma- or tab-delimited file with the data you want. Test it! Make sure it’s generating the output you need.
2. Create a text file. Use the template below as your guide.
3. In Excel, click Data > Get External Data > Run Saved Query.
4. Choose the file you created.

The template: WEB 1 [web address of script that generates your delimited file ]

Formatting=none PreFormattedTextToColumns=True ConsecutiveDelimitersAsOne=True SingleBlockTextImport=False DisableDateRecognition=False DisableRedirections=False

After a minute or two, your data will appear in the spreadsheet. You can format it, generate graphs, etc., as needed. The best part is, you only have to do it once.

To update your data, you can open the Excel sheet and click Data > Refresh. It’ll update your sheet, and add any new rows.

Here’s a very simple dashboard I use for Portent:

Security

If you’re in an organization where security matters a lot, you don’t want to be shooting delimited files around the Web willy-nilly. Someone in IT will definitely come knocking.

At Portent, we generate random keys that have to be included in the Web query template. The server checks for the result and makes sure it matches our key generator. If it doesn’t, you can’t grab any data.

That’s pretty basic — you can get fancier. But, it prevents any accidental data dumps.

Other Stuff To Think About

Other things we’ve tested with this kind of quick-and-dirty data warehouse include:

• Grabbing catalog sales data for cross-channel analysis
• Scaling to a lot more records (billions) – it gets ugly
• Grabbing seemingly random data, like weather, to check for sales drivers
• Importing SalesForce leads information

The possibilities are endless. The point is to start now.

What You Don’t See Can Make You Awesome

The dashboard is pretty. But, the real power in this approach is that we have a lot of data at our fingertips for comparison. With a few custom queries, we can compare things like:

• Sentiment and use of images in Facebook posts
• Organic visits and Twitter followers
• Instances of duplicate content and organic visits
• Instances of duplicate content and site performance
• Adwords quality score and bid cost

…

You get the idea.

Yes, noodling around with things like Python and LINUX can be a bit spooky. Do it anyway.

A few days’ of effort gives us access to piles of great data, for years. While the dashboard is cool, the ability to research and measure across lots of different channels is better. And, never having to tell your boss, “We weren’t recording that data,” is priceless.

Me and the many voices in my head have attempted to define Enterprise SEO many times. Is it about big sites? Internal politics? Higher likelihood of failure? IT teams and branding guidelines?

Nope. Apparently, enterprise SEO is something we say when we’re clueless. Say I’m on a call with a big development team and I make ridiculous request like, ‘Can we link to the home page at www.domain.com?’ Suddenly, everyone starts throwing enterprise SEO around like poo at a monkey party.

I don’t understand it. Sob.

I do understand a few of the total whopper-BS-bombs lobbed around the subject, though. I’ve tried to explain where they come from, and how to stamp them out:

1.  We’ll Put Links In The Footer, For The Crawlers

…or small-print text 800 pixels below the fold, as spider food.

When you see something like this:

Or this:

What words pop into your mind?

Are any of them valuable or useful or well-written? No? Then it’s not going to work as an SEO tactic, no matter how big your site. Slapping links in the footer or throwing poorly-written drivel at the bottom of the page may briefly help. At some point, though, one algorithm update or another will come up and bite your site squarely on the buttocks. If Panda and Penguin didn’t teach you that, well, best of luck.

Why, oh why? Some design teams love this tactic because it lets them keep their pages purty above the fold. They think words are fatal to design.

Instead: Consider writing real, useful text that folks want to read. Try creating navigation that makes sense to visitors. The truth is, words and design equal great architecture. That’s why so many of the very best sites on the Web blend them so well.

2.  We Can Handle SEO After QA

Yes. Yes, you can. In fact, I welcome this approach: it gets me clients who are panicked and ready to do anything to fix their plunging rankings, which means I can sign them that much faster.

I’m kidding. Mostly.

SEO campaigns strengthen three properties of websites: Visibility. Significance. Authority.

All three depend on smart development from the very bottom of the technology stack. Visibility depends almost entirely on sound coding and canonicalization practices from day one. None can be stapled to the website on launch day.

Why, oh why? Managers ask the development team, “Do you know SEO?” The development team hears, “Do you know how to put meta tags on webpages?” and says “Yes.” Chaos ensues.

Instead: Think of SEO as a set of best practices that start with basic server configuration and end with ongoing content strategy. Everything in between matters.

3.  SEO Is At Odds With Our Brand

Absolutely not. The overall experience customers have on your site helps define your online brand. All smart SEO enhances that experience by making a site clearer, faster, easier to read, easier to use and yes, easier to find.

Why, oh why? To most marketing managers, Enterprise SEO means slapping text on pages (see Whopper 1) or changes you have to make after launch (see Whopper 2). Neither creates the greatest brand experience: You’re either duck-taping lousy writing to an existing design, or delaying other site work for SEO modifications.

Instead: Consider search engines one of the first potential brand touchpoints. Learn what customers care about online (speed, clarity, relevance, transparency). Build accordingly.

4.  We Can Put The Blog On A Subdomain. It’s Fine.

No, it is not. Subdomains siphon authority away from primary domains. And no quantity of uncertain quotes you find and take out of context will change that.

Search engines treat subdomains and subfolders differently. You might, if you’re really lucky, dodge this bullet. But it’s damned unlikely.

Want proof? Take one category of your site — preferably one that ranks well — and move it to a subdomain. Watch what happens. Your rankings will spiral the drain.

Or, take a safer approach. Look at how Google advised sites penalized under Panda to get back into the rankings. One of the top recommendations: move the ‘low quality’ content off the site, to a separate subdomain. Why? Because that moved that content off the main site. If a subdomain is off the main site, will links pointing at the subdomain help the primary domain, exactly as if those links were pointed at that primary domain?

Or, listen to folks like Rand Fishkin, who know a thing or two about this: “Subdomains sometimes inherit and pass link/trust/quality/ranking metrics between one another. Subfolders always inherit and pass link/trust/quality/ranking metrics across the same subdomain.”

Why, oh why? Most enterprise e-commerce systems come in black boxes. Adding blogs to these sites is next to impossible. So, a subdomain was the easy way to get the SEO team to shut up.

Instead: Pick a decent technology stack that’ll let you add a blog. Or learn to use reverse proxies. Or simply accept that in this one area, your site will be a bit behind.

5.  Our Application Server Handles Broken Links Just Fine

Note: I added this one. It’s hardly strategic, but man, it’s fun.

Someone at Microsoft has a violent allergy to standards of any kind. The best example ever is the way .net-based websites handle broken links. Instead of delivering a nice, normal ’404′ response code — that would tell a browser or search bot that the link’s busted — .net returns a ‘200’ or ‘302’ code, depending on just how deranged the developer was at the time.

But it’s not just them. Lots of Java-based servers do the same thing. I have no idea why. The results, though, are hilarious. Target.com, for example, dominates the rankings for its own error message:

High-five, guys!!!

Why, oh why? The truth is, any rational person would assume a Web server comes configured to handle 404 errors. But that assumes that a rational person designed the Web server. Mistake.

Instead: Assume nothing. This is the company that brought us the Zune, after all.

No One Likes The Answers

The truth is, everyone knows better. None of these gigantic SEO lies pass the common-sense sniff test. But the answers require change, rather than the status quo. They require a lot of activism within marketing and development departments, not diplomacy. And they require real effort by key resources, not farmed-out Fiverr campaigns.

Somewhere, you’ve got a competitor who accepts these answers, even if they don’t like them. You need to get to that place, too, or they’re going to eat your market share, and your business, for lunch.

I promise to be on better behavior for my next post.

I quickly went through denial/anger/bargaining/acceptance:

1. I’m sure they’ll find another way to get this data; this won’t affect anyone else
2. What the #$)(*!*) people?!!!
3. Oh please, let SEOMOZ keep their ranking data, please please
4. Well, crap

Then I revisited my old, familiar place:

I hate the SERPs.

Yep, it’s bizarre: I’m an SEO who hates the fact that we have to track rankings. I hate that I can triple someone’s revenue from organic search and still get a thumbs-down because they don’t rank #1 for [horse flies]. I resent the sheer brain-thumping hypocrisy of the SERP screen shots I put in my own proposals while saying, “Woo hoo! Look at the rankings we got for our other clients! because I know that’s what folks want to see.

I could just triple my meds and stumble along, but I decided to give all this a more practical look.

Don’t Stop; Steer…

For 15 years, we’ve been hurtling down the road at 125mph, chasing rankings. Deny if you like, but it’s the truth — as much as I hate rankings, I rarely tell a client to ignore them. Especially if they worked in my favor.

Stopping now would be like slamming on the brakes while still going that 125mph. You might survive (somehow), but you’ll definitely end up with an airbag in your face.

You can’t stop people from looking at rankings. Like I said, SERPs are totally addictive. And, they’re always there.

You can gently steer folks towards other metrics. Do it a little bit at a time, and you can change rankings from 95% of the discussion to 50% or less. Which is a nice start, if nothing else.

I’ve made this work more often than not with three talking points. I hit these points again and again, like a candidate at a debate (without the flag pin).

Talking Point 1: SERPs Are A Symptom, Not A KPI

I tell clients to watch the SERPs as one input in a larger picture: the SERPs on their own aren’t a performance indicator. But a sudden change in the SERPs can be a symptom of good or bad stuff happening in their larger marketing campaign.

In our weekly or monthly reports, instead of saying, you rank #2 for ‘rhubarb helper, I’ll say, we haven’t seen any changes in the rankings, or “your rankings are stable. If pressed, I’ll present the actual ranking data, but only if pressed.

The key to success here: you have to focus on changes and symptoms and avoid pointing out specific rankings, even when the news is good. It’s hard to resist crowing about all the new #1 positions you just grabbed, but you have to. Instead, point out the overall rise in traffic and conversions from organic search, and explain that a sharp rise in rankings explains some of it. Then, move on.

Again, clients will press you. And again, I say, steer, don’t stop. When a client presses you to present individual keyword rankings, go ahead, but stick to your talking points. Here’s the next one.

Talking Point 2: There’s More Data Than This

Never, ever present rankings on their own. Nor do I present detailed, keyword-by-keyword ranking data in the first two layers of a report.

Instead, I present aggregate ranking changes in subjective terms, like, you’ve seen a net improvement in your rankings in the last month. Then, the client can drill down to keyword groups and average rankings in those groups, and then to the detailed phrase-by-phrase data.

I’m not doing this to hide the data. Instead, I’m doing it to emphasize the more important information shown alongside the aggregate rankings information:

• Number of unique key phrases driving traffic
• Conversions from organic traffic
• Organic vs. other channels for time on site, conversions, etc.
• Site index health
• Spend versus value

This approach has helped me emphasize the true KPIs while still making the rankings data available.

Talking Point 3: Remember Your Real Goals

I’m always pointing out that you didn’t go into business to rank #1 for ‘turnip slicer.’ Obvious? Yes. Corny? A little.

Worth repeating several thousand times? Absolutely.

I’ll also put the true goal of the campaign — increase sales X%, build share-of-voice Y%, etc. — front-and-center in all reports and correspondence.

I’m steering, a little bit at a time, away from SERPs and toward the goals that matter.

SERP Addiction Isn’t Going Away

We’ve all had the rankings aren’t a good metric conversation before; but the truth is, as long as there are rankings, our bosses and clients will care about our position in them.

At first glance, SERPs are pretty simple. They’re impossible to ignore. And, they bait the C-level ego like a choice parking spot or a corner office. They are 100% instantly addictive. I know whereof I speak.

So, try as we might to discuss visitor retention, conversion rates and sales, we won’t have happy clients until we grab that top spot. I don’t like it. You probably don’t, either. But after years of talking about rankings, we can’t just slam on the brakes.

Instead, start making little changes, and steer folks towards more meaningful metrics. You’re less likely to need airbags that way.

But the truth is, in the SEO world, the large enterprise is in trouble. I know about brand favoritism. And I know large enterprises have massive resources at their disposal.

All marketers have if only moments: after a particularly grueling client meeting or a day of fighting one fire after another, we think ‘If only I was an in-house SEO at a big, big company.’

The problem? They can never seem to use those resources. And except for a tiny slice of world-leading brands, most enterprises can’t lean on their overall market visibility.

My List Of Warning Signs

So, here’s my list of reasons enterprise SEO is in trouble. This list is an ongoing project for me, and it’s not just the usual Lurie cynicism—I’m hoping to figure out solutions. If you have input, post it in the comments:

1. Data over-dependence. The 21st-century enterprise wants to be “100% data driven.” Problem is, you’re selling to human beings. So some chunk of your Internet marketing budget has to be spent on creative thinking. SEO is no exception.
2. Lack of content agility. Big organizations can’t seem to support fast-changing, fast-adapting content strategies. Have you tried to get a blog post reviewed, approved and published in less than two weeks? I’ve seen exceptions, but in most organizations, two weeks is wildly optimistic. That doesn’t kill content strategy, but it sure wounds it.
3. Slow development turnaround. I’ve already whined enough about development turnaround times. You get it.
4. Fiefdoms. Teams end up silo’ed. The marketing department breaks up into separate content, e-commerce and social media teams. Their paths never cross again. At that point, a synchronized SEO plan requires Kissinger-esque diplomacy.
5. Lack of patience. At the same time, C-level executives and vice presidents want to see fast results. They often want results faster than their small business counterparts. Pressure from the board and from bosses makes for more pressure and shorter honeymoons. So you’re hobbled by bureaucracy and have less time to get results.
6. Organizational clarity. Top company priorities aren’t always clear. That makes it very difficult to place SEO within the hierarchy of goals. Which makes marketing meetings look like an Orwellian barnyard.
7. Access to data. Sadly, I’ve seen big, Fortune 2000 organizations where the marketing team can’t get access to basic Web traffic data. I’ve seen many more where simple adjustments to goal-tracking take months. Combine that with data over-dependence and lack of patience, and you’ve got head-exploding contradictions.
8. Risk aversion. Got a great, edgy content idea? Good luck getting it approved. Enterprise managers are justifiably shy about pushing the envelope when it comes to the brand.
9. Team stability. Average tenure for a CMO is about seven months. Average tenure for an enterprise marketing team member? One year. That makes continuity difficult, and SEO is all about long-term planning.
10. The confusion magnifier. You’ve finally gotten permission to start a serious site-speed improvement campaign. Just as you’re about to adjourn the meeting, someone pipes up with “I heard we can just buy some links! Why don’t we just do that.” The larger the group, the greater the chance for chaos.

I could make a board game out of all this. But you get the point: enterprise-size organizations have some big Internet marketing and SEO disadvantages. It’s an uphill battle.

The Good News

With all this, I still love working on enterprise gigs. Big organizations have more resources, access to more content, more pages and bigger brands. Small changes can have a huge impact. And, now and then, you get to do stuff that makes a brand dominant in the SERPs.

And small companies have their own issues: resource constraints, wildly changing expectations and lack of long-range planning make them a whole other kind of challenge.

So, I’d never say “ignore enterprise clients.” Au contraire.

My real point: enterprise SEO is subject to the same challenges every other enterprise initiative faces. If you get a shot at a big-time gig, grab it with both hands. But, learn how to blaze a trail for your whole organization.

What To Do To Fix It

A few quick tips, learned from hard trial-and-error:

1. Above all, set reasonable expectations.
2. Don’t scream and yell. You’ll rarely get a good response by throwing a tantrum. Always be diplomatic, and chip away at issues a bit at a time. Keep a punching bag in your office for stress relief.
3. Build relationships before you need them. It’s hard to build a good relationship under duress. Get to know the development team. Schedule a monthly lunch with the creative group. Network. You might even enjoy yourself.
4. Educate like crazy. Whatever you have to do, get folks to attend regular education sessions. Use video, or an internal blog, or in-person training. Bribe them with food.

Sorry, there aren’t any easy answers. But, enterprise SEO can be a blast, if you understand the special requirements of big organizations.

Quality matters, and it matters even more on large sites, because you can repeat the same issues so many times. That drags down your entire site.

Here’s what I include in any large site quality audit:

1.  Check For Busted Links

Well, duh. Hopefully I don’t have to write about this.

Do make sure you’re checking both onsite and outbound links, though.

2.  Check For Duplicate Content

Again, duh. You can use Google to do this with a site: search, or use a tool like Screaming Frog. Distilled has a great tutorial on analyzing crawl data in Excel that gives you everything you need.

3.  Look For Typos

If you can scrape site pages or get an export of your site’s content, run them through a grammar/spelling error checker, like After The Deadline.

I don’t necessarily trust this kind of automated stuff, but no one’s going to proofread 4,000 pages about server software. Automated is better than nothing, and we know for certain Google quality raters look at this stuff. It makes sense that Google would do the same thing algorithmically.

4.  Check Reading Grade Level

I know: Grade level? WTH? Bear with me.

The reading grade level itself doesn’t matter. But huge fluctuations in grade level from one page to the next may hint at:

• Near-duplicate content: Many times, writers stuck with rewriting existing content raise/lower the grade level of the content.
• Writing quality problems: If one page is written at a graduate level, and the next is written at a 7th-grade level, someone may have caught thesaurus-itis or thrown in a lot of run-on sentences.

Again, if you can scrape site pages or get a data ‘dump’ of all content, you can run them through a grade level assessment. There are lots of code snippets out there that’ll let you do it, or you can learn to calculate Flesch-Kincaid readability on your own.

5.  Assess Purpose

One of the top issues Google calls out in their quality rater guidelines is ‘purpose.’ Pages with ‘clear purpose’ get higher ratings than pages with a lack of purpose.

That doesn’t mean you should have each page start with “The purpose of this page is…”

It does mean it every page should have a clear reason for existence. Which makes you groan, I best—exactly how are you going to check 150,000 pages for clarity of purpose?

1. First, do it in small chunks, and start with pages that are already generating organic traffic, but could rank higher for their terms.
2. Second, keep track of pages you’ve checked, so you don’t have to duplicate effort.
3. Third, use your crawler! Export title tags and headings. Pop them all into a spreadsheet and zip down the list. If the title/heading tells you exactly why you’d want to read that page, sweet! You’re all set. If the title/heading read like Shakespeare did when you were 13, you need to check that page.
4. Fourth, look at bounce rates. Pages with bounce rates far above your site average may be unclear.

6.  Look For Near-Duplicate Content

You know that duplicate content is bad. You know that the best solution is to fix it. What you may not know is that near-duplication is bad, too.

Please do not ask me ‘how much duplication is ok?’ It makes me want to use vise-grips on my own brain. Assume near-duplication is a scale, like this:

So more is worse. Less is better.

Find duped supplemental content

The easiest first step for reducing near-dupes: Remove supplemental content. Disclaimers, specifications shared by huge numbers of products and author biographies can all be linked instead of placed on every relevant page.

If you suspect a chunk of supplemental content is duplicated throughout your site:

1. Copy a sentence or three from that content.
2. Paste it into a text editor.
3. Put quotes around it.
4. Add site:www.yoursite.com[space] before the quoted phrase.
5. Paste the whole thing into Google.

The search result should show all pages on your site that have that same phrase. If dozens of pages share content, think about moving the shared stuff to a single, central location and linking instead.

Find rewrites

Ah, the good old days, when men were men and you could spin 250 words into 10 different articles, and Google would rate them all as unique. Them days are gone, sparky. Time to deal with it.

There aren’t any super-easy ways to find rewritten content, but you can automate it. At Portent, we try to save our sanity by:

1. Scraping large numbers of site pages.
2. Putting the content into a monster database.
3. Crunching the text, looking for actual duplicate phrases, sentences and paragraphs (that’s the easy stuff).
4. If it’s really dire, we’ll use a bit of natural language processing to look for structural and semantic matches: If five pages all talk about potatoes, then french fries, then catsup, and then how to mix them up, it’s worth checking (I’m blogging hungry, sorry).

This can be a lot of work. But we’ve seen improvement in indexation and content visibility with a determined effort at near-duplicate reduction.

Quality Matters

In the last year, I’ve reviewed and audited a few dozen sites of 10,000 or more pages. On some, recommendations led to higher rankings. On others, they didn’t. All were large sites with high visibility, few technical problems and no penalties.

The one difference between them: Content quality.

In every case, the sites had at least 2 of the following problems:

1. Thousands of pages of rewritten content;
2. Hundreds of pages of duplicate content;
3. Really bad writing;
4. Utterly purposeless writing, clearly keyword-stuffed and placed on pages for rankings.

Quality matters. Tackle it and you can shatter the ceiling on your organic search traffic.

We talk about rankings, but we know rankings mean diddley squat.

We point at ROI, but no one believes us.

We try to lobby for changes like text-based navigation, but we’re told “We can’t risk the brand!”

There’s a better way, particularly for any SEO geek in an enterprise: share of voice (SOV).

What’s Share Of Voice?

In classic marketing and PR, share of voice is the number of times your brand is mentioned in relevant conversations, divided by the total number of conversations on that topic. Measuring SOV back in the Dark Ages of 1993 meant a lot of guessing, and very little math. In some cases, I literally measured column inches in publications and then divided my clients’ column inches by the total. Seriously. I’m not making this up.

Why did folks demand SOV data? They’re not dumb. SOV is one of the only ways to measure performance across entire channels, or in cases where you need to track exposure.

For example: my client sells anvils and buys a sponsorship on a TV cartoon. She cares about sales, clearly, but also wants to see how having a coyote repeatedly smashed by those anvils will grow her brand. That could lead to sales later.

So, this isn’t a straight-up ROI measurement. Brand impact matters, too. The first datapoint we’ll have there is share of voice. I can compare total TV time dedicated to anvil-ertising to the total time she’s purchased:

Share of voice formula

Voila! Share of voice.

SOV + SEO = A++

If you bring share of voice into enterprise SEO, you get two huge benefits:

• You’re speaking your client’s language. If an enterprise marketer, or anyone working with enterprise marketers, doesn’t know what share of voice is, measure their office chair. You’ll take over soon enough.
• You’re connecting exposure to sales. You already track sales over time (I hope). Now, you can track exposure over time. That helps you spot trends and, if you collect enough data, connect the dots between SEO visibility and overall sales. You’re adding another attribution metric.

Great! Now, how the heck do you actually measure SOV?

Getting SOV Data

There are a lot of different formulas out there. Some SEO toolsets, like Sycara and Brightedge, have built-in SOV analysis. They create pretty graphs like this:

Sycara SOV graph

Which is great, and I strongly recommend using one of those tools. Because calculating SOV yourself will be painful at best. At its simplest, SOV is impressions received for a topic/total impressions for that topic. Sounds easy, until you consider:

1. This is about impressions, not clicks. That’s because I can’t easily track clicks across the entire channel. I know my click traffic based on a phrase. I don’t know my competitors’.
2. Search rankings change based on personalization, geography, and Google’s mood.
3. Different rankings receive a different share of impressions.
4. Share of impressions changes based on the topics.
5. Merely measuring a single phrase won’t work. You need to measure as many of the top phrases relating to a topic as possible. The more you track, the more accurate your SOV number.

If you want to see just how complicated this gets, check out Brightedge’s patent for SOV analysis.

My recommendation: let a company that’s already figured it out do the SOV calculation for you. It’s worth every penny

Using SOV Data

Enough complaining! Here’s how you can use SOV data. It’s pretty slick.

Say my client runs an anvil blog. I’m tracking, oh, 5,000 keywords, and that doesn’t even scrape the surface of their search profile. But, if I give them a weekly report for those 5,000 keywords, they’ll slap me.

I need an aggregate metric that I can track over time: SOV.

I take the 500 phrases relating to ‘anvil shows’ and put them into a single topic. Then, I track share of voice over time. I also track share of voice of their top two competitors. That way, I can measure overall progress, and do competitive analysis, with a single metric. And, when my client calls to say “Lurie, what the hell?!! We’re doing all this content and I can’t tell if that’s what’s driving sales or not!!,” I can at least provide some evidence:

Not my prettiest graph ever, but you get the idea.

So, even if last-click attribution isn’t capturing all the information, I can still shed some light. And, I can track overall performance in yet another way.

One Thing…

Keep in mind that in SEO, you figure out share of voice based on total queries (impressions), links and/or rankings. True SOV would measure exposure by tracking action across the entire channel:

• Clicks on each ranking, regardless of whose site is ranking, in your phrase group
• Links that actually generate traffic
• Time on site after each click, on each site

While we’re at it, I’d also like a tiny microphone implanted in everyone’s left ear, so that we can listen in on actual conversations. Equally possible.

We can only track so much. SOV based on search rankings is a big improvement over SOV based on column-inches. And it’s a great additional metric to have on hand.

I lack diplomatic-fu. And skills are always open to debate. But the tools are forever. This is a very incomplete list of the tools you probably want to look at if you’re responsible for SEO in an organization so big you get lost finding the restroom. No favorites—if it’s on this list it’ll do the job nicely:

Rank Tracking

For the record, I do not support rank tracking as a success metric. But it’s a fantastic diagnostic tool, so you gotta do it. My favorite tools are:

Advanced Web Ranking. Great since day 1. AWR is especially useful if you’re focused on a small set of sites, but want to drill really, really deep into the data. It comes in desktop and server flavors, lets you tune it down to the color assigned each keyword, and it’s dang speedy.

Authority Labs. Has an API and allows for massive tracking sets. It has had a few hiccups of late but continues to be pretty solid.

SEOMOZ Campaign Tracking. Lets you track up to 2,000 keywords, and of course dials into all the other nifty SEOMOZ tools. While it doesn’t have the customizable oomph of AWR (yet) SEOMOZ keeps building stuff out. Definitely worth a look.

Raven Tools. Also awesome, and pulls in all of Raven’s other cool tools. If you’re looking at SEOMOZ, look at Raven, too.

BrightEdge. Less well-known, Brightedge tracks lots of rankings, but also includes some other competitive stuff you’ll want to look at. Super-powerful and super-tunable. Setup lag is annoying (you have to wait a few days after submitting a keyword set) but if you want big-time share of voice analysis, it’s the winner. It’s also by far the most expensive of the list.

Share of voice analysis in Brightedge

Link Tracking

This is one of the few areas where I recommend having multiple tools. Link data differs from one tool to the next, and you need multiple data sources to get a good sampling.

Majestic SEO continues to be an industry standard, with a great API, and a huge dataset. Use it.

SEOMOZ is the other standard. Use it or you’re nuts.

ahrefs is the new kid on the block. They have some really cool stuff, like the ability to track new link acquisitions. If you’re paranoid about negative SEO, you want this.

The ahrefs new links report

Raven Tools will keep popping up on the list. Raven actually pulls data in from MajesticSEO, then ties it to their other management tools, so it can consolidate for you.

Statistical Analysis

If you think links and rankings are enough for you to do, keep your resume handy. You won’t last long in a serious enterprise. Industrial-strength number-crunching is a must if you want to try to correlate, say, your last content campaign to leads and sales.

Microsoft Excel is universally accepted and used. I’m betting you know the URL for it, too. I still use Excel for 99% of my analytical needs, and chances are it’ll handle it all for you, too. Until you’ve exploited all the tools it contains, don’t bother with the next two, because you won’t get much out of ‘em.

If you’re on Linux, consider Open Office, or R (below), or any number of other toolsets that get increasingly complex and powerful.

That said, R is an open-source, free statistical engine and programming language that will let you analyze huge datasets. Companies like Revolution Analytics and Statsoft build add-ons and front ends to make it a bit easier and scale it, too. There’s a pretty steep learning curve, though, so only pick up R if you know you’ll be doing some industrial-strength stuff.

SAS is the only other ‘pro’ tool I’ve used. It’s really, really slick, but again has quite the learning curve. It’s especially useful if you want to create interactive reports based on super-advanced statistical black belt-edness.

Data Visualization & Charting

This category includes structural stuff, like sitemaps, and data presentation.

First off, Excel, R and SAS will all do the job. I’ve rarely seen visualizations I couldn’t do in Excel.

Created in Excel. How much fancier do you really need?

But other tools will let you customize and prettify in ways you otherwise could not:

OmniGraffle is the standard for Mac charting. If you’re on OS X, get it. You can do all manner of relationship diagrams, including site maps, and import outlines or XML to automate a lot of the work. You can also create some pretty nice graphs, but I still prefer Excel.

Visio is the favorite on the PC. Same capabilities and caveats as OmniGraffle.

Gephi is a super-specialized beast. It lets you visualize relationships, like force-directed diagrams. Worth playing with if you’re charting out social media data, semantic relationships, or other stuff I can’t even think of.

Processing is a bit hard to explain. Think of it as a designer’s toolkit for data crunching. You’ll have to try it to understand, but if you need to present data, and you can’t find any other tool, Processing can probably do it.

Tableau is Windows-only (boo) but man, it’s awesome. It adds a level of polish that Excel cannot. If you’re on Windows, you should really take a look.

On Linux, I’d again look at OpenOffice first, as the simplest of the tools.

Analytics

I almost skipped analytics. There are plenty of articles about basic Web traffic analysis tools. But it never hurts to revisit the basics, so:

Google Analytics now has an enterprise or ‘premium’ version that lets you process more data, faster, provides more support, gives you a 99.9% data collection guarantee (still not enough, if you ask me) and provides advanced attribution modeling. Definitely worth a look.

Adobe Digital Marketing Suite (formerly Omniture) has quite a learning curve but also lets you do a lot with attribution, path analysis and all the basics. An established tool now owned by Adobe, however. Take that as a positive or negative, based on your own experience with the big A.

Coremetrics, now owned by IBM (see my note about Omniture, above), is often overlooked but I love their API. Some of their control panels feel like I’m stuck in 1997, but in e-commerce they’re really sharp, and often more accurate than other javascript-driven solutions.

These are web-based tools, so no major Linux/Mac/Windows issues.

Log File Analysis

You must analyze your site’s log files. Must, must, must. OK? Now that that’s settled, here are my favorite tools:

GREP is the easiest way to start. Don’t let the help page scare you. If you’re on Windows, install CYGWIN and go for it. On Linux or OS X, it’s already built in.

Splunk wins for the best name. It can also analyze everything from a server log file to randomly-sorted Nirvana lyrics. It. Can. Do. Anything. I love it. You can process up to 500mb of data per day for free. If you have more than that to process, it may give your CFO a heart attack.

Sawmill is the venerable standard. It’s been around a long time, has a fantastic list of plugins and is super-configurable. As an enterprise tool, it’s cheaper than Splunk, but not by much. A five-profile enterprise install will run you $1,750. Still, I love Sawmill. I’d be sad if I didn’t have it.

Monitoring Tools

You can ask your IT team to alert you when the robots.txt file changes, sure. But don’t stake your career on it. A few low-cost monitoring tools you can use to do the same thing:

Server Density tracks MD5 hashes of any file or even a chunk of any file on your site, so you’d know right away if something changed. Plus, you can track up- and response time.

Hey! Who changed our robots.txt? Report from Server Density

Pingdom does uptime and response time monitoring. I haven’t tried to use it to track file changes, though. Comment below if you have any experience with that.

Loggly is a cloud-based logging tool. They take a different approach, letting generate custom-ish log data and send it to Loggly’s storage. I haven’t used it enough to speak to stability, but it’s super-tweakable, which I love.

So Much More…

There’s a lot more, of course: competitive analysis, custom solutions you script yourself and who-knows-what-else. I’ll write about those in future columns, so if you have ideas, list ‘em below.

Happy toolkit-building!

It’s time for a career tune-up, everybody. As an SEO, you have a lot to offer in the field of enterprise search. Here’s how you can help, and how you can capitalize on internal needs.

Content Collection

Enterprise search systems collect information through content collection, aka content ingestion (which always sounds like a stomach problem to me). This can happen using:

1. A crawler/spider, just like the search engines we’re all used to;
2. A direct connection to a source control or document management system;
3. A host of other Rube Goldberg contraptions I won’t even try to describe.

If your company’s search tool uses a crawler, a useful system requires content visibility. You probably know a thing or two about that, right?

If it uses a direct connection, the biggest issue is content preparation. See the next section.

If your company uses some bizarre technology originally coded on punch cards and later updated using a combination of Fortran and COBOL, it might be time to consider something less… steam-powered. If that’s the case, you can offer a lot regarding current tools and algorithms, and help evaluate replacements.

Content Preparation

Most internal search tools rely on a mix of metadata and raw text indexing. That means everyone providing content to the tool must follow best practices. Otherwise, the search engine can’t accurately index and rank what it finds.

Someone has to go out and teach those best practices: How to write a good document title, a decent summary, and otherwise prep content for indexing.

That’s one more place you can help. A lot. In 20+ years’ work on various intranets, document libraries and filing systems, I’ve seen, oh, let’s say zero organizations that teach that kind of content prep.

Testing & Analytics

Every enterprise search tool includes reporting. You can use this reporting to see which documents get found most often for what queries. You can also use it to test different search results layouts, delivery methods and tweaks to indexing.

The older your internal search tool, the more likely it is that everyone at your company has forgotten where this data lives. Prepare to do a little research. But there won’t be many people better qualified than you to review this search data and draw conclusions about search quality.

Check out query reports and learn what folks in your company most want to find. Find the content that’d best answer their questions. Then, if they queries and the results don’t match (they probably won’t) figure out how to get your search tools in sync.

Getting Started: Don’t Be A Jerk

Some large enterprises have entire teams of information scientists working on internal search tools. Many more buy a search appliance, plug it in, and then promptly forget about it.

Don’t assume one or the other—ask nicely, first. Remember how you feel when a client brings in another SEO who treats you like an idiot? Remember how much you didn’t want to cooperate with them?

You want to avoid that response with the current internal search team. Especially since they’ve been doing this a lot longer than you have, and likely know a lot more than you do about search.

Talk to your boss, first. Start with a small idea—something where you can help improve internal search, just a little. If that goes well, you can do more, later. Don’t be pushy.

I just advised you “Don’t be pushy”. Me. The guy once labeled ‘Most likely to start a fist fight in a meditation session.’ Both my cats just snickered at me. So as always, take my advice with a grain of salt.

Prepare For Much Venting

My company is all of 35 people. We’re not an enterprise. But if I ask each person, I’ll get 35 reasons why our internal search tools suck. If and when you start researching current internal search problems, you’ll hear a lot about what stinks, and not much about what works.

Listen and record, but don’t necessarily throw out the current system. No one ever thinks an enterprise search system is good—at best, you can hope they hate it only mildly.

Huh, that sounds a lot like being an SEO. See? You’re more qualified than you think!

1. Angling for a raise.
2. Selling me their SEO widget.
3. Trying to re-define their now-banned content spinning service.

Those aren’t good criteria. So, I’m tackling the question: What is enterprise SEO? Are you an enterprise SEO professional?

Here’s a test. Check off each item, adding up points as you go. Then see the scoring at the bottom:

1.  You Don’t Control Site Changes

Your site recommendations shuttle between at least 3 e-mail inboxes before implementation. You say a silent prayer every time you submit an SEO tweak. And your last three title tag changes came back rejected because the game of SEO telephone morphed ‘title tag’ into ‘tidal hag’.

If you read this and then nodded in agreement, +20 points.

2.  Content Changes Have To Go Through A Branding Team

Most of the time, branding teams make great SEO partners. They know what makes folks buy. They research little turns and twists of phrase that can turn a one-time customer into a loyal community member. And they’ve probably fought a lot of the battles over wording and content that you’ll face.

But now and then, they run amok. Ever heard something like this?: Never mind what folks are buying. We make bicycle stabilization and traction surfaces, dammit, not tires.

Nurture your relationship with the branding team, or you’ll live to regret it.

If you report to a branding team, +10 points.

If you take the branding team to lunch once a month, +5 point diplomacy bonus.

3.  The Legal Department Is Watching

This is a uniquely American issue, I suspect. Americans are sue-happy. Presidential candidates might start suggesting National Lawsuit Month as an economic stimulus strategy.

OK, that last bit isn’t true. But in the USA, companies get sued. A lot. So they hire lawyers. A lot.

If your content, social media and linking strategies are subjected to legal scrutiny, +10 points.

4.  You’re A Diplomat

Here’s a little role play:

You: We should change our home page title tag to start with ‘Bicycle tires’.
Other: No can do, sorry.

You:  Why?
Other:  Our CEO likes to see the company name first.

You: But our CEO also likes profits, right?
Other: Yes, but this won’t help profits.

You:  But… we rank #2 for ‘bicycle tires’. If we change the title tag, we might move up. The extra traffic…
Other: Sorry, but you can’t prove that. Until you can, we can’t make the change.

How would you handle this?

• Scream, grab a beer from the fridge, pop the emergency slide and leave. -10 points
• Nod quietly, while imagining how it’d feel to shove someone out a 10th story window. -5 points
• Laugh, thinking the person is kidding (I did this once). +0 points
• Explain, calmly, why the title tag change would help. +5 points
• Write a detailed 2-page explanation of clickthru rates vs. ranking, the title tag’s importance to ranking, and the potential earnings impact. +10 points
• Produce the above report, since you knew this would happen and already wrote it. +20 points

Note: Do not come to me for advice on this topic. On a diplomacy scale, I rank somewhere between a manic wolverine and Joe Biden.

5.  You’ve Got Complex Infrastructure

If your site uses a content management system (CMS), +5 points.

If your site uses two content management systems, +10 points.

If your site uses:

• Multiple CMS’s;
• An e-commerce platform;
• A discussion forum;
• Three different blogging packages set up by the last three CMOs (all with an average 2-month tenure); and
• An undocumented custom store location tool that was written in PERL 1 in 2000 and crashes if a store address has a space in it.

+30 points.

6.  There’s Content Everywhere

Enterprise SEO isn’t just about challenges. There are some huge advantages, too. Big companies spawn heaping mounds of content:

• Sales training;
• Product documentation;
• Technical specifications;
• Knowledge bases;
• Marketing copy; and
• All other content imaginable.

Harvest all that stuff and you’ve got an SEO powerhouse.

If you’ve got more content resources than hours in a year, +20 points.

7.  Size Doesn’t Matter (As Much As You Think)

Note what I left out of this test: Site size. Here’s why: I’ve seen plenty of 2-person companies with 100,000+ URL websites and none of the challenges/advantages I listed above. And, I’ve worked with many corporations that had anemic 50-page websites.

Enterprise SEO isn’t so much about big sites as it’s about big teams, big resources and complex challenges.

If you already knew this, +100 points. You’re an enterprise SEO.

Score Yourself!

Check my handy-dandy scoring matrix:

• 0-30 points: No enterprise SEO for you!
• 31-60 points: In the ballpark. Chances are, you’ve started the enterprise thing.
• 61-100 points: You’re a battle-scarred enterprise SEO veteran.
• 100+ points: Boba Fett gives you a respectful nod. You are an enterprise SEO machine.

Where This Post Came From (I Go Edgar Allen Poe)

Once a month, I write this column. My topic selection goes something like this:

• Have I written about this SEO challenge/problem/bugaboo before?
• Is it unique to large organizations?
• Does it make me grind my teeth down to small, flat, domino-shaped nubs?

Then I type furiously until done.

But every time, there’s a teeny voice in the back of my head. And it keeps getting louder, saying:

“Why is this ‘enterprise’ SEO?”

It’s my own personal Telltale Heart. Louder and louder, until finally, this week, as I sat and stared at my keyboard, I shrieked, “Enough already! I’ll tackle the question!”

And so, I wrote this list.

Leave your comments below, as always.

Your website.

I understand SEO’s keyword obsession. It’s hard to let go, and a nice, high ranking for a really juicy phrase tends to justify budget. But, as I’ve written before, there’s more to justifying enterprise SEO than keywords alone.

In fact, in enterprise SEO, I’d say keywords should be the last thing you look at. You can get your biggest, best wins with a a general focus on site visibility, content clarity and site performance. Here’s why:

Someone’s Already Picked Your Keywords

Most enterprises have been around a while. If you fit that description, you already have a brand. Your audience already uses certain non-branded phrases to find you. And there are hundreds or thousands more long-tail terms that they’re using.

But you’re not getting any traffic from those terms, because you don’t appear. See “You’ve got bigger problems” below.

You’ve Got Bigger Problems

Show me a website larger than 10,000 pages, and I’ll show you a site with:

• Massive duplication
• Thousands of un-crawled pages
• Poor page load times
• Improper response code configuration
• Broken external and internal links
• Utterly nondescript, probably duplicate title tags on at least 10% of all pages
• Content slightly less compelling than Ishtar (the movie, not the deity)

So far this year, we’ve crawled and tested 10-15 sites I’d qualify as ‘enterprise’. Every one of them had the problems above. Address any three of them, and you build organic traffic.

If Sears wants to rank for dozens of 'automotive' phrases, they should fix this 302.

The bigger the site, the more easy wins you’ve got. Go after those first — each easy win boosts every page’s SEO-readiness just a bit. Multiply that by every page on your site = Big Wins.

Keyword Selection In A Large Organization = Herding Angry Cats

Have you ever tried to get 5+ product teams to agree on a keyword set?

If you want a preview:

1. Get four grumpy, hungry alley cats.
2. Hold them in opposite, equidistant corners of the room.
3. Put a nice big chunk of tuna in the middle.
4. Let go of the cats.

When you send out the fateful “We’re starting an SEO campaign. Please send keywords.” e-mail, it’s like that, only uglier.

Everyone wants their ‘share’ of the keyword ‘pie’. Product managers who have poo-poo’ed the idea of SEO for years suddenly buy you lunch. Sales teams start calling you complaining they lost a sale because the company isn’t #1 for ‘Failover hardware with bright green enclosures.’

As difficult as it may be to convince your VP that yes, shaving 1 second off site-wide page load times is worth it, getting everyone in a large organization to agree on a keyword set is even harder.

Don’t say I didn’t warn ya.

Keywords Are Getting Harder To Track

With Google’s lovely not provided rollout, it got a whole lot harder to track keyword-specific traffic.

You can track:

• Leads from organic search
• Overall traffic from organic search
• SEO’s contribution to the bottom line

And those metrics are what really matter.

It’s About Business Objectives, Not keywords

Most important: Enterprise SEO is not about a few fat head keywords. It’s about visibility and discoverability in the long tail. That’s the best path to a natural rise in the rankings for relevant ‘head’ terms, anyway.

Your efforts help a page on your site rank for ‘legacy COBOL integration’. But they do more than that: They help you optimize for ‘COBOL integration’ and ‘integration’ at the same time. You get the early long-tail gains while building for more competitive terms later on. Trying to optimize your homepage for ‘integration’ is like emptying the Pacific Ocean with a thimble. You won’t make much progress.

Enterprise SEO is all about business objectives. I know what you’re thinking: All SEO is about business objectives. True. But in enterprise SEO a focus on visibility, clarity and site performance achieves those objectives far more easily than chasing a keyword list.

So please, do your brand a favor: Stop worrying out keywords. Focus on business objectives, instead.