Optimizing A Large-Scale Web Site? Do The Two-Step

The need for keyword research is pretty much a given. But applying that research on a massive scale is where things usually break down. If you’re dealing with 100,000 web pages, you don’t have time to touch every page. How can one do this scalably? The answer: programmatic optimization.

In a previous article “Scalable On-Page SEO Strategies,” I discussed various approaches to optimizing large-scale web sites. In the article I alluded to creating “recipes” or formulas for optimized title tags, and allowing that automation to be overridden by manual optimization. Let’s look at this process in more detail, and applied not just to page titles but other HTML tags as well.

Programmatic formulas

To raise the content optimization of every page on the site to a 60-75% optimal level, first create formulas for the title tag, H1 heading, meta description and (optionally) the meta keywords that correspond to data fields in the database. Each template will have different data fields available to it, so each template needs its own formula. Here is a simple example from Amazon.com:

Subcategory Template: Business & Investing Books (http://www.amazon.com/Business-Investing-Books/b?ie=UTF8&node=3)

Title tag: {subcategory name} {category name} at Amazon.com

H1 Heading: {subcategory name} {category name}

Meta Description: Earth’s Biggest Selection of {subcategory name} {category name}. Amazon.com strives to be Earth’s most customer-centric company, where customers can find anything they might want to buy online at the lowest possible prices.

Meta Keywords: {subcategory name}, {category name}, amazon.com

Manual Review

Programmatic optimization is the first step to large-scale content optimization. But then a second pass is required.

For pages that drive high traffic and revenue (or that have the potential to be), manual content optimization should be done to target the more competitive keyword terms and phrases in your keyword list.

To make that keyword list actionable, you’ll need keyword to URL pairings. In other words, you’ll need to map your desired keywords to URLs on your site. It’s possible to do this algorithmically (automatically), though a solution that can do this well isn’t going to be cheap. Unless you have an enterprise-sized budget, probably you’ll be going with a manual approach.

Here is a manual approach that works well. Create a spreadsheet that contains every page you want to manually optimize. Paste in the keyword research you’ve already done, terms and popularity data. Then place one keyword next to every page — this is your primary keyword for the page, the keyword that you will target content optimization around to send the strongest signal to the search engines. Highest popularity keywords should be matched with the strongest pages, usually the pages at the highest levels of your site. If you still have keywords left, match the remaining keywords to pages as secondary keywords. These should support the keyword theme for the primary keyword. We refer to the primary keyword as the “declared search term.”

Once the keywords are mapped to the pages you plan to optimize manually, all that’s left is the writing. Easier said than done.

Long tail strategies

For this, consider a “Long Tail” SEO approach that we at Netconcepts call “thin slicing” — a term and concept popularized by Malcolm Gladwell in his best selling book Blink. Gladwell uses “thin slicing” in the context of “rapid cognition,” where one makes snap judgments in their field of expertise. Surprisingly, those snap judgments are often times more accurate than considered opinion, i.e. assessments that have been labored over. There’s one important caveat: the phenomenon only holds true for experts, not for amateurs.

Thin slicing in the context of SEO involves touching key elements such as title tags, keyword URLs, H1 headings, and meta descriptions across thousands of pages quickly, monitoring for impact, and refining based on those results. Don’t try to optimize everything on the page, or you’ll get bogged down and lower your productivity.

We believe that thin slicing is as effective for the SEO expert as it is for Gladwell’s art historian. This may sound heretical, but with a daunting number of pages to get through, deciding on synonyms, verb tenses and word order should rely more on your intuition and a less on your keyword research. Make quick decisions; don’t overthink it or overanalyze it. Logically, by agonizing over the very best string of words to use for a title or H1 and continuously consulting keyword research tools, you get caught in the weeds and thus it costs you opportunity. After all, it’s holding you up from getting to the remaining thousands.

The “How” for thin slicing depends very much on your web site’s infrastructure. For example, a WordPress based site can employ the “mass edit” admin capability of our free SEO Title Tag plugin. It supports optimizing title tags and URLs (post “slugs”, more accurately) across many posts and pages quickly, without having to go to each post’s Edit screen individually.

Then there’s the “bulk upload” approach, where you import an updated list of title tags (or H1s or whatever) into your website’s underlying database. You start with a database export in CSV (comma separated values) format of your current title tags — along with the corresponding item ID numbers for each record, of course. Load the CSV file into Microsoft Excel and do your title tag optimization in the spreadsheet. Then upload the optimized title tags back into the database. Note that if your database does not have a field for the title tag, you’ll have to create it and re-code your site to override the programmatic title with the contents of this new field when it is populated with data. Rather than having to maneuver through phpMyAdmin or rely on your database administrator, have a CSV file upload function built into the admin interface of your content management system (CMS). When we added this capability to our GravityStream proxy admin, our optimizers and those at our clients and partner resellers experienced a nice boost in productivity.

If you prefer a forms-based web interface over working in Excel, you could build a “mass edit” view into your content management system’s admin interface, similar to the “mass edit” interface in SEO Title Tag. One feature we found invaluable when using web forms for thin slicing was to make the number of rows displayed per page user-configurable. Some users will want to display hundreds of records per screen, others will want much fewer, as too big of a web page will cause their web browser to crash or time out.

Next time you feel like optimizing 100,000 pages, remember the Two-Step: programmatic optimization first, then manual content optimization second.