If you’ve been following along in this series, we covered the first steps in the previous two articles.

1. Choose The Keywords To Focus On

In Organic Keywords: The First Step In Search Engine Optimization, I covered how to use Google analytics to choose the organic keyword phrases to focus on first in your optimization efforts; those keywords already contributing to the business goals. We then looked at how to use Google analytics to help “map” those keywords to existing pages on the site.

Below is a copy of the prioritized keyword mapping we developed for a client in one of the previous steps. I’ll refer to this later.

Prioritized keyword mapping

2. Prioritize Your Keywords

In How To Prioritize Keywords For Optimization Based On Organic Competition,I covered how we prioritize the keywords by evaluating the difficulty of reaching top organic results.

3. Check That Important Content On The Page Is Getting Indexed

As I start optimizing a page, I check to make sure all the important content on the page is getting indexed by checking Google’s “text-only” version of the page’s cache, Bing’s cache, and/or using one or more page analyzers or crawlers.

Find the page in Google or Bing’s index with a “Site:domain.com keyword phrase” search. In Google, hover over the search listing then hover over the arrows that appear at the end of the listing to see the page snapshot. Then click on the “cached” link above the page snapshot.

View Google Page Cache

Finally, when the Cache version of the page appears, click on the “text-only version” link.

View Google’s Text Only Version of Page Cache

In Bing, you also hover over the listing, then hover of the arrow that appears. A window with info about the page will appear. Click on the “cached page” link.

By doing a “Site:domain.com keyword phrase” search, the keyword phrase will be highlighted if Bing can read it. You can also select sections of text on the page to see if the text is editable text that can be read by search engines or whether the text actually part of an image.

View Bing Page Cache

I put the cache version of the page side by side with the actual page and compare the two. I’m looking to make sure that all important content on the page is getting indexed.

I make note of any issues that we need to discuss with the web developers such as the use of techniques that are not indexed easily by search engines (Flash, content included by JavaScript calls etc). I also make note of text in images that we’d ideally like change to indexable text.

If employing more search engine friendly methods isn’t currently an option, I’ll make a note not to optimize that content since it isn’t being indexed anyway.

4. Ensure The Indexed Text Is Unique

Next, I make sure the content on the page I’m going to optimize is unique.

Copy a snippet of what appears to be unique text from the page that you are going to optimize. Then search for it in Google with parenthesizes around the text.

If you get multiple results for a search on unique text, examine the files to see if a large portion of the content is indeed duplicate or very similar.

If the pages are duplicates or very similar content located on multiple websites, this may be a case of multiple web sites using similar content (very common with affiliate or ecommerce sites that use content provided by manufacturers etc).

If you find this to be the case for a significant amount of the content on a page you are going to optimize you’ll need to decide whether to rewrite the content or attempt to include “enough” unique content on the page otherwise you’ll be competing with all those other sites for listings.

However, before you start making significant changes like this to a page see the note below About Potential Effects on the Conversion Rate.

There are other reasons you may find duplicate content on other websites such as the content having being scraped and used elsewhere which you’ll want to deal with (that could mean getting a lawyer involved).

You might also find duplicate paths (URL’s) to the same pages or other pages on the site with very similar content which is another issue that can affect rankings. See How To Improve Organic Search Results With A Simple Site Audit for more about duplicate paths.

A Note About Potential Effects on the Conversion Rate

Before we get started actually optimizing, I do want to point out that I’m very careful not to make extensive changes to content on a page that is already converting well.

If you’ve been following the steps outlined in this series of articles, then you are likely optimizing pages that have been generating business. Just about any change you make on a page can affect the user experience and have an effect, either positive or negative, on the conversion rate.

Many of the changes we make are fairly subtle changes to existing text and images that shouldn’t have a dramatic effect on the conversion rate.

However, if you are going to make fairly extensive changes such as adding new content, completely rewriting content etc then consider testing those changes first using either an a/b test within a PPC campaign or a multivariate testing system such as Google’s Website Optimizer to make sure you don’t significantly decrease the conversion rate.

5. Try To Improve The Search Listings For The Keywords

Search for each keyword phrase you are going to focus on and determine how the search engine is generating the search listing so that you can attempt to improve it if needed.

In this case, you’ll want to search for the phrase the same way most people search, that is without putting quotes around the phrase. So for one of the phrases in the keyword mapping example above, I’ll search for [chrome frame sliders] without putting quotes around the phrase.

In North America, you should check the listings for at least Google and Bing (The Yahoo listing will be very similar to the Bing listing).

Tip: Rather than searching through pages of listings to find a search result you can append the keyword phrase to a Site:DomainName search. It’s the same listing. I’ve checked this many times. A Google search for “site:domainname.com chrome frame sliders” for example returns the same listing as does a search for [chrome frame sliders].

Determine How The Search Engine Generated The Listing

 Next determine how the search engine generated the listing; by extracting text on the page, from the Meta Description Tag, some combination, etc.

Google may still display the listing from the Open Directory Project, but typically only for the home page. I believe Bing/Yahoo may still display the listing from the Yahoo Directory, although I haven’t seen this in some time. I also see Bing displaying listings from the Open Directory Project.

If the results being displayed from a directory are not ideal, you can try to update the directory listing (which can take quite a while with the Open Directory Project) or see this Google article for information about how to use a noodp Meta tag to tell search engines not to display the description from the Open Directory Project. See this article for info on how to use a noydir meta tag to stop Bing/Yahoo from displaying the listing from the Yahoo Directory.

Try to Influence the Search Listing

I list all the current search listings for the phrases I’ll be focusing on as well, as well as average rankings, CTR etc., as I optimize a page and try to improve the listing by modifying the text the search engine is pulling from on the page whether it’s text on the page or from the Meta Description Tag or some combination.

For example, a typical Google search listing for an important keyword phrase was being extracted from the main paragraph of text on a client’s home page.

The listing was not horrible (many are) but the CTR for this listing was low. I reworked the message on the home page to make it more client focused which lead to a much more enticing search listing.

6. Update Or Add A Headline

Page headlines are usually read by your human visitors so I try to make sure most pages I work on have one. If I can include one or two important keywords in it all the better. This not only should help rankings but also help assure site visitors who just searched on the keyword that they came to the right place.

7. Optimize Existing Text

Next I make suggestions to optimize the rest of the visible text on the page. These are usually very subtle changes as I try not to negatively impact the conversion rate. I look for ways to include variations of the  important keywords without forcing them into the text and making the messages awkward (Remember your human visitors come first).

Here’s an example of some existing test from the motorcycle frame sliders page:

“These were produced by Diamond Powersports and are made of high quality Delrin or Chromed Billet Aluminum and are designed to protect the motorcycle’s fairings and frame in the event of a fall over or other accident. The materials used maintain the color even if scratched or chipped. The color is solid throughout the part and is not a surface coating. These sliders come complete with superior grade metric bolts and high quality machined mounting brackets if needed for the make and model of your machine.”

And here’s how I’ll suggest they modify this text.

“These frame sliders were produced by Diamond Powersports and are made of high quality Delrin or Chromed Billet Aluminum and are designed to protect the motorcycle’s fairings and frame in the event of a fall over or other accident. The materials used maintain the color even if scratched or chipped. The color is solid throughout the part and is not a surface coating. These frame sliders come complete with superior grade metric bolts and high quality machined mounting brackets if needed for the make and model of your machine.”

I could have “forced” keyword phrases into this text even more but at the expense of readability. I then continue on and make subtle changes in the rest of the visible text on the page.

8. Look For Text In Images

There is often content on webpages that can’t be read by search engines including text within images. I’m working on a website now, for example, where the only visible text on the home page is actually within an image.

As search engines are extracting text to create search listings from the only text they can read, they are only grabbing a few news items that appear on a mouse over. This is leading to irrelevant search listings that won’t help entice people to click though to the site. Plus, we can’t optimize the text to improve rankings and listings.

So, I’ll be recommending they have a Web developer rebuild the image using text that the search engines can read. A skilled web developer should be able to rebuild most images so that text is search engine friendly using CSS, background images, etc. I’m amazed at how many clients use Web developers that have no clue how to do this. If search engine results are important to you (you’re reading SEL so I assume they are) then find an experience developer who can do this.

9. Possibly Add New Content On The Page

After editing the existing content, I may look for places to add new content on the page, especially if we need more content in order to focus on important keywords. Here are some ideas.

Left/Right Column Content

I look for places on the page to add content that can be both helpful to the user and may help improve search results and conversions.

For example, I might suggest some sales or positioning messages in the right of left columns. I’ll ask if the client has any testimonials and possibly insert snippets from one or an entire testimonial in right column, ideally on that includes one or more important keywords. If they have some useful articles they’ve written, I might suggest putting a summary in the right column that links to the full article.

Content Below The Fold

I sometimes develop a paragraph or more of text and suggest they add it below the fold. This is usually the case for the often image heavy homepages. In that homepage, I mentioned above the text contained in the image only talks about a couple of broad important keywords.

I’ll suggest they add some new text lower on the home page to talk about some important services they offer and ideally link to pages for more information about each service.

10. Develop Alt Tags for the Image Links

Alt Tags for static images (images that are not used as links) are used to provide information about what is in the image for those who mouse over the image, those with slow speed connections, those who turn off image downloading, or for the visually impaired who use page readers.

Alt Tags for images that are used as links should provide additional information about what can be expected when someone clicks on the image link.

If you can develop keyword rich Alt Tags for image links this can help improve the rankings for both the page where they are embedded in and the page they link to (see below for more on internal linking).

11. Update The HTML Page Title & Meta Description Tag

After I’ve developed all the suggestions for optimizing the content on the page, I work on the HTML Page Title and Meta Description Tag.

There are plenty of articles in Search Engine Land about crafting HTML Page Titles and Meta tags such as these:

• Writing HTML Title Tags For Humans, Google & Bing
• Google’s Tips On How To Write A Good Meta Description

I will make a few points about these tags.

Check for CMS Issues with Tag Handling

Before spending your time crafting beautiful tags check to see if there are any issues with how these tags are generated on your website. There are often limitations with CMS systems like WordPress, Joomla, Drupal, etc.

You may need to see if there are settings or plug-ins available to improve HTML Page title and description tag handling. Sometimes, you’ll need to develop your tags within limits constrained by the web development system.

Longer, Truncated Titles

I usually make titles longer than what the major search engines display, which is currently about 65 to 70 characters. However, I make sure the first 65 to 70 characters is crafted to do the best job of enticing people to click through from search listings.

I focus on 2-3 important keywords even if they are not all contained within the first 65 to 70 characters of the page title as the search engines likely index beyond what they display.

Title and Description Should Entice Searchers to Click

Remember that the purpose of Titles and Description tags is to tell people what the web site, site section/category, or the specific page is about. So craft Title and Description tags to work together to try to attract visitors scanning search results to click to your website.

Keywords in Both Title and Content

You’ll usually want to focus on 2-3 important keywords that are included in the visible, indexable content on the page within the Title. Again, the title and description tag should relate closely to the content on the page.

Plus, it will take the combination of the keyword in the page title and within the content on the page (along with other factors such as internal and external linking) to reach top rankings for competitive phrases.

12. Increase Internal / External Links  & Social Engagement

Internal links from within the Web and even more so external links from other sites affect rankings. So I look for how we might increase the internal links to the page such as adding it to one or more of the navigation systems if appropriate, adding it to the HTML site map etc.

More importantly, I look for ways to add links to the page from within the main body of content on other pages, ideally those that are ranking well for other competitive phrases. This should help rankings. However, again remember that your human visitors come first: these links should be added only if they make sense for moving humans along to the page

For the Frame Sliders page, for example, the current product line is a closeout so maybe we can put text or image links on the home page and some other pages such as the following:

Frame Sliders Clearance

Up to 50% Off

I then look for ways to get social mentions and more external links pointing to this page or to pages one click away such as mentioning the topic in a press release or article with a link to an inner page, talking about the topic and pointing to a page in some social venues, etc. Frame sliders at 50% off could be mentioned in the client’s social media venues for example.

Make Sure The Recommendations Are Implemented Correctly

Finally, if you are not the person who implements the suggestions you developed make a point of checking them as soon as possible after they have been completed. Most of our suggestions are implemented either by a client’s IT department or their Web development company and it’s very rare that the suggestions are implemented correctly the first time.

Dalai Lama (of keyword research?)

First, realize that the real Dalai Lama feels no need to prove himself, he just *is*. Project an aura of confidence, and realize the most elusive concept in our industry. It is not about you, it is about the customer.

Sidebar: Even if you’re an in-house marketer, you can still use these techniques. Pretend your VP of Product Development or someone similarly entrenched in the product/service is your “customer”.

Similarly, that first meeting should be all about the customer. This is your best chance to get an outsider’s perspective of how your customer views their products and what language they use to describe them.

After this first meeting, you’ll be an insider, and asking some of these questions will make it seem like you don’t know what you’re doing. So let your customer do most of the talking.

As you listen to the answers, jot down key phrases, jargon, and abbreviations they use to inform your keyword research later. Don’t forget to ask them to clarify anything you don’t understand.

Note that this is by no means an exhaustive list of questions you should be asking; merely a sample of questions for keyword research purposes.

Question 1: I’ve reviewed your website, and have learned about your business. However it always helps to hear you explain it in your own words. So, Mr. Customer, how would you describe what you do?

The answer to this is likely to be the same words you read on their website or see in a brochure. Point out any jargon that you don’t understand, as this will set the stage for later, when you tell them they need to change the way they describe their product.

Question 2: In your opinion, what is it that makes your product/service special? What differentiates you from your competitors?

These are their value propositions; the key elements that need to come across on their pages to compel a conversion. If one of them is that they offer the lowest cost, then you know to research keyword modifiers like [cheap], [low cost], [price]. Alternatively, if they’re not low cost, you know to avoid these keyword modifiers. More on this in my next article.

Question 3: What do you think are similar services/products that you do not consider competitors?

The keywords that come out in this answer will help you refine the research. Often, keywords that are very similar may have a completely different meaning in a particular clients’ industry.

For example, “phone lines” and “phone trunks” are very different and each appeal to a distinct target market. You’ll only want to explore the right one in your research.

Question 4: Which products/services are most profitable for you? Are there other reasons (inventory, seasonality, location) that you would want to push one product/service over another?

Again, the answer to this question will help focus your research. Spend the most time expanding and refining the products that the client indicates are most important. This can sometimes save you from exploring an entire product line, if the customer says something like, “Product A is a necessary evil. We have to carry it, but we also have to price it below cost.”

Obviously, that’s not an area you want to focus on. You’ll include some keywords to be thorough, but you’ll spend more of your time on the “money” keywords.

Question 5: What do you think are your top ten most important keywords?

Ask for ten keywords. The reason for this is that some customers think they need to rank for their entire keyword universe of 1000 × 10¹⁰⁰ keywords.

On the flip side, there are clients that think they only need to rank for one keyword and it will solve all their problems. Chances are that’s a virtually unattainable keyword like “tablet”. This question will help you determine which type your customer is, as well as let you know what keywords absolutely must be included in your final research.

Asking these five questions will complete a formidable amount of your keyword research before you even sit down at your computer. It will also help you focus priorities and set realistic expectations with the very first client meeting.

Next time: how to take what you’ve learned and conduct the research.

Photo © UW-Madison University Communications

In the world of Internet spam, scams, and shams, we’ve learned to be wary of what we find online.

Search engines are no different. Too often we forget that search engines aren’t just a tool to help us find news articles or guacamole recipes. Search engines are running a business—a business whose success relies on providing you legitimate, relevant results.

Fail to give their searchers what they want, and they’re fast on their way to joining Dogpile and Ask Jeeves in the search engine graveyard.

Naturally, Google, Bing, and Yahoo all want to ensure they’re only returning trustworthy and legitimate search results to their users. To ensure quality, the engines work to determine a site’s credibility.

Theoretically, the more you please and earn trust from your past visitors, the more likely you’ll please and earn trust from new visitors. But how do the searchbots measure an abstract human concept like credibility?

Factors To Improve Upon & Measure Your Site’s Credibility

The search engines evaluate a variety of factors when determining your site’s legitimacy. We’ve known about certain tried-and-true factors that can boost your site’s credibility for years, including:

• In-Links: The search engines see in-links (links leading to your website from external sites) as signs of your trustworthiness. In-links are the virtual equivalent of someone “vouching” for your site. The more successful and credible the other site, the better you look to Bing and Google.
• Out-links: Contrary to in-links, out-links are the links on your site that link to external sites. If your links are broken or the external site is irrelevant or outdated, your search rankings can take a hit.
• A clean, error-free site: Broken/missing images, spelling mistakes, or 404 errors are all poor signs of a site’s credibility.
• Traffic: Theoretically, the more traffic you get, the more relevant you are to searchers.
• An easy-to-navigate website: A site that’s easy to navigate can reassure new visitors that your site contains the information they need. If your site is difficult or confusing to navigate, your site’s bounce rate (the percentage of users who immediately leave your site) will be higher— indicating to the search engines that your site wasn’t a successful match for those search queries.
• An XML sitemap: Just like your site must be easy to navigate for humans, it should be similarly easy to navigate for the search spiders, too. An XML sitemap acts as a “road map” that leads bots down each interconnected page of your site, allowing them to index your page more quickly and accurately. Most websites have two sitemaps: a text-based list of pages within the sites and an .xml file for the spiders.

Social: The New Factor That’s Increasingly Affecting Your Credibility

For a long time, SEO and social were two entirely separate departments. A successful company had a social team and an SEO team and the two rarely mingled. It made sense: SEO was all about being found, and social media was about keeping those who’ve already found your brand. SEO set the bait and social kept ‘em on the hook.

Today, SEO and social are inseparable. Customers can just as easily find a company through Twitter as they can Google — and those social media links are busy boosting a company’s SEO signals.

As Rand Fishkin from SEOmoz noted, social now has the ability to affect your search results — meaning that all that traditional SEO legwork you put in could be displaced because a searcher’s Aunt Myrtle shared a related link.

That’s not necessarily a bad thing, however—after all, if Aunt Myrtle shares your link, it might be your own site that gets a free pass to the top of the search rankings.

Social Puts The Human Back In Search

Incorporating social into search has one huge benefit for users: it lets humans, not algorithms and search spiders, have a say in search results.

Generally speaking, which link would you rather click: one chosen for you based on keyword density, or one deemed relevant by 10,000 other users?

Hence, the Google +1 button: it gives any user with a Google account the opportunity to publicly vouch for a link. The same applies for social media: before Twitter and Facebook, only content creators and site owners had the power to send in-links to other sites; today, anyone with a social media account can share (and thus “vote” for) any link.

Naturally, now that Google and Bing are bringing social to the SEO party, users now have a variety of social factors to play with when attempting to boost site or brand credibility.

Granted, social SEO is still a new concept, but it’s assuredly the direction where search is heading. Below you’ll find some of the emerging social factors that can boost your site or your brand’s credibility.

Author / Social Authority

The search engines have recently started evaluating a social media user’s credibility just like they evaluate a site’s credibility. Google and Bing have both admitted to considering an author’s authority when incorporating social signals into search results.

Take it from Bing, in a Search Engine Land interview with Danny Sullivan from December 2010:

“We do look at the social authority of a user. We look at how many people you follow, how many follow you, and this can add a little weight to a listing in regular search results. It carries much more weight in Bing Social Search, where tweets from more authoritative people will flow to the top when best match relevancy is used.”

Google also considers a user’s authority when ranking search results. In the below screenshot, a Google search for “blogging” reveals an article by ProBlogger’s Darren Rowse—along with Darren’s Google+ profile picture and the number of G+ Circles Darren appears in.

To make a quick (if imperfect) analogy, author authority is similar to Klout. On Twitter, for example, the search engines might consider such factors as:

• Follower to following ratio
• Number of postings a day
• Number of lists the user appears on
• Number of @mentions a day

The Retweet Factor

Retweets from a power Twitter user can affect your search rankings just like in-links from an authoritative, established source.

As Jennifer Lopez revealed in her case study on SEOmoz, a retweet from the power user Smash Magazine about the SEOmoz Beginner’s Guide to SEO led to the site receiving significant traffic for the search term “beginner’s guide”—a term that had never brought the site traffic prior to the Smashing Magazine RT. The SEOmoz guide still ranks #2 in the Google search rankings for “Beginner’s Guide” to this day.

Search engines can also consider a tweet’s retweet rate. If a link tweeted to 1000 followers gets 100 retweets, it’s got a 10% retweet rate—and that link may do better in search rankings than a link with a 3% retweet rate.

Network, Network, Network

It’s not what you know, it’s who you know—or rather, who you’re engaging with on social media. Interacting with Twitter power users boosts your own credibility.

Of course, that doesn’t mean a food blogger should start tweeting Lady Gaga or Anderson Cooper. Relevance matters: if that food blogger starts a Twitter conversation with Wolfgang Puck or Anthony Bourdain, the blog’s credibility as an authoritative source for recipes may get a huge bump.

Build Your Links and Your Social Presence at the Same Time

Why do companies launch linkbuilding campaigns?

• To increase their visibility and credibility.

What do shared links on social media do?

• Increase visibility and credibility.

In-links (via BOTH social media sites and “traditional” external links from sites) are incredibly valuable to search engines, since they prove a user found it useful, not a searchbot.

The more shares you get, the more people are “voting” for your site. As I wrote previously, going viral can boost both your SEO signals and your credibility.

Encourage Sharing (& Trick Out Your Sharing Options)

If you want people to share your content (and thus boost your credibility), you’ve got to ask for it. A simple “If you found this content useful, share it!” can do a world of sharing good for a website.

Likewise, the more sharing options you include on your website, the easier it’ll be for users to share your site. It’s much easier for readers to click one link and instantly share your content than to go to individual social media sites.

The Obvious Downside of Social Credibility

In theory, credibility works the same way in the virtual world as it does in the real world. Before search engines will list your site or bump you up in the search rankings, they’ve got to trust you—similarly, your customers have got to trust you before they’ll buy from you.

The bright side of building your social credibility is that you’re building trust with both the search engines and your future customers at the same time.

However, in the real world, no one will instantly trust your brick-and-mortar business: you’ve got to earn your customers’ trust through quality customer service, word of mouth, and good publicity (staying in business for a long period of time helps, too). Online credibility is built in the same slow, eventual way.

Establishing your credibility takes time. There’s no shortcut. There’s no get-trust-quick scheme. Do good work over a long period of time and you’ll establish credibility. It’s as simple—and as difficult—as that.

Recently captured eye-tracking data for Google searches found that users are gravitating toward the Google Places listings over the organic listings, whether it’s the 7-pack or mixed with natural listings, as noted by the heat map below.

Google likes to make things tricky, and since it’d be too easy to have the same algorithm for Google Places and organic listings, you’re playing in an entirely different ballgame with local SEO. It should go without saying that you need a Google Places page to get ranked in local listings. Once it’s set up and optimized with exact categories, pictures and videos, the real fun begins: getting citations.

Citations are for local SEO as what links are for natural listings. The more places your business information is listed consistently, the easier search engines can find your business, and the better your local rankings will be. So, where do you get those citations?

Join The Aggregators

People don’t just rely on search engines to find local businesses. There are hundreds of local directories that that people use to search, and you should get a page set up on each. Not only will it help your local SEO, but it also gives another way for people to find you. Each industry will have its own set of aggregators, and doing a search for “City + industry” will help you find your options.

Here’s a list of the biggest general ones that you need to be listed in:

• Yelp
• Superpages
• City Search
• Urban Spoon and OpenTable (for restaurants)
• Yellow Pages
• Angie’s List
• Express Update USA (formally infoUSA)
• Yahoo Local
• Trip Advisor
• Merchant Circle
• Dex Knows
• Insider Pages
• Localeze
• Shop City
• Judy’s Book
• Yellow Bot
• Kudzo

Search For Your Competitors

You should always check to see what your competition is doing, and for local SEO, you want to find where they’re listed. Searching for their business name and phone number in quotation marks will bring you additional places that you can also list your business information.

Searching without the area code typically brings up more localized places. You can also search the business name with the address and city name. You’ll likely need to click through a couple of pages in Google since the first page will be results related to that business specifically.

Social Media & Local Blogs

Local bloggers also typically have a local business/vendors page that typically list places in their area. Searching for your city name + blog should be your first starting point.

You can also narrow down the results some by adding in your industry: If you’re a restaurant, search for city name + food blogs. Blogs.com and Networked Blogs are also good places to start looking for local blogs in your area.

Finally, search for the national blogs that have a local counter part. For example, The Knot has a local vendor database where you can put your business information if you provide any wedding services, like venues, catering, hair salons, florists, DJs, and transportation. Other options include Patch, The Business Journals, GalTime and BlogHer.

Local Directories & Newspapers

If people are looking for a local business, they don’t just look to search engines. Local newspapers and local directories are an easy way to grab a citation, and these typically have a higher value since they’re focused only on one area.

If you host events, submit them to your local newspapers’ calendar listing. Some local newspapers will also have a business listing database that you can submit your information to.

Search for “city name + business listings” and “city name + directory” to find more results. With these searches, you’ll likely also see PPC ads of additional places to get listed.

Again, putting in your industry will also narrow down results, and you can also just search for your industry, IE “dentist directories” or “Atlanta dentist listings.” And don’t forget your city’s chamber of commerce website.

Whitespark has also created a local citation finder that does much of the leg work for you — for a price, of course, but they do have limited free version available. The most important thing to remember is consistency is key. For example, if you have “Street” on your Places page, make sure it’s “Street”, not “St.”, across all citations as well.

In this article, I’ll continue on to the next step and show you how I prioritize the keywords by evaluating the difficulty of reaching top results.

Check Keyword Rankings

The next step I take before beginning to optimize a page is to go through the “short list” of keywords I mapped to a page and prioritize them.

Take the keyword mapping you developed for a page you’re optimizing and check the organic rankings and clicksthroughs using a tool such as Google Webmaster Tools.

Note: To get a more accurate reading of the average position for a keyword in Webmaster Tools click the Filters tab and choose Search: Web and choose your most prominent location.

Check Keyword Rankings In Google Webmaster Tools

You should give a high priority to any keyword in your mapping that is already delivering traffic to the page from organic results. Check the rankings and if a keyword that is delivering traffic to the site isn’t reaching the highest average positions then it makes sense to try to improve the search results for this keyword (both rankings and clickthroughs).

You already know this keyword is contributing to the business goals so any improvements in organic results should increase leads, sales, sign-ups, or whatever the site goals may be. If the keyword is already reaching top results it should be given a high priority as you optimize so you don’t inadvertently hurt its rankings.

Plus, you’ll want to see if you can improve search listings which can improve click-throughs.

Check Organic Competition

On the other hand, if there are keywords in your mapping for this page that aren’t delivering clickthroughs to the site from organic results, then these keywords made it onto the mapping for this page because of your PPC results.

Before optimizing for any keyword that is not already bringing traffic from organic search results, it’s best to check the competition for the phrase to see if your website and this page are in a position to reach top results for this specific phrase (i.e. the important factors that influence organic search results such as the search friendliness of the site, site architecture & internal linking, incoming links and social engagement, etc).

Many of the available keyword tools have some method of rating the competition for a keyword phrase.

We use Wordtracker which provides IAAT data for each keyword. IAAT stands for ‘In Anchor And Title’, in other words it shows how many pages in a search engine’s index include both an HTML Page title that contains the keyword phrase and where the keyword phrase appears in the anchor text of an external link to that page. This is a decent indicator of the competition as is shows you the pages that have likely been optimized. However I would also look at the amount of competition since there are often pages that have never been optimized that rank well too.

Note: Don’t rely on the Google AdWords Keyword tool for organic keyword competition.

The High, Medium, and Low Competition data displayed in that tool are meant to give you an idea of how many advertisers are bidding for a particular keyword. That’s of little use for organic results.

If you don’t have a decent tool available to help determine organic competition, you can get a reasonable idea of the competition by performing a few special queries in Google. Here are three searches you can perform to estimate the competition of a keyword phrase.

Exact Phrase Search. In Google’s search field put quotes around the keyword phrase such as “frame sliders” to get the number of files in Google’s index that mention the exact phrase in the content on the page

Exact Phrase Search in Google

AllinTitle Search. In Google’s search field add “allintitle:” before the keyword surrounded in quotes (e.g. allintitle: “frame sliders”). This will give you the number of pages in Google’s index that have the keyword phrase in the HTML Page Title.

inanchor Search. The above two searches are usually enough to compare keyword competition but you could go further and do an inanchor search. In Google’s search field add “inanchor:” before the keyword surrounded in quotes (e.g. inanchor: “frame sliders”). This search will list pages that have the keyword phrase in the anchor text of an external link to that page.

Now, compile the same data for a number of keyword phrases for which a tool like Google Webmaster Tools shows this page is ranking high. At the same time, compile the data on a number of keywords for which the site is ranking from other pages. Check a number of keywords for which the site is ranking in the top 20 or 30 search positions.

Compare the results for the keywords for which the site and this page are already ranking to the results for the organic keywords that you are considering optimizing. If the competition for those keywords is far greater than any keywords the page is already ranking for, or worse, far greater than any keywords the entire site is ranking for, your chances of reaching top results just by optimizing a page for the keyword may not be very good.

It may be better to focus on keywords that you have a better chance of improving results for now. You may need to work on other issues before you’ll be able to reach top results for more competitive keywords such as site issues, link building and social engagement etc.

Here’s an example of the keyword competition for a client’s website including keywords for which the site is reaching decent search positions. It also includes data for some keywords we would like to improve organic search results for as they are bringing people to the site from PPC who then fill out an inquiry form.

A keyword competition chart

First, notice that regardless whether you sort the data using the Wordtracker IAAT competition data, the “Number of Pages” column, or the “allintitle” column you get about the same ordering of the competition for these keywords.

Compile the competition data using two or more data points and if you get about the same ordering of the keywords for each data set then you can be reasonably confident in your keyword competition ordering. You might want to assign a High, Medium, Low label for competition as I did in the above screenshot.

As you can see in the above screenshot, this new client’s website is only ranking for keywords that are not very competitive.

These folks, a national company, have had a small informational website for years. They have done very little online promotion and their search results reflect this. We need to fix some issues on the site and progress with link building, social engagement,etc. before we’ll be able to reach top results for the more competitive keywords. We’re optimizing pages on the site, focusing primarily on keywords with low to medium competition for now.

In future articles, I’ll cover more steps we take when optimizing existing pages on a site and adding new content to improve search results. If you want to submit some pages and a few keywords that I could use as examples, just submit them below in the comments.

First, they gave us meta data. Then they took it away (as a ranking factor). Now, search engines are once again asking us for meta data. His tone did not indicate any type of excitement about the Schema.org announcement or the use of the tags as an SEO strategy.

Other panelists commented on how the Schema.org tags could lead to additional code bloat, and in the end, none of the panelists recommended re-coding your entire site with the new Schema.org tags, as there was very little empirical evidence to support using Schema.org tags as an SEO strategy.

That’s a lot like where we are right now with HTML5. While people have been writing about HTML5 for several years, no one is out there providing evidence about the effects of HTML5 on organic traffic or really even SEO in general.

In fact, none of the well-known SEO blogs or writers are even discussing HTML5 very often, and those writing about HTML5 are all saying the same thing.

What HTML5 Can Offer

• Brings several improvements in usability and user experience
• Has several new tags that will help developers classify important content
• Is awesome for sites rich in media (audio & video)
• Is an amazing alternative to Flash and Silverlight
• Is SEO-friendly when it comes to website crawling and indexing
• Is going to be heavily used for mobile apps and games

So, survey says: HTML5 will someday make everything better. But that someday is not here just yet.

Most bloggers/writers fail to mention an important point: to date, none of the popular Internet browsers fully support all of the features and codes in HTML5. To be clear, today’s browsers will render HTML5, but there are still many aspects of HTML5 that are not fully supported by or compatible with all Internet browsers.

Google & HTML5

On May 22, 2010, Google created a Googledoodle tribute to the Pac-Man video game. It was an animated logo that was also a playable version of Pac-Man. The logo was created with HTML5, and it had a Flash option for browsers that didn’t support HTML5 at the time. I’d bet that the Pac-Man Googledoodle was most Internet users’ first experience with HTML5 and its capabilities.

Personally, I thought it was exciting. For me, it provided a tangible glimpse into the future of the Internet, website browsing, mobile applications and games, and website functionality. On the SEO sides of things, it opened up my imagination even more, and it got me thinking about all the potential that HTML5 would have in the realm of SEO.

But how would Google use HTML5 for SEO? Would Google provide any advantages to websites moving to HTML5?

In a post from Oct-2010, Barry Schwartz brought up HTML5 and Googlebot. Barry cited a Google Webmaster Help thread where Googler JohnMu implied that Google will “wait and see” and “adapt” to HTML5 as it grows in popularity:

In general, we work hard to understand as much of the web as possible, but I have a feeling that HTML5 markup is not yet as widely in use (and in use correctly) that it would make sense for us to use it as a means of understanding content better. As HTML5 gains in popularity and as we recognize specific markup elements that provide value to our indexing system, this is likely to change, but at the moment I would not assume that you would have an advantage by using HTML5 instead of older variants.

While that thread is over a year old, it says a lot about Google’s disinterest in providing any SEO advantages for websites using HTML5. If any of my clients are thinking about moving to HTML5 strictly for the SEO benefits, at this point I don’t think I’d recommend it as an SEO play. I would recommend HTML5 for other reasons, but not strictly for SEO…at this point in time.

If HTML5 is so much better than HTML4, when are we supposed to use it? And when will HTML5 help with my SEO strategy? Those are great questions. I’m glad I asked.

How SEO Benefits From HTML5

I do not want to create the impression that HTML5 is bad for SEO.

For some websites – especially those heavily reliant on Flash – SEO is a godsend. If you have a site that is nothing but Flash, you will definitely see benefits to switching to HTML5.

First and foremost, searchbots will be able to crawl your site and index your content. All of the content that is currently embedded in animations will be readable to search engines. In basic SEO theory, this one aspect of HTML5 will do wonders for your website’s ability to drive organic search traffic.

Another benefit of using HTML5 is that it tends to generate buzz (read: links). On Monday, Facebook launched their long-awaited HTML5 iPad app. Pandora recently launched a new version of their music player in HTML5. Gaming platform Zynga recently launched 3 new HTML5 games that will run on mobile browsers.

More and more stories are being written every day about websites re-launching HTML5 websites and/or HTML5 features. Being on the leading edge of new technology is great for users, but don’t forget about the link-building, press-generating aspects of such moves. It’s all about inbound marketing, right Rand?

Finally, from a usability perspective, HTML5 has the ability to transform how users interact with websites. Websites that are rich in media stand to gain the most, as there are several new elements in HTML5 that allow for easier audio and video streaming, particularly on mobile platforms.

By now, we are resigned to the fact that the iPhone and iPad will never support Flash. But that’s okay – because now we have HTML5! Developers can now embed audio and video without having to worry about browser compatibility and/or platform capabilities.

New Tags For HTML5

For quick reference, here is an HTML5 cheat sheet. If you are familiar with HTML, a lot of the tags look very familiar.

However, there are a few tags I would like to point out, as they will likely be critical to the SEO success of HTML5 websites:

• <article> specifies an independent block of content. The contents of an article tags should be entire self-contained. A blog post or new article could be wrapped in an <article> tag, for example.
• <section> specifies a subsection of a block of content, such as an <article>. If a blog post was broken into several sections by subheaders, each section could be wrapped with a <section> tag. Just as books have chapters, blocks of content can have section.
• <header> could server two purposes: (1) to specify the header of a page or (2) to indicate the header section of a self-contained block of content (an <article>). <header> tags might contain navigation, branding or the document headline.
• <hgroup> is used to wrap a section of headings (<h1> through <h6>). A great example would be an article with both a headline and subheadline at the top:

  <hgroup>
       <h1>Main Headline</h1>
       <h2>Article tagline or subheading</h2>
  </hgroup>

• <footer> is a bit like the <header> tag. It could specific the <footer> of an entire HTML document or the footer of an <article>. This may contain things like footer navigation or meta-data about an article (author, data, etc)
• <nav> is mean to enclose site navigation. This can be used anywhere: main site navigation, previous/next article links, or pagination.
• <aside> is for content related to the parent element in which is resides, but not strictly part of the main document. In other words,
• <video> is for video content. Its purpose is to provide a cross-browser compatible way to display video.

Final Thoughts On HTML5

If you are thinking that HTML5 can help with code bloat, you are correct. However, when we combine HTML5 with Schema.org tags, there is actually the possibility for more code per page. Such is life. The Web is getting massive, and there is a mind-boggling amount of content out there.

So when Google and Bing decide that they need more meta data because their super-sophisticated searchbots and index warehouses cannot possibly sort through all the content on the Internet, I say we should help them. And at the same time, why not be nice to our website’s visitors by making websites more fun and easy to use? HTML5 can help us achieve both goals.

Sure, a lot of SEOs will push the boundaries to test the algorithmic weight placed on the new HTML5 tags. Then search engines will constantly update their algorithms to adjust to shortcuts and strategies that SEOs find in HTML5.

You know, it actually sounds a lot like the last 15 years in the SEO world. Except now we don’t have to talk about indexing Flash anymore. Knock on wood.

Is that because to many of our prospective clients, SEO is a purely technical issue unconnected to marketing?

Personally, I don’t think that’s correct. At all. True, there is a technological side to any SEO plan.

But the strategy setting and business analysis components that draw upon a foundation in a Maslow-ian hierarchy of human needs isn’t “purely technical”.

Let’s take a home repair analogy. You could find a good roofing contractor to lay the shingles you’ve already ordered in the neatest, most efficient manner. But if a shingle roof itself is not the best choice for your building, you’ll only enjoy a sub-optimal result.

Ditto for any website’s SEO initiative. When SEO consulting is grounded in psychology and strategy, the entire method has more punch when it reaches the stage of coding and content structuring. It can harness synergies with marketing ventures like social media, press releases, public relations and more.

Indeed, as David Meerman Scott explains in his book about the new rules of marketing, to unthinkingly slap on ‘technical SEO’ to a Web structure wastes so much potential impact (and money) of a synergistic marketing plan.

Do you (like me) believe that, as an SEO Strategy Expert, you should always be engaged and deeply involved in the very heart of a business you consult for?

My hardest task whenever I’m called in for SEO consulting has been to convince business owners that, by adopting a strategic SEO approach, they will:

1. Increase sales
2. Reduce costs (saved on other marketing channels)
3. Improve resource efficiency
4. Create synergies (that often last for years)

My positioning efforts rely upon painting a stark contrast between a low-priced SEO firm that relies purely on keyword-stuffing and basic link building which leaves a lot of money on the table from missed opportunities. As Google evangelist Avinash Kaushik puts it, “Your keyword strategy shouldn’t be a fishing expedition!”

Many Internet marketers understand the importance of beginning with keyword research. Sadly, too many end right there. Keywords certainly matter. Done well, they will help you understand exactly what your market wants, in specific numbers, and suggest methods of optimization that best meet those interests.

But look at deeper intent and these keywords reveal a lot more about your prospective buyers that helps boost conversions. A keyword that’s a specific expensive food supplement suggests an affluent, health conscious prospect who seeks out quality products, and is willing to pay for the best.

This psycho-graphic profile assembled from keyword research is key to crafting landing pages that compel and convert at unbelievable levels.

Businesses need to look at every page as a ‘landing page’.  And they need to ask themselves these questions:

1. What are visitors arriving at this page looking for?
2. What problem are they facing and how can we help them?
3. How can we deliver an experience they’ve been looking for everywhere else?

When any business stops looking at a website as a brochure to display all that’s happening inside their business and shifts perspective to view things from their prospects’ standpoint, dramatic changes happen.

Every time a visitor upon arriving at the business website, says to herself, “This is exactly what I need! Why didn’t I buy this before?” a significant battle for mind share has been won.

The optimized webpage has earned a new fan and evangelist for the business. This shifts the focus away from price and into the realm of value, of having an amazing product or service in their lives.

This is exactly how effective SEO consulting should work. It isn’t complex or confusing, even a child can do it!

SEO isn’t mythical or mysterious. At its core, SEO is merely applied psychology that integrates with other elements like economics, search engine function, Web analytics and website architecture.

What Does Intelligent, Effective, Psychology-driven SEO Look Like?

• SEO is less about technology and more about human behavior. How and why do people use specific keywords? What concerns, worries and pain underlies them? How do you develop a content strategy that delivers solutions and value?
• SEO is less about you, and more about your clients. Boasting and bragging about accomplishments can be counter-productive for businesses. The more relevant questions to ask about your content are “Who will +1 this on Google?” and “Is this ‘Likeable’ on Facebook?”
• SEO is less  about ‘clicks’ and more about ‘sales’. If 9 in 10 visitors to your site leave without converting into buyers (or subscribers), then the #1 ranking on Google you worked hard for becomes less valuable. What can you do to better understand (and fulfill) a prospect’s needs?
• SEO is less about ‘persona’ and more about ‘keywords’. Many businesses strive hard to develop an SEO strategy around a persona. But ‘persona’ is immaterial. Keywords your prospects use matter a lot more. How can your keyword strategy avoid becoming a “fishing expedition”?
• SEO is less  about monitoring irrelevant metrics, and more about being smart. Set goals that are specific, measurable, attainable, realistic and timely. Don’t let an obsession with page views, clicks and rankings detract from more important measures. How will your SEO boost sales, conversions and profits?
• SEO is less  about cost-cutting, and more about treating your website like a precious and valued salesperson. After all, doesn’t your Web presence tirelessly act as a sales rep, pulling in prospects day and night? Why not treat it with respect and appreciation? Even spend a little more on it?

If parts of this post sound provocative, or even rude, I apologize. That isn’t my intention at all. I just had to let off steam about how SEO is widely perceived in the competitive world of online business.

I’ve worked as an SEO consultant and strategist in Norway for over 10 years, and one of my biggest grouses over this period is how an SEO expert is sidelined, usually being the last person to be consulted about a web development project. This leads to that most frustrating question every SEO specialist asks himself or herself from time to time: “Knowing how important SEO is to the success or failure of any online enterprise, am I getting enough respect and attention?”

My recommendations, therefore, are typically resented because they involve major restructuring and content rewriting – things which could be easily averted by getting an SEO consultant involved earlier in the process.

Speaking as an SEO consultant, I suggest that my fellow professionals also express an eager willingness to truly understand the business you’re consulting with. Get involved in understanding the heart of your client’s business. Know exactly what’s happening inside it and know its customers really well. Yes, it can be expensive in time and money. But then, it is an investment that pays rich dividends by creating a cost effective and successful online presence for that business.

Maybe I’m unconventional, but research and analysis play a major role in my SEO consulting and I believe it should in every SEO professional’s work too. Do you agree? Or not? Feel free to convince me… by sharing your thoughts in a comment!

Image credits: Crestock.com (Brain), Wikipedia (Maslow’s hierarchy of needs)

I’m really pleased with how the video turned out, and I hope you are, too. We worked with Common Craft to produce it, and they have many more great explainer videos like this in their Common Craft video library, so check that out!

You’ll find our video on YouTube in our Search Engine Land channel, and it also has a permanent home here on Search Engine Land as part of our What Is SEO / Search Engine Optimization? page.

If you haven’t visited that page, do check it out. It also provides background our about our more in-depth Periodic Table Of SEO Ranking Factors, links to free guides about SEO, resources here at Search Engine Land and across the web on the topics.

Hey, if you like that page, links and social shares to it are appreciated!

By the way, if you’re a Search Engine Land member, don’t forget a new feature we’ve added — our Search Engine Land Guide To SEO as a downloadable eBook. It’s just one of many benefits you get. Not a member? Consider signing-up!

Incorporating images as part of your overall content mix is a good idea for search optimization in and of itself. Images can provide additional opportunities for keyword signals on a page, and they represent good opportunities for ranking in search results under Universal Search.

But, if they’re also associated with places properly, they can convey additional location signals, helping a business and/or its website to be considered even more relevant for local searches.

If you’ve explored around Google Maps for any length of time, you’ve probably noticed that there are quite a few images associated with local places in the interface. Google uses a few different methods for identifying image locations.

If you geocode your images in some way, it gives Google and other search engines high confidence that the content should be associated with a particular place.

Geocode Images With Photo Sharing Services

One of the easiest ways to geocode an image is through using one of the top image sharing services such as Picasa, Panoramio or Flickr. (FYI, Picasa may be renamed “Google Photos” soon.)

In each of those services, you can upload a photo, then add various elements such as titles, descriptions/captions, tags and associate images with locations by dragging them over a map:

When you map photos via these sharing services, they take the longitude and latitude geocoordinates associated with the map location and store them with the image’s information. Google Maps reads in syndicated feeds of the images from these services and then is able to use the geocordinates to pinpoint them in the images layer in Maps.

But you may be wondering, once you have geocoded the pictures in an image sharing service, how should you then associate those pictures with your company?

You could leverage the hosting through the image sharing service, and display them on the pages of your site. Alternatively, you may also link to your site from the image sharing service’s pages. Both of these methods may augment your local search signal, and may be particularly efficacious when used in combination.

Using GPS Enabled Cameras & EXIF Data

In addition to the above, a slight variation on this technique would involve using a GPS-enabled camera upfront in the process to take your photos, and then uploading them into the photo sharing service.

GPS-enabled cameras will store the geocoordinates in the image file’s EXIF (EXchangeable Image File format) data. EXIF is a format for storing meta data about the image, such as date/time of the photograph, type of camera used, the image’s color profile, geolocation, and other info. Flickr, Panoramio and Picasa will all read out the geolocation from the EXIF info, and translate it into the geocoordinates when mapping the image.

Having the geocoordinates in the EXIF data may have some slight advantages to merely mapping a non-geocoded image, because image sharing services like Flickr will publish the EXIF data onto HTML information pages in conjunction with the image — so, the coordinate pair provides yet another crumb of local signal on the pages associated with your image content.

Using An Image Sitemap

Another method which you can use is to store images directly on your website, and then add an image sitemap file which includes a geolocation element for each image.

However, it’s unclear precisely how Google uses the geolocation info from image sitemaps. The XML schema only requires a string for the location, and uses a city and country like this, for example:

Image sitemaps appear to be mainly geared towards enabling Google’s Image Search to find and rank pictures. So, the geolocation code probably only affects how relevant an image is considered to be for a query, depending upon the geolocation of the searcher or if the query includes local qualifiers such as the city name.

It’s possible that you could enter an entire street address into the image sitemap, or even the geocoordinates for more precise pinpointing. But it does not appear to me that the geolocation data with image sitemaps is communicated over to the Google Maps silo at this time, and perhaps using the lat/long coordinates wouldn’t work at all. (If inclusion of a full street address or lat/long coordinates would work, Google should provide more examples or a better description in their help pages covering the topic.)

In the past, some developers might have geocoded their images by storing each image on a separate webpage, and then geotagging the webpage.

However, I consider this a poor technique at present, because it doesn’t clearly communicate to search engines whether the geolocation is specifically referring to the image, the website, or other subject matter that may be on the same webpage. All the other techniques here make it clear that the geolocation is referring to the image itself.

Manually Embedding Image Location Data

Another technique for geocoding an image would be to use an EXIF editor and manually embed the longitude and latitude coordinates into it. Google suggests this as a method for adding geocoded images into Panoramio, so this is not a risky method, even if it may seem a trifle arcane.

There are a number of programs out there which will allow you to edit an image’s EXIF data. Google’s Panaramio documentation recommends Exifer, although the Exifer site says the software hasn’t been updated since 2002.

Even so, it probably would function just fine, so long as it will run on your system. There are other EXIF editor packages, too. Here’s an example where I input the coordinates of the CN Tower in Toronto, using the free version of the Zoner Photo Studio:

As you can see, the software provides a fairly intuitive interface for inputting the lat/long values, and a map interface to dynamically show where the coordinates are plotted. (In Zoner, this interface can be accessed under the File -> GPS menu.)

The map can also be used to pinpoint the picture location by panning and zooming to locate the place and then clicking where the image was shot. (If you don’t know how to get the precise geocoordinate numbers for a street address, refer to my earlier article on How To Geocode An Address & Optimize Location Pages.)

To view the EXIF data for my example image of the CN Tower, you can use an online EXIF viewing service such as Jeffrey’s Exif viewer — click here to check it out. There are various other browser extensions and software packages that can be used to view and edit the EXIF data as well.

Once it’s geocoded, you can upload it to one of the photo sharing services such as Google Panoramio, Google Picasa, or Flickr (you should first enable settings allowing geocoded photos to be automatically mapped when uploaded). Once uploaded at one of these services, it will eventually be automatically spidered and made available via Google Maps.

Be sure that the image upload service you use is now correctly displaying the image’s location on a map, and does not make any mistakes reading the coordinate pair.

Google’s Panoramio instructions indicate that some EXIF data cannot be interpreted by them, so be sure the data is accessed and that the image is mapped to the correct location in Google Maps afterwards (the Google Maps interface that is integrated into Panoramio).

Likewise, the image’s mapped location may be checked in Flickr and Picasa as well, via the map interfaces and EXIF content pages they generate.

Having the geolocation embedded in the EXIF part of the image file may be the best option from an optimization perspective, because it not only allows you to leverage one of the image sharing services to get your images connected with maps, but also because you may then store the image on your website where the locational data might help to further augment all the other basic local SEO methods you’re using.

Schema.org As A Geocoding Technique

Probably the newest image geocoding technique would involve using the schema.org protocol which the search engines recently announced, and which allows you to tag individual images on your webpages using Micro Data. Under this protocol, you could use the ImageObject type and embed a contentLocation property within it to specify the place.

At present, though, I think that the location content is likely not yet being absorbed by Google’s Image Search or by Google Maps, nor by other search engines. While this technique has a lot of advantages, it is still too early to effectively leverage — but, stay tuned and expect that this method may well become a defacto standard along with embedded geolocation in EXIF data.

Thus far, I believe you get more local search benefit from uploading geocoded images to the image sharing services I highlighted here, since they are well-integrated with Google Maps.

However, as the search engines become more sophisticated, and as more cameras (and camera phones) integrate location data in image files, we might reasonably expect this information to be harvested and used in local search ranking evaluations wherever images may be stored on the internet.

It seems weird now, but Inktomi made more money from Traffic Server than it did from the search engine. Such were the economics of the pre-Google Internet. It was a great business until 1) bandwidth got really, really cheap and 2) almost all of the customers went out of business in late 2000/early 2001. (Most of Inktomi was acquired by Yahoo! in 2002, and Traffic Server was released as an open source project in 2009.)

Because of my work with proxy caches, I’m always surprised when I do a technical review of a site and find that it has been configured not to be cached. When optimizing a website for crawling, it’s helpful to think of a search engine crawler as a web proxy cache that is trying to prefetch the website.

One quick note: When I talk about a “cached” page, I’m not referring to the “Cached” link in Google or Bing. I’m referring to a temporarily stored version of a page in a search engine, proxy-cache, or web browser.

As an example of a typical cache-unfriendly website, here are the HTTP response headers from my site, which is running my ISP’s default Apache install and WordPress more or less out of the box:

The three lines circled in red are HTTP-ese for “Don’t cache this ever, under any circumstances.”

A little more detail about these headers:

1. Expires: indicates how long a proxy-cache or browser can consider a document “fresh” and not have to go back and get it. By setting this to a date two decades ago, the server is indicating that it should never be considered fresh.
2. Cache-control: is used to explicitly tell proxy-caches or browsers information about the cacheability of the document. “no-store” and “no-cache” tell it not to cache the document. “must-revalidate” means that the cache should never serve the document without checking with the server first. “post-check” and “pre-check” are IE-specific settings that tell IE to always retreive the document from the server.
3. Pragma: is an HTTP request header, so it has no meaning in this instance.

Cache Control Headers & Technical SEO

So what do cache control headers have to do with technical SEO? They matter in two ways:

1. They help search engines crawl sites more efficiently (because they don’t have to download the same content over and over unnecessarily).
2. They increase the page speed and improve user experience for most visitors to your site. It can even potentially improve the experience for first-time visitors.

In other words, by adding a few lines to your Web server configuration to support caching, it’s possible to have more of your site crawled by search engines while also speeding up your site for users.

Let’s look at crawl efficiency first.

Crawl Efficiency

Only two pairs of cache control headers matter for search engine crawling. These types of requests are called “conditional GETs” because the response to a GET will be different depending on whether the page has changed or not.

Searchengineland.com happens to support both methods, so I will be using it in the examples below.

Last-Modified/If-Modified-Since

This is the most common and widely-supported conditional GET. It is supported by both Google’s and Bing’s crawlers (and all browsers and proxy caches that I’m aware of).

It works like this. The first time a document is requested a Last-Modified: HTTP header is returned indicating the date that it was modified.

The next time the document is requested, Googlebot or Bingbot will add a If-Modified-Since: header to the request that contains the Last-Modified date that it received. (In the examples below, I’m using curl and the -H option to send these HTTP headers.)

If the document hasn’t been modified since the If-Modified-Since date, then the server will return a 304 Page Not Modified response code and no document. The client, whether it is Googlebot, Bingbot, or a browser, will use the version that it requested previously.

If the document has been modified since the If-Modified-Since date, then the server returns a 200 OK response along with the document as if it were responding to a request without an If-Modified-Since header.

ETag/If-None-Match

If-None-Match requests work in a similar way. The first time a document is requested, an Etag: header is returned. The ETag is generally a hash of several file attributes.

The second request includes an If-None-Match: header containing that ETag value. If this value matches the ETag that would have been returned, the server returns a 304 Page Not Modified header.

If the ETag doesn’t match, then a normal 200 OK response is returned.

ETag/If-None-Match is definitely supported by Bing, but it’s unclear whether Google supports it. Based on the analysis of log files that I have done, I’m pretty sure that Googlebot web requests don’t support it. (It’s possible that other Google crawlers support it, though. I’m still researching this, and I’ll post a follow up article if/when I get more information.)

One common problem with ETag/If-None-Match support pops up with websites that load-balance between different back end servers. Many times, the ETag is generated from something that varies from server to server, such as the file’s inode, which means that the ETag will be different for each back end server.

This greatly reduces the cacheability of load-balanced websites because the odds of requesting the same document from the same server decreases in proportion to the number of back end servers.

In general, I recommend implementing Last-Modified/If-Modified-Since instead of ETag/If-None-Match because it is supported more widely and has fewer problems associated with it.

When To Use These Conditional GETs

Conditional GETs should be implemented on any static Web resources, including HTML pages, XML sitemaps, image files, external JavaScript files, and external CSS files.

• For Apache, the mod_cache module should be installed and configured. If the server still isn’t supporting conditional GETs check for a CacheDisable line in the httpd.conf or a .htaccess file somewhere.

• For IIS7, caching is controlled by the <caching> element in the site configuration file. I’m not sure how to enable it in IIS6, though it appears to be enabled by default.

For dynamic, programmatically generated files, the HTTP headers associated with conditional GETs need to be sent from the page code. You need to do some back of the envelope calculations on two factors to determine if this is worth it.

1. Does it take as many resources (for example, calls to back-end databases) to determine whether the page has changed versus generating the file itself?
2. Does the page change often compared to how often the page is crawled by search engines?

If the answer to both questions is yes, then it may not be worth implementing support for conditional GETs in your code for dynamic pages.

Page Speed

I also recommend setting expiry times for static resources that don’t change often, such as images, JavaScript files, CSS files, etc.

This allows browsers to store these resources and reuse them on other pages on your site without having to unnecessarily download them from the Web server.

Also, it is likely that these resources will get stored in a proxy cache somewhere in the Internet where it will be served more quickly to other users, even on their first visit.

There are two ways to set an expiry time using HTTP cache control headers.

1. Expires: <date>, which indicates the date before which a resource can be stored.
2. Cache-control: max-age=<seconds>, which indicates the number of seconds that a resource can be stored.

The expiry time can be set up to a maximum of one year, according to the HTTP spec. I recommend setting it at a minimum of several months.

Configuring Expiry Time

For Apache, it requires installing the mod_expires tag and creating some ExpiresDefault or ExpiresByType lines. Cache-control also requires mod_headers.

IIS7 can be configured through IIS Manager or some command line tools. See this link for more details.

For resources that are generated dynamically, these headers can be added programmatically like any other header. Just make sure that the Expires: date is in the right format or it likely will be ignored.

Other Resources

Below are some additional resources relate to caching, since this article only scratches the surface of the HTTP cache control protocol. I recommend checking out the links below to learn more about it.

Testing cache control headers

• Redbot.org, written by “mnot“, is the best cache-checking tool I am aware of. I use it all the time when assessing sites.
• Microsoft has a very useful tool for looking at headers that is available here.

I’m also a big fan of using curl -I from the command line to look at headers directory.

Advanced reading

• Google’s page speed article on leveraging caching.
• Yahoo’s best practices article for speeding up a web site contains some information about caching (click on the “Server” category):[[[]]]
• Bing outlines their support for conditional GETs and includes some helpful links here.
• Mnot has an excellent, thought slightly dated, overview of caching that is very useful.