David Ingram and I recently had the pleasure of sitting with Tom Critchlow from Distilled (SEO consultants) for a few hours to asses our local business directory, brownbook.net. A whole load of notes and actions flowed from the meeting — some of which may be very useful to others.
Here are a few points to consider (any input, comments, tips, notes, thoughts… gratefully received):
1. Homepage footers are bad news and may even carry penalties – solution: integrate the footer navigation into the content on pages.
2. Information architecture is key to a well indexed site — how do the pages flow and can page rank easily flow throughout your site? Think of it as a pyramid of champagne glasses being fed by a bottle pouring into the glass at the top of the pyramid; can Google get to all the pages?
3. Add RSS feeds to regularly changing content to make doubly sure it gets indexed.
4. The concept of crawl allowance. When Google spiders your site, it will have a certain amount of time or an allowance of pages that it spend in your site. The allowance will be driven by many factors including the quality of content; the better the content, the more likely Google will spend more effort trying to index the whole of your site. You may be lucky and have your whole site spidered and indexed; on the other hand, you may have only a fraction of your site indexed. Because Brownbook.net has 34 million business details pages, our challenge is how do we get the highest value pages indexed, in preference to the lower quality pages and therefore encourage Google to dig deeper.
5. Google MayDay update reportedly de-valued pages with little unique content.
6. Sitemap. Pay little attention to ‘crawl rank’ within the sitemap, as Google does not seem to take it in to account. Do not have any re-directed pages in the site map.
7. Separate TLDs (top level domains eg .com, .uk) for each country. A good ‘rule of thumb’ is that if you have a separate marketing department in that country, then yes, separate the site with its own TLD for that country. If you don’t, then stick with the one TLD and use sub directories for each country (eg www.brownbook.net/UK/xxxx) and channel all your marketing effort in to the one TLD. This way all marketing effort is contributing page rank to one TLD and the efforts are not split in to multiple TLD’s. Use Google Webmaster Tools to tell Google which directory is for which country.
8. Microformats. Use wherever possible and then test using Google’s rich snippet tester.
9. Google Analytics. Use the new asynchronous Analytics code to increase page load speed.
Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.
Related Topics: Small Is Beautiful








Oooh, great notes! I’m especially intrigued by #1 since we’ve been trying to prune the footers on some of our older client sites down to the bare essentials on pretty much that premise. Plenty in here to spark some ideas. I’m making this my Friday Link Love post :-)
I highly doubt ‘asses’ is a good semantic field placement for this content.
Hi Mark,
Thanks for sharing this info. Nice to see how some great minds work, especially on really large sites.
On #1, do you have any examples of a homepage footer drawing a penalty? I realize they’ve been devalued for a long time now, and they aren’t the best practice, but short of it being incredibly spammy I’ve never heard of a penalty involved. Could you elaborate more on this please?
Crawl allowance is a good name for that concept. I’ve struggled on what to call it when describing it in-house. It is something I’ve been trying to make a point of in order to relay how important it is to eliminate duplicate content issues with our platform. When the search engines spend X amount of your “crawl allowance” on duplicate content, there is X amount of unique content not being crawled.
Regards
Premium member since 03/2009
Thanks Adrian for spotting the typo…
matthiaswh, thanks for your comments, I will ask Tom if he could comment with a few examples of his findings that have drawn him to this conclusion.
My thinking is really doing things in an open and honest way that benefit users. Watching the Matt Cutts videos (http://www.youtube.com/googlewebmasterhelp) continually re-iterates my view on this – less spammy, more user value is good and therefore more likely to do well.
I don’t have any examples I can share regarding footer links specifically unfortunately but this post from SEOmoz has some interesting insights:
http://www.seomoz.org/blog/footer-link-optimization-for-search-engines-user-experience
Tom