Topic: Googlebot

Google Offers Robots.txt Generator

Google’s rolled out a new tool at Google Webmaster Central, a robots.txt generator. It’s designed to allow site owners to easily create a robots.txt file, one of the two main ways (along with the meta robots tag) to prevent search engines from indexing content. Robots.txt generators aren’t new. You can find many of them out […]

Google Webmaster Tools: Now On Your iGoogle Homepage

Google’s Webmaster Central is giving you a new way to access the diagnostics and stats for your sites. Rather than log into your Webmaster Tools account, you can add the Webmaster Tools gadget to your iGoogle page and access information from there. You can choose what components to display on your iGoogle page. Below are […]

MSNbot 1.1: Live Search Implements A More Efficient Crawl

Today, Microsoft announced changes to their Live Search crawler intended to reduce bandwidth resources during the crawl of a site. MSNbot (upgraded to version 1.1) now supports both HTTP compression and conditional get. The post on the Live Search Webmaster Center blog describes each feature in detail and includes links to tools you can use […]

Google: We Didn’t Censor Aboutrika’s Gaza T-Shirt Photo

Google’s Matt Cutts has lent his personal blog to the Google Images search team so they could debunk a rumor that Google censored an image of an Egyptian football player, Mohammad Aboutrika, stripping off his team jersey during the African Cup Of Nations to show a "Sympathize With Gaza" T-shirt underneath. No censoring, says Google […]

Google Universal Search Expands

It’s just over a half-year since Google launched Universal Search, its method of blending results from its own various topically-focused or “vertical” search engines. Since that time, the system has evolved. In particular, Google Universal Search now fills more than just 10 spots on the page, while shopping and blog search results are among new […]

The Google Webmaster Tools Quick Start Guide

Google has created the Google Webmaster Tools Quick Start Guide to help webmasters learn the features and reasons to use the tool. The ten step guide displays details on the dashboard, the overview section, the diagnostics section, the top search queries screen, crawl stats page, the “What the Googlebot Sees” page, the links page, the […]

Googlebot In Aisle Three: How Google Plans To Index The World?

Robots reading cereal boxes in the supermarket? Googlebot at the art museum? Street signs and building addresses snatched from Street View images for local search, image search, and product search? Three new patent applications published at the U.S. Trademark and Patent Office this week explore the intricacies of reading text in images taken from Google’s […]

Google Adds Sitelinks Control To Webmaster Tools & Much More

Google has informed us that today they are releasing a new update to Google Webmaster Central’s tools. The new features include a way to manage your Google Sitelinks, more historical query statistics, an extension of sitemaps to support code search, an improved message center and a warning that may help notify you of possible duplicate […]

Which Google Products Should A Small Business Use? (Part Two)

Google offers a variety of products beyond its traditional search engine, and many of those products could help a small business owner grow an online business. Even more enticing is the fact that many of Google’s products are free. But that doesn’t necessarily mean a small business should use all of Google’s products. In part […]

The Open Directory’s Home Page Goes Missing In Google

The Open Directory’s home page appears to have gone missing from Google’s search results. For example, a search on dmoz (the Open Directory’s nickname) does not return the home page in the search results. Here is a screen capture: Similarly, searches for open directory or open directory project also don’t list the site at the […]

Cheat Sheet For Google Cheat Sheets

Lifehacker points to a two-page Google Cheat Sheet PDF document that contains information on the company, their services, the IP addresses for Googlebot, Google domains, famous operators and much more. But aren’t there more Google cheat sheets? Sure — here are some others:

Google To Add “Unavailable After” META Tag

Getting Into Google by Jill Whalen reports Dan Crow, director of crawl systems at Google, saying that Google is releasing a new META tag named “unavailable_after.” The “unavailable_after” tag will allow you to tell Google when Googlebot should no longer crawl that page. Jill explains that this tag comes in handy when you have a […]

PPC

Can You Get By Without Google & Major Search Engines?

There are two projects going on today to see if we can survive without the major search engines. The first project is for searchers, to see if they can search alternative search engines without using the five major engines. The second project is for search marketers, to see if they can generate enough traffic and […]

SEO

Google Features Volkswagen, Which Happens To Be Search Spamming

The Google Enterprise Blog recently featured the Volkswagen web site for using Google Enterprise search to power a new feature on the VW web site. As you can see, the Volkswagen home page has a huge search box in the middle of the page. Cool, right? Danny and I think so. As Danny was explaining […]

SEO: Real Skills That Can Protect Your Traffic

Maybe SEO isn’t rocket science, but the wrong SEO decision (say knowing the difference between a 301 and a 302 redirect) could be costly. That’s illustrated in a Wall Street Journal article today on news search engine Topix and SEO issues in general. A look at that, revisiting Google’s webmaster support efforts plus new confirmation […]

SEO

Meta Robots Tag 101: Blocking Spiders, Cached Pages & More

Last week, I covered a new command for the meta robots tag — one to prevent search engines from using Yahoo titles and descriptions. In doing that, a number of questions came up about the meta robots tag syntax itself. Google Webmaster Central has now posted “Using the robots meta tag,” providing some clarity from […]

YADAC: Yet Another Debate About Cloaking Happens Again

Sigh. Double sigh. Triple sigh. I guess now that the SEO industry has had the required twice-yearly debate about the reputation of SEO, it’s time to do the go round about cloaking once again. A quick word about cloaking has Google’s Matt Cutts trying to clarify concerns that Philipp Lenssen of Google Blogoscoped has been […]

Matt Cutts On Google Rumor Debunking Rampage

Matt Cutts has a fun weekend debunking Google rumors at his blog. He covered rumors that the toolbar leads Googlebot to index pages, a story about Google funding terrorism (a topic I avoided on purpose), Google working with the CIA as a conspiracy, and Google cheating on AdWords for their own ads. He also wrote […]