The Google Webmaster Central blog announced Google has published a new set of documents in the Google Code section on how to control Google from crawling and indexing your site. You can read the set of documents over here. The technical documents are broken down into five section: Getting Started Robots.txt specification Robots meta tag […]
As our culture continues to move online, personal information about us that previously may have only been seen by our friends and family is increasingly shared in public spaces on the internet. What can you do if you search for your name on Google and find information you’d rather keep private? Or find yourself or […]
As a site owner, you generally want Google to index as many pages of your site as possible. But there are certainly times when you find that you’ve accidentally let Google index confidential content or other information you don’t want published, and you want to get it removed as quickly as possible. Read on for […]
The condition of standard SEO metrics search engines provide the web community (if the term “metric” can even be used here) is pretty deplorable right now. Back in February, I wrote: “…the convention of using indexed page counts and backlinks to benchmark and report on site performance is facing a similar demise [to ranking reports].” […]
Earlier this week, site owners noticed that substantial Yahoo Site Explorer link data seemed to be missing. A glitch? Gone for good? The first signs of integration into Bing webmaster tools? I talked to Yahoo and got at least some initial answers. Many site owners regularly use Site Explorer for information on external links to […]
No one seems to be immune from a Google indexing problem that has many site owners baffled. Blogs and websites, big and small, aren’t being indexed as quickly as they normally are — if they’re being indexed at all. CNN.com, with its PageRank score of 10, is typically crawled frequently and deeply by Googlebot. But […]
It’s a long-standing complaint of mine with Google. You can do a search, then repeat the same search and “subtract” a word from your original set, and Google will return more matches — not less. It shouldn’t happen. But does, and here’s why. Subtracting Gives You More? Consider a search for cars, which reports that […]
As I reported days ago, over the weekend the popular, but virtually useless, add URL form on Google did not work. Google posted a known issue on Saturday, October 2nd. Today, October 6th, I am finally hearing reports that the add URL form is now working again. Google’s message that the add URL form was […]
Google’s been under fire in various quarters, including the New York Times, that it should open up about how its ranking algorithm works. I think offering the exact formula would be difficult plus potentially give away trade secrets. But how about just listing some of the basic “ingredients?” Nope, said Google CEO Eric Schmidt, this […]
In January, I was invited to speak to Google’s search quality team about issues I had with Google’s search results. My topic? For queries where I know a subject really well, I often found Google provided some pretty poor results in the top listings along with the good ones. I wanted more perfection! With the […]
Webmasters build links for two reasons: to drive traffic and to influence search engine placement. Neither option is as cut and dry as I’ve stated here but when it gets down to it, that’s why webmasters deliberately build links. If you’re building links to support ongoing SEO efforts, there are a handful of key tips to […]
Earlier I wrote about the depature of Yahoo’s Srinija Srinivasan. Yahoo’s lost another person, and another big figure in search, that of Tim Mayer. Tim just tweeted, as shown above, that he’s enjoying his first “post-Yahoo” day. Tweeted using the Twitter account I personally set up for him ages ago, I might add! Who Didn’t […]
That’s one of the big questions people are asking after yesterday’s launch of Facebook Questions. While many have assumed the answer would be “yes,” a Facebook spokesperson tells us that assumption is wrong. Currently, search engines cannot access questions and answers through our Questions product. That may be something we consider for the future but […]
Today, Microsoft launched a revamp of their Bing Webmaster Tools. I talked to them back in June, when they previewed the tools at SMX Advanced, and they told me that they were starting from scratch and rebuilding the tool from the ground up. So how are things different? They say they are focused on three key […]
Blogger and former journalist Tom Foremski reports he had a conversation with the head of engineering for Google in Brazil and says the guy supported the idea of open-sourcing the search index: I mentioned an idea I had about making the Google index into an open database that anyone could access, I said that this […]
Since Google launched XML Sitemaps back in 2005, they’ve added specialized formats to enable site owners to submit content other than web pages. Until now, site owners have had to create separate Sitemaps for each content type: Mobile News Video Geo Images Code Search That’s now changed. You can now create a single XML Sitemap […]
SMX Advanced kicked off this morning in sunny Seattle (yes, seriously) with a packed house for the opening session of the SEO/Social Media track, “SEO for Google vs. Bing: How Different are They?” I won’t be recapping all of the sessions I attend today and tomorrow, but will do my best to share any interesting […]
There’s been some commotion from bloggers and Twitter users who noticed this week that retweets had suddenly disappeared from Twitter search results. But Twitter says this isn’t part of a “war” on traditional retweets, and things will soon revert back to normal. The issue is that retweets weren’t showing up in search results for logged-in […]
Twitter Search has added a new “expand” option that allows you to turn a shortened URL into back into its original longer version. That’s handy to know where the URL is taking you to. It may also help increase the relevancy matching of some tweets.
The Google Webmaster Central team posted a known issue report with a bug in the reporting engine within Google Webmaster Tools. The indexed URLs report is currently showing 0 pages indexed for most sites who have submitted sitemaps to Google Webmaster Tools. The notice reads: The indexed URL counts for Sitemaps that you have submitted […]
The startup 80legs launched in September of last year to make web crawling affordable and accessible to anyone that wanted to subscribe to the service. The company makes crawling the web or customized slices of it available to even small companies. More flexible than something like the Google Custom Search Engine, pricing ranges from free […]
The home page of OpenOffice.org, the well-known Microsoft Office competitor, is missing from Microsoft’s Bing search engine. While it sounds suspicious, the problem has nothing to do with Bing itself — it’s a technical problem on OpenOffice.org’s end. Ian McAnerin noticed earlier today that OpenOffice.org doesn’t show up in Bing on searches for [open office] […]
Yahoo announced a search update to their index. The update includes possible updates to Yahoo’s crawling, indexing, and ranking algorithms. Dan Rampton, Yahoo’s Product Manager said: The Yahoo! Search engineering teams are rolling out updates to crawling, indexing, and ranking algorithms. Similar to previous updates, you may notice some ranking changes and page shuffling during […]
In my role as president at GetListed.org, I sometimes receive emails from users asking why one local search engine or another is displaying an old location for their business, or why the search engines still aren’t showing their new website address, or why the phone number listed for their retail location is actually the one […]
While some webmasters anxiously wait for Google to roll out its Google Caffeine search infrastructure any day now, Google says it may not happen for “months.” What’s more, the IP address of the one data center where Caffeine was said to be available may have changed, and Google is no longer saying where users can […]
As both the Wall Street Journal and Telegraph UK are reporting, the EU has received complaints into Google’s competitive practices and potential abuse of its dominant market position in Europe. These come from some private companies, which we discussed in part before. One of those companies, the shopping engine Ciao, was acquired by Microsoft in […]
In October 2009, Google proposed a new standard for implementing AJAX on web sites that would help search engines extract the content. Now, there’s evidence this proposal is either live or is about to be. Read on for more details on the proposal, how it works, and why it might be past the proposal stage. […]
The Yahoo Search Blog dusts itself off to announce a search update happening right now. The last “weather report” Yahoo announced was in September 2009 – typically, Yahoo would announce these updates monthly. We actually expected this to be announced. I was tracking a WebmasterWorld thread where SEOs were noticing ranking changes at Yahoo Search. […]
Looking for old tweets? You won’t find any that are more than a week or so old on Twitter. Put the blame on all the massive number of new tweets coming in. The engines, they canna hold captain! Or that is, the search index behind Twitter Search can’t hold it all. NOTE: Since this was […]
When it comes to search marketing, the Big Three is about to become the Big Two. The deal heard round the world this past July comes to fruition in 2010—and the much-anticipated partnership between Microsoft and Yahoo! will have significant implications for search engine optimization. Paid search will no doubt face its share of opportunities […]
Via Search Engine Roundtable comes news of refreshing candor from a Bing employee about the performance of Bing’s crawler, which is still referred to as MSNbot. The admission happened yesterday in the Bing Webmaster Center forums, when Brett Yount, a program manager for Bing, replied to a question about why Bing has only indexed one […]
Google’s “First Click Free” (FCF) policy was intended to provide a better experience for Google users and for premium content and news publishers (e.g., the Wall Street Journal) by allowing their content to be indexed and exposed for subsequent sale to consumers. However, as most of you are aware, the publisher pay walls are going […]
As expected the Google Caffeine index is now live on one Google data center. Note, it will not be added to more Google data centers until after the holidays. As I reported at the Search Engine Roundtable, Matt Cutts of Google confirmed that the IP 209.85.225.103 is hitting the data center that is running the […]
Bloomberg has a story out suggesting that the publishers of the Denver Post and the Dallas Morning News may be joining Rupert Murdoch’s coalition of the willing to block Google. But I don’t think that’s what going on at all, and I’m trying to clarify things. Dean Singleton, CEO of MediaNews Group, which publishes the […]
Forget PageRank, at least if you’re a news publisher looking to do well in Google News. Google’s news service doesn’t rely on the same algorithm used by “regular” Google, of which PageRank is a part of. Instead, Google News taps into its own unique ranking signals, which include user clicks, the estimated authority of a […]
Google is buying ads to explain why an offensive image of the First Lady is back in its index and ranking so highly on searches for Michelle Obama in Google Image Search. As we reported last week, Google had originally removed the Michelle Obama image on the grounds that the site hosting the image violated […]
Backtracking on a controversial decision earlier this year, the National Association of REALTORS® (NAR) has adopted a new policy that allows real estate professionals to have their sites — including home listings that belong to others — indexed by search engines. The controversy reached a peak this year when the NAR agreed with a local […]
Google’s got a present for searchers and site owners. The long anticipated new Google Caffeine search index will go live soon, just after the holidays. The Google Caffeine Sandbox, where people have been invited since the summer to test Google’s next generation search infrastructure, has been closed, replaced with this “it’s coming” message: We appreciate […]
The long-running debate over Google and its impact on newspapers and journalism took another turn today when News Corp founder Rupert Murdoch said his company may makes its sites invisible to Google, and Google fired back by saying, in essence, bring it on. It began with this interview on Australia’s Sky News (which Murdoch owns), […]
Microsoft is retiring its MSNBot1.1 and introducing its next bot: 2.0B. According to the Bing Community blog: As MSNBot 2.0b enters full-scale production, the time has come to retire our previous generation bot, MSNBot 1.1. By the end of the first week in November, you will no longer see the following user agent in your […]