Is Google Broken? Sites Big & Small Seeing Indexing Problems

No one seems to be immune from a Google indexing problem that has many site owners baffled. Blogs and websites, big and small, aren’t being indexed as quickly as they normally are — if they’re being indexed at all.

CNN.com, with its PageRank score of 10, is typically crawled frequently and deeply by Googlebot. But the most recent article added to its home page (under the “Latest news” title) still hasn’t been indexed by Google more than three hours later.

google-cnn

Here on Search Engine Land, the most recent article we’ve published was my piece about the new Chrome extension for reporting spam. That was published an hour ago, and at this moment, Google is only showing a FriendFeed post that links to our article.

google-sel

That FriendFeed post seems to come from an RSS-based mashup that grabs posts from a variety of tech-related sites.

Perhaps related is this situation with Mashable’s post today about the Wikileaks Iraq document release; rather than indexing the actual article URL, Google has indexed the URL with several UTM tracking parameters for Google Analytics. The question mark in the green display URL (below) gives it away.

mashable

When Google Caffeine launched this past summer, faster indexing and a fresher index was Google’s big selling point. As Google’s Matt Cutts explained at the time, Caffeine meant “that all content … can be searchable within seconds after its crawled.”

Except not this week. And the indexing problems are affecting web site owners of all sizes. As Search Engine Roundtable reported this morning, there are active threads in the Google Webmaster Central forums about this, along with various blog posts out there, such as here and here. The reports there appear to be mainly from bloggers using Google’s blogspot.com platform, but as you can see above, the indexing trouble isn’t limited just to Blogger users.

In one of those threads, Google employee John Mueller says Google is following the user reports and “looking at ways to resolve the issues brought up here.” We’ll keep an eye on any new developments or updates.

If you’re seeing or experiencing indexing problems in Google, let us know in the comments.

Postscript by Barry Schwartz: John from Google replied to the thread in the Webmaster forums saying:

I just wanted to provide a short update. This issue should be resolved soon, but it will take a several days for everything to catch up and for these changes to be visible. I’ll provide another update after the weekend.

In the meantime, there’s no need to make any changes on your side. Content from your sites will continue to be indexed during this time (though perhaps at the moment not as quickly as before in some cases).

I’ve been in contact with the indexing and diagnostics teams since your initial reports — we are taking your reports seriously and doing what is necessary to resolve this problem as soon as possible. We love sending users to your sites quickly after you have published something too :-).

Just to be clear, the issues from this thread, which I have reviewed in detail, are not due to changes in our policies or changes in our algorithms; they is due to a technical issue on our side that will be visibly resolved as soon as possible (it may take up to a few days to be visible for all sites though). You do not need to change anything on your side and we will continue to crawl and index your content (perhaps not as quickly at the moment, but we hope that will be resolved for all sites soon). I would not recommend changing anything significantly at this moment (unless you spot obvious problems on your side), as these may result in other issues once this problem is resolved on our side.

Postscript #2 by Barry: John from Google commented on the Search Engine Roundtable claiming it has been fixed. I did spot check the forum thread and many webmasters and bloggers confirmed. John did add:

If you are still seeing issues with indexing of new content, it’s likely to be a different problem & I would recommend posting in our help forum with all the details.

Related Topics: Channel: SEO | Google: SEO | Google: Web Search | Top News

Sponsored


About The Author: is Editor-In-Chief of Search Engine Land. His news career includes time spent in TV, radio, and print journalism. His web career continues to include a small number of SEO and social media consulting clients, as well as regular speaking engagements at marketing events around the U.S. He recently launched a site dedicated to Google Glass called Glass Almanac and also blogs at Small Business Search Marketing. Matt can be found on Twitter at @MattMcGee and/or on Google Plus. You can read Matt's disclosures on his personal blog.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://www.iselect.com.au derekdunne

    Hi I am seeing some weird results in a search for car insurance in Australia.

    The title in the search results for iSelect has car insurance at the start. However on site title doesn’t contain car insurance at the start? Is Google Combining H1 and title if the title tags are not long enough? Or is Google Broken? Is anyone else seeing this? I have screen grabs for anyone who wants to have a look at it.

  • http://www.michael-martinez.com/ Michael Martinez

    There was a similar curious report in the Google Webmaster forums this week where a page had a strange title (which, I discovered, matched the anchor text from an internal link on the site).

  • http://www.web-savvy-marketing.com rebeccagill

    The CNN website appears to be getting indexed regularly again, but I still have many client websites that are not.

    My hat is off to Google for quickly and publicly acknowledging the problem in their forum and on Twitter. Now we just need to get their “fix” spread to the rest of the website waiting to be spidered indexed.

  • http://www.dragonsearch.net DeidreDS

    Thank you, Matt, for this post. We’ve seen some indexing problems with out clients recently as well. Thought it was something on our end, but it’s good to know that the issue lies in Google. Thanks for helpful info!

  • angle45media

    Oh no! Google’s taking three hours to index a news article!

    Great article, but sometimes reading Search Engine Land is like listening to stock analysts for day traders; reporting on every .01 movement.

  • http://searchengineland.com/ Danny Sullivan

    Google, by its own standards, should be indexing fresh news articles within minutes — and doing the same for many other web pages.

    When it fails to do this, and for a period of days, it’s not a .01 movement. It’s a major failure of its indexing system, which it has touted quite loudly earlier this year.

  • LiveOnlineChat

    We have had a similar sort of problem and it has cut our traffic in half. We found that doing a daily traffic stats report, there was a sudden slump in traffic. Our site was number one for the search term “Free Chat Rooms No Registration”, but doing a Google search on this search term, we had been replaced by a site showing adult content. Checking other key search terms, some had be de-listed. Our community is not showing latest posts, neither is our forum, and yet our site is crawled constantly by Google robots. If you take a look at our homepage here http://www.liveonlinechat.net you will see that the search term above is dominant within the text using H1 tags and strong text. All our websites on our network also produce incoming links on an hourly basis to point that search term to our homepage. So, in all honesty, I really do think that Google is broken somewhere as the sites above us now do not concentrate on our main search terms but have risen above us.

  • http://www.bestereaders.net tammiej

    We’ve noticed a similar indexing problem, and like the poster above, have seen our traffic cut in half on http://www.BesteReaders.net, and our ranking drop a couple of spots. Fortunately, we found your article, and are planning to wait a couple more days to see if the problems persist or if we need to take corrective actions on our end.

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide