Why Google dropped Bing Discover pages from its index

Earlier this month, Google dropped a subset of Bing’s website from its index. The Bing Discover pages were completely dropped out of Google’s search results and resulted in Bing losing millions of pageviews from Google search.

Bing.com/discover not in Google’s index. Here is a screenshot of Google showing no results from Bing Discover for a site:bing.com/discover command.

How much traffic was lost? SEM Rush reports show a significant amount, just look at this chart showing the visibility drop:

Here is one of the tweets about this huge traffic loss:

Massive traffic. Roey Skif wrote an article back in April of this year about how Bing was able to use Google search to drive insane traffic to its pages. He showed how “Bing piggybacks on the low competition of Google’s image search and how to get huge amounts of organic traffic.” Then a few months later, Google seemed to have enough.

Google’s guidelines. Google’s webmaster guidelines say, “Use the robots.txt file on your web server to manage your crawling budget by preventing crawling of infinite spaces such as search result pages.” Google specifically has said for a long time they do not want search results in its search results. And Bing Discover was just that.

Google and Bing comments. Google would not send us a comment on the issue but Microsoft told us “this was a change that Google made so we cannot answer on their behalf.” Microsoft did not make any changes to Bing Discover that would lead to this change. I do believe they made changes after Google removed those pages from its index, but prior, no – no change.

Why we should care. If you go against Google’s guidelines, no matter how big or small you are, Google may eventually take action against the site. You can lose millions of visitors as a result. Read and understand Google’s webmaster guidelines, stay within those guidelines and do not risk your site from being penalized or removed from Google search.