Yandex, The Popular Russian Search Engine, Begins Crawling JavaScript & CSS

Yandex warns webmasters to not block their JavaScript and CSS files because they are starting to crawl those resources for indexing.

Chat with SearchBot

yandex-flag-ss-1920

Yandex, the large Russian search engine, announced (in Russian) that they are now crawling, as a test, through CSS and JavaScript files. The purpose is for Yandex to better understand web pages and the content on those web pages that are designed for more modern browsers. They are starting to test this crawl process on a small subset of known pages, but they plan to expand it later to a larger set of the index.

Because of this effort, Yandex is informing webmasters to make sure their CSS and JavaScript files are not blocking the Yandex crawler. In the same way, Google and Bing have recommended you do not block their bots from crawling your CSS and JavaScript files and resources.

The Russian Search blog summarized this also in English, since they speak fluent Russian, and I do not. They added, “this change in Yandex’s indexing routines can possibly have effects on Yandex SEO, at least in the long run.”


About the author

Barry Schwartz
Staff
Barry Schwartz is a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.

Get the must-read newsletter for search marketers.