Platforms

New SEO Tools: Robots.txt Builder, Term Targeting & Crawl Test

Three new SEO tools came out this week that are worth a look. Dave Naylor released a robots.txt builder tool, while SEOmoz released term targeting and crawl test tools. Some more about the tools, below:


The Robots.txt Builder Tool
David Naylor, the creator of the tool, explains that one of his clients accidentally blocked all robots to his site and found himself out of the search engines. So he decided to build this tool to make it easy for webmasters to make valid robots.txt files.

Term Targeting
SEOmoz created this tool for their premium members and say they may open it up later to all users. The tool “analyzes the content of a given page and extracts the terms and phrases that appear to be targeted at search engines.”

Crawl Test Tool
This tool from SEOmoz, previously only available to premium members, now is free for the public. They say the tool can “test how accessible your site is to search engines and can help you quickly diagnose potential crawling issues and give you an overview of your site’s search friendliness.”

Explore more on

Dave Naylor Moz Professional Baking Robots.txt Search engine optimization Web crawler
Add Search Engine Land as a preferred source on Google

Editorial disclosure

Search Engine Land is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.

About the author

Barry Schwartz
Barry Schwartz is a technologist and a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics.

In 2019, Barry was awarded the Outstanding Community Services Award from Search Engine Land, in 2018 he was awarded the US Search Awards the "US Search Personality Of The Year," you can learn more over here and in 2023 he was listed as a top 50 most influential PPCer by Marketing O'Clock.

Barry can be followed on X here and you can learn more about Barry Schwartz over here or on his personal site.