• http://www.seo-theory.com/ Michael Martinez

    You’re just bound and determined to teach people how to do black hat SEO tactics. What’s up with that? I would REALLY appreciate you’re not teaching people how to scrape Websites. This is highly unethical behavior. This is a really shitty thing for you to be doing.

  • http://www.patrickwagner.com/ Patrick Wagner

    I agree with @Michael_Martinez:disqus – I want Google to be even harder on those who practice black hat SEO tactics.

  • TomSchmitz

    I think you’re painting with too broad a brush. If all web scraping were unethical then Google, Bing, SEOmoz, and Majestic SEO are all unethical companies. Of course that’s patently ridiculous. Part of being an SEO professional is data collection. It’s great to purchase or subscribe to tools. But sometimes, to get the data we want, it’s best to do it ourselves. I could do it by hand or outsource with Mechanical Turk or I can make my own tool in the form of a script or macro. The work being done and the end result is the same. Using a script does not make it black hat, intent and extent does. Yes this is powerful knowledge. Yes it can be misused. So can programs like Xenu and Screaming Frog, but we don’t lock those up and throw away the key.

  • TomSchmitz

    Would it surprise you to know I agree with your sentiment? The methods and procedures I present are neither good nor bad. It all depends on how you use them.

  • http://www.seo-theory.com/ Michael Martinez

    Google sends traffic. The rest of those companies ARE unethical. You’re just rationalizing your article with bullshit. There is nothing professional about scraping other people’s Websites for your own personal gain — especially when it’s for “Link Research”, which is THE most idiotic time-wasting practice people in this industry invest themselves in.

    People aren’t going to get links from these sites — they’re just going to eat up server resources and bandwidth without compensating the site owners for the inconvenience.

    Only human filth try to justify this kind of behavior.

  • http://www.seo-theory.com/ Michael Martinez

    No, the methods you’re presenting are bad. They are teaching bad SEO practices (time-wasting nonsense) that are couple with theft of bandwidth. Put a macro in your spreadsheet that looks for robots.txt and honors exclusion directives and I’ll lighten up on you.