Search Engine Land
  • SEO
    • > All SEO
    • > What Is SEO?
    • > SEO Periodic Table
    • > Google: SEO
    • > Bing SEO
    • > Google Algorithm Updates
  • PPC
    • > All PPC
    • > What is PPC?
    • > Google Ads
    • > Microsoft Ads
    • > The Periodic Tables of PPC
  • Focuses
    • > Local
    • > Commerce
    • > Shopify SEO Guide
    • > Content
    • > Email Marketing Periodic Table
    • > Social Media Marketing
    • > Analytics
    • > Search Engine Land Awards
    • > All Focuses
  • SMX
  • Webinars
  • Intelligence Reports
  • White Papers
  • About
    • > About Search Engine Land
    • > Newsletter
    • > Third Door Media
    • > Advertise

Processing...Please wait.

Search Engine Land » Channel » SEO » Google Is A Scraper Site, Says National Association Of Realtors

Google Is A Scraper Site, Says National Association Of Realtors

With support from the National Association of REALTORS® (NAR), the Indianapolis Metropolitan Board of REALTORS® (MIBOR) has forced some of its members to stop allowing certain MLS listings to be crawled and indexed by Google because Google (and other search engines) is considered a “scraper” site. This is the latest episode in a long-running battle […]

Matt McGee on May 12, 2009 at 2:09 pm

With support from the National Association of REALTORS® (NAR), the Indianapolis Metropolitan Board of REALTORS® (MIBOR) has forced some of its members to stop allowing certain MLS listings to be crawled and indexed by Google because Google (and other search engines) is considered a “scraper” site.

This is the latest episode in a long-running battle over who controls home listings that are part of the Multiple Listing Service, or MLS. Some of the affected real estate professionals plan to ask the NAR to change its opinion on search engines at its national convention this week.

Paula Henry is one of about 15 real estate agents affected by the NAR/MIBOR decision. Like many agents/brokers, she shows not only her own listings on her web site, but also listings from other agents and brokers who participate in a data sharing agreement through the Internet Data Exchange (IDX) system. There are strict rules over how listings from other agents may be displayed. In a post on AgentGenius.com, Paula explains the one that has led to Google’s classification as a scraper site:

Section 15.2.2 – participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent ‘scraping’ or other unauthorized accessing, reproduction, or use of the BLC database”

(BLC is the Indianapolis version of MLS.)

After apparently getting a complaint from another agent, MIBOR consulted with the NAR and the NAR confirmed that the rule above applies to search engines. On March 27, MIBOR sent Red Door Real Estate (where Paula Henry does business) a cease-and-desist letter detailing two issues:

  1. “…the National Association of REALTORS® (NAR) is in agreement with our interpretation of the policy that the above described practice of ‘indexing your Web site’ as you have called it, is a method of scraping or reproducing the data”
  2. “Under IDX policy … participants have no authority to advertise those listings [from other participants] in any other way, including Internet search engines”

The C&D letter asks Red Door to either use robots.txt (actually referred to as “rebot.txt” in the letter) to block search engines from crawling the listings on their sites that belong to other agents/brokers, or to remove the non-Red Door listings altogether. In her blog post, Paula writes that additional demands were made in April requiring Red Door to remove the MLS number, street address, and other listing data from the Page Titles and Meta Description tags of pages on their web sites.

Not mentioned in all of this is the fact that Google is practically running its own national MLS database, with the same kind of search and sort options (Price, Beds, Baths, Area, etc.) in Google Maps that you’d expect to find on full-fledged real estate sites.

goog-real-estate

Real estate brokers and agents submit their own listings to Google (or they have a third party do it), so the rules about how other agents’ listings are displayed don’t apply. And it’s not just about Google; Yahoo Real Estate also gets listings from agents and brokers, and places them on highly-optimized pages that, in my experience, often rank well for popular real estate search terms.

As for Google being a scraper site, Paula Henry will be one of two real estate agents speaking to the NAR on Thursday. She says they’ll ask the organization to review and change its policy so that Google (and other search engines) is allowed to index all real estate listings from the IDX system, no matter whose web site they appear on.


Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.


New on Search Engine Land

    More FAQ rich results being displayed in Google Search

    Webinar: Benchmark your social media performance for a competitive edge

    Google releases May 2022 broad core update

    Spotify, Meta update political ad offerings for 2022 election cycle

    Take web hosting to the (NVMe) extreme

About The Author

Matt McGee
Matt McGee joined Third Door Media as a writer/reporter/editor in September 2008. He served as Editor-In-Chief from January 2013 until his departure in July 2017. He can be found on Twitter at @MattMcGee.

Related Topics

SEO

Get the daily newsletter search marketers rely on.

Processing...Please wait.

See terms.

ATTEND OUR EVENTS

The SMX Conference logo.

Learn actionable search marketing tactics that can help you drive more traffic, leads, and revenue.

March 8-9, 2022: Master Classes (virtual)

June 14-15, 2022: SMX Advanced (virtual)

November 15-16, 2022: SMX Next (virtual)

Learn More About Our SMX Events

The MarTech Conference logo.

Discover time-saving technologies and actionable tactics that can help you overcome crucial marketing challenges.

Start Discovering Now: Spring (virtual)

September 28-29, 2022: Fall (virtual)

Learn More About Our MarTech Events

Webinars

Take a Crawl, Walk, Run Approach to Multi-Channel ABM

Content Comes First: Transform Your Operations With DAM

Dominate Your Competition with Google Auction Insights and Search Intelligence

See More Webinars

Intelligence Reports

Enterprise SEO Platforms: A Marketer’s Guide

Enterprise Identity Resolution Platforms

Email Marketing Platforms: A Marketer’s Guide

Enterprise Sales Enablement Platforms: A Marketer’s Guide

Enterprise Digital Experience Platforms: A Marketer’s Guide

Enterprise Call Analytics Platforms: A Marketer’s Guide

See More Intelligence Reports

White Papers

Reputation Management For Healthcare Organizations

Unlock the App Marketing Potential of QR Codes

Realising the power of virtual events for demand generation

The Progressive Marketer’s Ultimate Events Strategy 2022 Worksheet

CMO Guide: How to Plan Smart and Pivot Fast

See More Whitepapers

Receive daily search news and analysis.

Processing...Please wait.

Topics

  • SEO
  • PPC

Our Events

  • Search Marketing Expo - SMX
  • MarTech

About

  • About Us
  • Contact
  • Privacy
  • Marketing Opportunities
  • Staff

Follow Us

  • Facebook
  • Twitter
  • LinkedIn
  • Newsletters
  • RSS
  • Youtube

© 2022 Third Door Media, Inc. All rights reserved.