PPC

Google streamlines product listings via website crawl

Google is making it easier for merchants to pull in product listings from their sites through automated “website crawl” feeds in Merchant Center.

How it works. Google uses structured data markup on merchant websites to automatically extract up-to-date product information like titles, pricing, availability and images.

The big picture. The automated feeds aim to reduce the work required to keep product listings current as assortment and pricing changes – a challenge with traditional feed file uploads.

Why we care. This update could help reduce manual grunt work when it comes to updating product listings in the Merchant Center, although there may be a learning curve to ensure all products have the correct schema markup, which may not be the advertisers’ role in the marketing team.

The details. To use automated feeds, merchants must implement required schema.org markup for product details like:

  • Title [title]
  • Price [price]
  • Availability [availability]
  • Image [image_link]

Additional attributes like GTIN, brand, size and condition can further enrich listings when provided via markup or supplemental feeds.

What’s new. Some merchants now see an alternative “Website” feed experience that allows them to simply toggle automated product ingestion on/off in the Merchant Center.

Products get automatically added or removed from the Merchant Center based on the website crawl.
It complements other product data sources without duplication.

The challenges. Proper structured data implementation is key, as is managing crawl directives via sitemaps and robots.txt to avoid indexing issues.

What they’re saying. “Supplemental feeds provide additional information…and you can refresh [them] more often,” per Google’s automated feed documentation.

Between the lines. Automated feeds align with Google’s push toward entities and structured data as core to its modern “AI-first” approach across products.

First spotted. I first spotted this news on Thomas Eccel’s profile, but it was initially picked up by Georgi Zayakov:

Screenshot 2024 06 14 At 17.22.17

Screenshot 2024 06 17 At 18.18.08

Bottom line. By reducing manual feed labor, automated product ingestion could lower barriers for merchants to participate across Google’s growing array of surfaces like Search, Shopping, Images and more.

Explore more on

E-commerce Global Trade Item Number Google Google Merchant Center Google Shopping Image Robots.txt Schema.org Search engine optimization Structured data Web crawler Web scraping
Add Search Engine Land as a preferred source on Google

Editorial disclosure

Search Engine Land is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.

About the author

Anu Adegbola

Paid Media News Editor, Search Engine Land

Anu Adegbola has been Paid Media Editor of Search Engine Land since 2024. She covers paid search, paid social, retail media, video and more.

In 2008, Anu started her career delivering digital marketing campaigns (mostly but not exclusively Paid Search) by building strategies, maximising ROI, automating repetitive processes and bringing efficiency from every part of marketing departments through inspiring leadership both on agency, client and marketing tech side. Outside editing Search Engine Land article she is the founder of PPC networking event - PPC Live and host of weekly podcast PPC Live The Podcast.

She is also an international speaker with some of the stages she has presented on being SMX (US, UK, Munich, Berlin), Friends of Search (Amsterdam, NL), brightonSEO, The Marketing Meetup, HeroConf (PPC Hero), SearchLove, AdWorld Experience (Bologna, IT) and more.