Publishers push Common Crawl to stop collecting content for AI training
Could AI lose a key source of training data? Major publishers want Common Crawl to stop collecting and sharing their content.
Search Engine Land is your source for news publishers and media organizations. You’ll find a variety of up-to-date and authoritative resources, including the latest news, tactic-rich tutorials, and the latest data to help you work smarter and make better decisions.
What you read here is written by our experienced in-house team, and subject matter experts from leading brands and agencies.
Don’t miss out! Get the Search Engine Land newsletter delivered daily to your inbox. Subscribe for free!
Could AI lose a key source of training data? Major publishers want Common Crawl to stop collecting and sharing their content.
After years of Google algorithm updates and huge SERP changes, Condé Nast now plans its business as if search traffic barely exists.
The first systematic analysis of Google's invitation-only enhanced publisher profiles on Discover, based on monitoring 46,926 publishers.
Clicks are declining, but utility content still builds visibility and trust. Here’s how to adapt your newsroom strategy for AI search.
Google Discover is a personalized content feed that surfaces articles without a search. Learn how it works, where it appears, and how to optimize content for Discover traffic.
AI fetcher bots siphon traffic and revenue from publishers, delivering real-time answers that bypass your site and cut referral visits.
Publishers face new risk as Google tests AI-written titles in Search, rewriting tone and intent to better match queries and boost engagement.
Google Search pageviews fell 34% year over year and Discover dropped 15%, as ChatGPT referrals rose 200% but remained under 1% of traffic.
Publishers are losing Google organic search traffic, but breaking news traffic jumped 103% and Discover clicks are surging.
New SDK findings reveal why some pages never reach ranking, how pCTR predicts clicks, and why images and recency drive Discover traffic.
Google now blends social posts, short-form video, and AI into news results. Adapt your strategy for visibility across SERP features and Discover.
AI answers are taking clicks from publishers. Media companies warn that Google Overviews and chat experiences are speeding up traffic loss.
A deeper look at the traffic losses, revenue shifts, and emerging models reshaping publishing and search in an AI-first era.
Discover the data-backed insights that show where SEO efforts actually pay off as Google weaves more AI into search.
Reddit debuts Pro tools to help publishers track stories, auto-import content, and tap into communities as referral traffic shifts.
Nearly every top U.S. news site lost traffic in July. From paywalls to shifting habits, publishers saw single or double-digit declines.
Google Discover now leads publisher referrals as direct and social traffic slide, keeping news sites reliant on Google, data shows.
Google Discover is rolling out to desktop, which could be a big traffic win for publishers hurt by AI Overviews and generative AI.
Google Search Liaison Danny Sullivan addresses concerns around freelance writers, writing teams, manual actions, and more.
I read the news today, oh boy. AI search engines give wrong answers 60% of the time and often cite fabricated or broken URLs.
AI bot scraping doubled from Q3 to Q4 of 2024. But the click-through rates from AI search remain abysmal – at less than 1%
This comes after Google announced in April 2024 that Publisher Center will stop allowing you to add publications manually.
The legacy media brand is shifting focus from reach metrics to engagement, such as YouTube watch time.
OpenAI's Varun Shetty also said publishers will be fairly compensated through 'significant incremental traffic from new audiences.'
Instead, Google will automatically add eligible publishers and will eventually remove those publishers not eligible.
The search engine threatened to stop serving news in response to a new law forcing tech companies to pay for news content.
News publishers are concerned that AI answers will replace Search traffic, leading to less revenue, lost jobs and other damages.
The Washington Post has shared a document outlining 'proven strategies and guidelines' around SEO and optimal site practices.
Plus, Google re-announced the Google Reader Revenue Manager.
Meta strikes back at 'flawed' legislation that would force it to pay news publishers to use their content on Facebook and Instagram.
Instead, Google says block your syndicated content from being indexed.
Reader Revenue Manager publishers can now measure traffic from Google Search queries and fix structured data.
Google News-driven results send over 24 billion clicks per month to publishers. Here's how to claim your share.Â
If you are not seeing traffic from Google for new content you published, you are not alone.
Google also has re-launched Google News in Spain.
Find out why, and how, it happened – and whether that Discover visibility will remain over the long-term, in this case study.
Ryan Kellett, VP of Audience at Axios, shares how SEO helps this news site compete against the world's biggest news publishers.
The Opinions section contextualizes news-related listings for users, which may help reduce misinformation.
Google search may show local news stories more often in Top Stories. Google News Showcase also gets updates.
Bing News PubHub launched in 2016 at its own location but has now been migrated into Bing Webmaster Tools.