Publishers push Common Crawl to stop collecting content for AI training
Could AI lose a key source of training data? Major publishers want Common Crawl to stop collecting and sharing their content.
Topic: CCBot
The 5-layer framework for measuring GEO performance
AI visibility dashboards can’t answer the ROI question on their own. Here’s how to build a more credible GEO measurement model.
Your managed WordPress might be blocking AI bots and you can’t see it
Your SEO tools show nothing wrong. But platform-level limits may be quietly blocking AI systems from accessing and citing your content.
Why log file analysis matters for AI crawlers and search visibility
Track how AI crawlers access your site, identify crawl gaps, and understand what content gets missed using log file data.
SEO in 2026: Higher standards, AI influence, and a web still catching up
Technical SEO is getting easier by default — but decisions around bots, LLMs.txt, and structured data are becoming more complex.
1/3rd of publishers say they will block Google Search AI-generative features like AI Overviews
42% said they would not block Google and 25% are still unsure.
What the AI Visibility Index tells us about LLMs & search
Discover how LLM visibility is reshaping SEO. Learn to track brand mentions in AI responses, new metrics beyond rankings, and optimization strategies for 2025.
Seize the AI search opportunity with site health
AI search is redefining visibility. Proper site health ensures your brand is ready to be discovered, understood, and recommended by AI systems.
Log file analysis for SEO: Find crawl issues & fix them fast
Unlock hidden SEO insights with log file analysis. Learn how to track crawler behavior, spot indexing problems, and optimize your site for search engines.
Dave Davies on LLM content SEO shortcuts, attribution loss, and agentic AI
SMX Advanced keynote speaker Dave Davies shares insights on LLM pitfalls, weird tech gotchas, and why generative engines are the future.
12 new KPIs for the generative AI search era
We’ve spent years tracking clicks and rankings. But in the age of LLMs and AI search, are we still measuring what matters?
Inside the AI-powered retrieval stack – and how to win in it
The page is dead. Long live the stack. Here's how vector databases, embeddings, and Reciprocal Rank Fusion have changed the search stack.
AI optimization: How to optimize your content for AI search and agents
To be visible, optimize your site with clean HTML, metadata, fast responses, and bot-friendly configurations.
Number of websites blocking Google-Extended jump 180%
The New York Times, Yelp and 22 Condé Nast properties are among those who have chosen to block Google-Extended so far.
Report: Generative AI violates copyright law, rights of news publishers
News publishers are concerned that AI answers will replace Search traffic, leading to less revenue, lost jobs and other damages.
26% of the top 100 websites are now blocking GPTBot
The number of popular websites blocking ChatGPT has increased 250% in the past month. Pinterest, Indeed and more big brands now block GPTBot.
Dozens of big brands have blocked GPTBot, OpenAI’s new web crawler
At least 15% of the top 100 websites and 7% of the top 1,000 websites are blocking GPTBot, a new analysis finds.