An in-depth guide to technical SEO optimization
If your technical SEO is poor, search engines will struggle to discover, crawl, and index your site quickly and correctly
Topic: Web scraping
Google Search demoting content that is starkly different from the main content of the site
A Google spokesperson told Glenn Gabe, "Our systems aim to understand if a section of a site is independent or starkly different from the main content of the site. This helps us surface the most useful information from a range of sites."
Decoding Googlebot crawl stats data in Google Search Console
Dive into the lesser-known Crawl Stats reports, plus how to monitor Googlebot activity and identify actionable insights for enterprise sites.
Is Google using Chrome for mobile-first indexing and user signals?
SEO expert Cindy Krum explains how she believes Google is using Chrome user data for rendering, indexing, ranking and beyond.
How to fix ‘Crawled – Currently not indexed’ error in Google Search Console
Uncover the reasons behind the 'Crawled – Currently not indexed' error in Google Search Console and solutions to get your pages indexed.
Reddit CEO to Microsoft and AI search engines: Pay for our content
Reddit CEO Steve Huffman does not believe all web content – especially theirs – is free for AI models and search engines to crawl and use.
Microsoft AI CEO: Web content is ‘freeware’
Mustafa Suleyman believes almost all web content can be used for AI training unless explicitly restricted by the creator.
Google streamlines product listings via website crawl
Google is making it easier for merchants to surface product listings across its surfaces through automated 'website crawl' feeds.
Driving traffic to gated content and paywalled sites: SEO tips + examples
Explore SEO tactics to effectively market premium content and drive site traffic while preserving the value of your gated assets.
Google Search fixes issues with Site names not appearing for internal pages
This has been an issue since last December but Google said it still needs to reprocess your page for the issue to be resolved.
Google won’t index sites that do not work on mobile devices after July 5
if your website is not accessible using a mobile device, then Google will no longer index it and rank it.
WordPress 6.5 gains lastmod date for sitemaps files
Google and Bing both encourage sites to include accurate lastmod dates within sitemap files.
Structure, consume, learn and retire: Google’s pattern of learning
Google's cyclical process of giving us capabilities only to later remove them, seems to be a core part of its strategy.
3 underutilized Google Search Console reports for diagnosing traffic drops
Decode your website's traffic declines with GSC's detailed reports, covering device segmentation, search appearance and crawl stats insights.
3 reasons not to block GPTBot from crawling your site
There’s much more to gain than lose by embracing GPTBot. Explore three primary benefits of adopting OpenAI’s bot technology.
Googlebot crawl rate tool is now gone
You can no longer set your desired crawl rate within Google Search Console.
Google says search result snippets primarily come from page content, not other sources
Google removed references to rich results the snippet control help document.
Number of websites blocking Google-Extended jump 180%
The New York Times, Yelp and 22 Condé Nast properties are among those who have chosen to block Google-Extended so far.
Googlebot crawl rate tool in Search Console is going away
The tool is going away on January 8, 2024 because Google says it is no longer useful.
Google Search Console adds robots.txt report
Google has sunset the robots.txt tester.
Google says mobile-first indexing is complete after almost 7 years
Google said it will turn off the indexing crawler information in the settings page in Search Console.
Report: Generative AI violates copyright law, rights of news publishers
News publishers are concerned that AI answers will replace Search traffic, leading to less revenue, lost jobs and other damages.
How SEOs can detect and address user data leaks
Discover the hidden dangers of user data exposure and why SEOs play a crucial role in data safeguarding and privacy protection.
26% of the top 100 websites are now blocking GPTBot
The number of popular websites blocking ChatGPT has increased 250% in the past month. Pinterest, Indeed and more big brands now block GPTBot.
Dozens of big brands have blocked GPTBot, OpenAI’s new web crawler
At least 15% of the top 100 websites and 7% of the top 1,000 websites are blocking GPTBot, a new analysis finds.
GPTBot: OpenAI releases new web crawler
You can now prevent OpenAI's ChatGPT from accessing your website, or parts of it, using robots.txt.
5 free SEO tools and plugins to try
Use these five handy and free tools and plugins for your SEO efforts to save time and become more efficient.
Daily Mail ‘set to sue’ Google over Bard copyright issues
Google is accused of illegally taking hundreds of thousands of the newspaper's articles to train its ChatGPT rival, Bard.
Crawlers, search engines and the sleaze of generative AI companies
AI companies see the openness of the web as permitting large-scale crawling to obtain training data. So what should publishers do?
A guide to diagnosing common JavaScript SEO issues
Learn how to spot SEO issues in JavaScript-powered sites and what you need to know when it comes to rendering.
Google has done its last mobile-first indexing switch today, a six-year process
Any site not yet moved over will remain on desktop-indexing going forward.
Search the 15.7 million websites in Google’s C4 dataset
Now you can find out whether data from your website helped train large language models that help power Google Bard.
5 Python scripts for automating SEO tasks
Looking to streamline your SEO workflows? These five scripts are great starting points for exploring the possibilities of Python in SEO.
How to master Google News optimization to boost content visibility and traffic
Google News-driven results send over 24 billion clicks per month to publishers. Here's how to claim your share.
Google warns against using 403 or 404 status codes for Googlebot crawl rate limiting
Instead, Google recommends you use Google Search Console's crawl rate feature.
Missing links: Where are sources in Google’s AI search results?
Forget zero-click. Now we have zero-link search results from Google. Is this an act of war on publishers?
New Bingbot user-agent now fully live
After a couple years of testing, Microsoft has now stopped using the old BingBot user agent.
Microsoft Bing to rely more on lastmod date in XML sitemap files for crawling
Microsoft also released some interesting stats on lastmod dates in XML sitemap files.
This day in search marketing history: January 16
Google explains crawl budget, plus: website outages and blackouts the right way, Google doing 1 trillion searches, and more.
Changing URLs for SEO reasons: When is it OK?
Google’s John Mueller warns not to change URLs for SEO reasons. Is he right? It depends, of course. Here are some exceptions.