Robots.txt is not the answer: Proposing a new meta tag for LLM/AI
Using robots.txt for managing data usage in LLMs is the wrong approach in this new age of generative AI products. Here's why.
Identify traffic-stealing competitors
Find untapped keyword opportunities
Create content that ranks higher
Fili is a renowned technical SEO expert, ex-Google engineer and was a senior technical lead in the Google Search Quality team. At SearchBrothers he offers SEO consulting services with SEO audits, SEO workshops and successfully recovers websites from Google penalties. Fili is also a frequent speaker at SMX and other online marketing events.
Using robots.txt for managing data usage in LLMs is the wrong approach in this new age of generative AI products. Here's why.
Fili Wiese | Jul 18, 2023 at 10:00 am ETAudit your website and check all the technical signals to understand how search engine algorithms are handling your content.
Fili Wiese | Jan 3, 2020 at 12:19 pm ETBing's Frédéric Dubut and ex-Googler Fili Wiese covered a wide range of questions in Seattle and will be sharing more insights at SMX East.
Fili Wiese | Oct 31, 2019 at 8:00 am ETSMX London attendees asked SEO expert Fili Wiese a wide-ranging number of questions about penalties, indexing, crawling and more.
Fili Wiese | Sep 10, 2019 at 2:49 pm ET