Publishers want Congress to stop AI crawlers from hiding

More than 300 news publishing executives are heading to Washington, D.C., today. They plan to urge Congress to crack down on AI crawlers that disguise themselves while scraping websites.

Executives from Condé Nast, Hearst Magazines, USA Today Co., and The Seattle Times are expected to meet with lawmakers and their staff. They will focus on the Stealth Bot Prohibition Act, which would require automated crawlers to identify themselves and disclose their purpose.

What the bill would do. The Stealth Bot Prohibition Act, introduced in Congress in July, targets bots that hide their identity or purpose. That includes crawlers that pose as humans or use residential IP proxies and scraping services to bypass website controls. The goal is to give website owners a clearer way to identify and block unwanted automated traffic.

Publishers push for control. News/Media Alliance CEO Danielle Coffey told Digiday that requiring crawlers to identify themselves could help publishers control access to their content. It could also help them negotiate licensing agreements with AI companies.

• USA Today Co. CEO Mike Reed said publishers deserve “attribution, control, and fair compensation” when their journalism contributes to AI products.

Bot traffic keeps growing. Bots now account for 64% of internet traffic, according to Cloudflare Radar.

• TollBit detected more than 22 billion AI bot scrapes across the sites it analyzed in the first half of 2026. AI bot traffic on TollBit-tracked publisher sites also surged in 2025. The ratio of AI bot visits to human visits rose from about 1 in 200 in the first quarter to 1 in 31 in the fourth quarter.
• The lobbying push follows New York’s passage of a similar Stealth Crawler Prohibition Act in June. The federal proposal would go further. It would cover websites and digital platforms generally, not just news publishers.

Why we care. Blocking AI crawlers only works if you can identify them. If crawlers pose as human visitors or hide their identity, robots.txt rules and other technical controls become far less useful. Website owners have less control over which AI systems can access their content.