Cloudflare lets sites block AI training without losing Google search traffic
Stay discoverable in search while disallowing AI training
Cloudflare's new Disallow AI Training setting publishes a no-training preference in robots.txt while keeping mixed-use crawlers like Googlebot and Bingbot indexing your site for search. Apple, Google, and Microsoft have committed to honor it. The move gives site owners granular control over search, training, and agent crawling, and sets the stage for controlling AI summaries next year.
Without proper controls, website owners have long faced a difficult tradeoff: allow your content to be used for AI training, or risk losing discoverability in search.