Web Defaults

Cloudflare’s new web defaults, effective September 15, will automatically block AI agent and training crawlers in real-time on ad-supported pages. This move aims to differentiate bots from human users and establish clearer rules for the “agentic internet.” Publishers and AI developers must navigate these changes, with potential shifts towards negotiated access and new monetization models like “pay per use” as free web access for AI becomes less common.