WorkJuly 1, 2026via TechCrunch AI
Cloudflare’s new policy pushes AI companies to pay for publishers’ content
Why it matters
Cloudflare's infrastructure-level policy creates the first major mechanism forcing AI companies to negotiate content licensing rather than scrape freely. This shifts the economics of AI training data and could reshape how foundational models source content.
Key signals
- September 15, 2026 deadline for AI companies to separate search crawlers from training crawlers
- Default blocking of AI training crawlers on publisher sites via Cloudflare
- Policy applies to AI training and agents specifically
- Infrastructure provider (Cloudflare) acting as enforcer for publisher IP rights
- Affects crawler behavior across web-scale distribution
The hook
September 15 deadline. Cloudflare just forced the AI industry's hand on publisher payouts.
Cloudflare is giving AI companies until September 15 to separate web crawlers used for search from those used for AI training and agents, or risk being blocked by default on many publisher sites.