The question this layer is asking
When AI takes your content to train on and to generate answers, can you set conditions — or even charge? Since 2025 a whole layer of standards has competed to answer this. The main players:
| Approach | Role | Status |
|---|---|---|
| RSL 1.0 | License terms embedded in robots.txt + pay-per-crawl/inference | Self-declared standard, 1500+ brands; ~zero AI compliance |
| Cloudflare Content Signals | robots.txt flags for search/ai-input/ai-train use | Product, 3.8M domains; advisory (voluntary) |
| Cloudflare Pay-Per-Crawl | Charge AI crawlers per request via HTTP 402 | Product, partial beta |
| TollBit / ProRata | Two-sided paid marketplaces with real money flowing | Production, still small |
| IAB CoMP | A commercial agreement before crawling | 2026/3 draft, not finalized |
The reality: many approaches, little compliance
The problem here isn’t a lack of options — it’s that they coexist, compete, and AI-side compliance is near zero. RSL calls itself a standard but AI vendors ignore it; Content Signals is advisory; the real money flows through marketplaces like TollBit and ProRata, but at modest scale. None is a finalized, enforceable standard.
For small businesses: the point isn’t charging, it’s not blocking the wrong thing
The players actually getting paid by AI are Reddit- and major-media-scale. For the vast majority of sites, what this layer demands now isn’t “how do I charge” but “while setting licensing/blocking, don’t accidentally block yourself out of AI answers” (see Block the wrong bot).
So should you act
Track it, don’t invest. Once the layer consolidates and AI actually starts complying, then talk monetization. For now, energy spent on the site-health that gets you cited beats energy spent on blocking and charging.
