🚨New paper alert! 🚨
We've conducted the first detailed audit of consent and permissions for AI pretraining data, from 14,000+ of the most popular domains on the web.
Data consent is changing rapidly and we need to understand it!
Read more here: ⬇️
✨New Preprint ✨ How are shifting norms on the web impacting AI?
We find:
📉 A rapid decline in the consenting data commons (the web)
⚖️ Differing access to data by company, due to crawling restrictions (e.g.🔻26% OpenAI, 🔻13% Anthropic)
⛔️ Robots.txt preference protocols
