Common Crawl Foundation

@CommonCrawl

Common Crawl is a non-profit foundation dedicated to the Open Web.

San Francisco, CA

Common Crawl Foundation
@CommonCrawl
16h
Getting Started with Common Crawl Data on Hugging Face We made parts of our crawl archive available experimentally on a Hugging Face Storage Bucket. This blog post explains how to use the data on Hugging Face, with tools and examples on which you can build.
1
Common Crawl Foundation
@CommonCrawl
Sep 15
CommonsDB, ISCC, Spaghetti, and Meatballs Notes from the CommonsDB final conference in Alicante: a registry of 6.5 million open works keyed on ISCC, and the EUIPO's push for federated copyright infrastructure.
1
Common Crawl Foundation
@CommonCrawl
Sep 15
Bodhium Labs: From web pages to a question corpus FineQuestions draws on 110 snapshots of FineWeb, the filtered English-language corpus derived from Common Crawl, spanning 2013–2025. Extraction yielded roughly 13.4 billion raw questions filling 22 TB. Normalization and
2
Common Crawl Foundation
@CommonCrawl
Sep 11
How Common Crawl and AWS Open Data built the foundation for the AI revolution by Rich Skrenta and Dr. Beryl Rabindran
How Common Crawl and AWS Open Data built the foundation for the AI revolution | Amazon Web Services
From aws.amazon.com
2
Common Crawl Foundation
@CommonCrawl
Sep 8
Web Graph Embeddings: An Experimental Dataset Release We are publishing an experimental dataset of graph embeddings built from the Common Crawl Web Graph: a 128-dimensional vector for each of 52.9 million web hosts, learned from hyperlinks alone, along with two interactive
3