Free🇺🇸
Hugging Face pretraining dataset of 1.3 trillion tokens of educational web pages, selected from Common Crawl by an AI quality classifier.
Common Corpus overview and key facts (October 2026)
Open pretraining corpus of 2.27 trillion tokens from Pleias in Paris, containing only uncopyrighted or freely licensed documents.
Free
What is Common Corpus?
Common Corpus is an open dataset for training language models that contains only public domain or freely licensed text, with license information for every document. It suits teams that need traceable training data.
The corpus has six collections, among them OpenCulture with public domain books and newspapers, OpenGovernment with financial and legal documents such as SEC filings and Europarl, OpenSource with code, OpenScience with papers and OpenWeb with content such as Wikipedia. German, Spanish, Italian, Polish, Greek and Latin each exceed 10 billion tokens. Pleias, a company at Station F in Paris, created it with partners; the work was presented as an oral paper at ICLR 2026.
Common Corpus AI features and use cases
- Trains models on licensed data onlyTeams that must avoid copyrighted training material can train or fine-tune models on text that is public domain or openly licensed. This lowers legal risk for models used in products.
- Filters by license typeEvery document carries license and provenance metadata. Legal or compliance teams can restrict training to public domain works or attribution-only licenses and document that choice. This record helps when documenting training data for customers or regulators.
- Adds legal and financial textThe OpenGovernment collection holds hundreds of billions of tokens of regulatory, financial and legal documents. Models for finance or legal work gain domain vocabulary from sources such as SEC and Europarl.
What sets Common Corpus apart
- License and provenance recorded for each individual document as well as for the dataset.
- Built by a Paris-based company; the card states it meets the AI Act Code of Conduct requirements for data transparency.
Where Common Corpus is strongest
France Available internationally
Pleias is based at Station F in Paris, and English and French make up most of the corpus.
Questions to ask before you buy Common Corpus
Check whether attribution requirements of the individual licenses fit your model documentation; filter out licenses you do not want. Public domain and government documents can still contain personal data, so assess data protection separately. The full corpus is very large, so start with the collections you need.
Using Common Corpus in the EU: GDPR and AI Act
- Data locationNot published by the vendor.
- Data processing agreementNot published by the vendor: ask for one (Art. 28 GDPR).
- EU AI ActCheck whether the AI prepares decisions about people. If not, the high-risk rules usually do not apply. The high-risk obligations apply from December 2, 2027 (Digital Omnibus).
General information with sources, not legal advice.
Common Corpus reviews
No reviews yet. Be the first to share how your team uses it.
Every review shows its verification level. We read each one before it goes live. How we check reviews
Questions about Common Corpus
May we use Common Corpus commercially?
According to the dataset card, all data is either uncopyrighted or freely licensed and may be used for commercial and non-commercial purposes. License terms such as attribution still apply per document.
How much German text does it contain?
The card lists German among the languages with more than 10 billion tokens. Most of the corpus is English and French.
Who maintains it?
Pleias, a company based at Station F in Paris, created the corpus with several partners and continues to expand it, currently with more Asian and Arabic documents.
Common Corpus update historyno updates yet
No changes recorded for Common Corpus yet.
Changes the vendor made and confirmed user reports, each checked against the vendor's own pages. We review every tool monthly.
Alternatives to Common Corpus
Tools in the same category (Training data) with the most overlapping markets.
Free🇺🇸
Multilingual pretraining dataset from Hugging Face covering more than 1,000 languages, including a large German subset.
Contact for pricing · Free planAvailable internationally
Free, open-source prompt library, formerly Awesome ChatGPT Prompts, available as a website, MCP server or self-hosted company library.
Free🇮🇳
Pretraining dataset of 251 billion tokens in 22 Indian languages from AI4Bharat at IIT Madras, with verified, filtered and translated parts.