Free🇭🇰🇨🇳
Chinese evaluation suite with 13,948 multiple-choice questions in 52 disciplines and four difficulty levels for foundation models.
MTEB overview and key facts (October 2026)
Massive benchmark and toolbox for embedding and retrieval models, with more than 1,000 tasks and a public leaderboard on Hugging Face.
Free
What is MTEB?
MTEB is the standard benchmark and toolbox for embedding models, which power semantic search and retrieval-augmented generation. Its leaderboard helps teams choose an embedding model for their language and task.
MTEB bundles more than 1,000 tasks in over 1,000 languages across retrieval, classification, clustering, semantic similarity and pair classification. Named benchmark suites cover English, multilingual and European content as well as domains such as law, medicine and code. It started with text and now includes image and audio. Installation works via pip or uv, and the interactive leaderboard runs on Hugging Face Spaces. The project is Apache 2.0 licensed and maintained by many contributors.
MTEB AI features and use cases
- Chooses an embedding model for searchTeams building an internal search or RAG system compare embedding models on retrieval tasks in their language. A better embedding model finds more relevant documents, which improves the answers of the whole system.
- Tests a model on own task selectionWith the Python API or CLI, teams run only the tasks that match their use case, such as retrieval in European languages or legal text. Results are more relevant than the overall score.
- Evaluates image and audio embeddingsMTEB now includes image, audio and multimodal tasks next to text. Teams planning search over product photos, scanned documents or recordings can test embedding models on those modalities with the same toolbox.
What sets MTEB apart
- Benchmark suites for single language groups and domains such as law, medicine and code.
- Covers text, image and audio embeddings in one framework.
Where MTEB is strongest
Available internationally
MTEB covers more than 1,000 languages and is maintained as an open community project used worldwide.
Questions to ask before you buy MTEB
Leaderboard scores are averages; run the tasks closest to your documents and languages before deciding. Some models may have been trained on public MTEB data. Running large task sets needs GPU time for bigger models. Also check the license of the embedding model you pick, since MTEB only measures quality.
MTEB reviews
No reviews yet. Be the first to share how your team uses it.
Every review shows its verification level. We read each one before it goes live. How we check reviews
Questions about MTEB
What does MTEB measure?
It measures how well embedding models perform on retrieval, classification, clustering, semantic similarity and pair classification across many languages and domains.
Where is the leaderboard?
The interactive leaderboard runs as a Hugging Face Space under mteb/leaderboard.
How do we run it?
Install mteb with pip or uv, select a model and tasks, and run the evaluation in Python or with the command line tool.
MTEB update historyno updates yet
No changes recorded for MTEB yet.
Changes the vendor made and confirmed user reports, each checked against the vendor's own pages. We review every tool monthly.
Alternatives to MTEB
Tools in the same category (Domain benchmarks) with the most overlapping markets.
Free🇺🇸
Collaborative benchmark of 162 English legal reasoning tasks from 40 contributors, covering contracts, statutes and court opinions.
Free🇨🇳
Open-source LLM evaluation platform from Shanghai with CompassRank leaderboards, the CompassHub benchmark catalog and VLMEvalKit.
Free🇭🇰
Benchmark of 369 real computer tasks on Ubuntu, Windows and macOS that tests multimodal agents operating desktop and web apps.