Freemium🇺🇸
Independent benchmarking of AI models and API providers on intelligence, speed, latency and price, with image, video and coding arenas.
LiveBench overview and key facts (October 2026)
LLM benchmark that adds new questions regularly and scores answers against objective ground truth, without an LLM judge.
Free
What is LiveBench?
LiveBench is an open benchmark for language models designed to limit test set contamination. It regularly adds new questions and scores answers automatically against objective solutions.
LiveBench releases new questions over time and bases others on recently published datasets, arXiv papers, news articles and movie synopses. Each question has a verifiable answer, so hard questions can be scored without a language model as judge. The benchmark covers several task categories, including coding and agentic coding that needs Docker. It was a spotlight paper at ICLR 2025, the leaderboard is at livebench.ai, and the repository's license file carries the Apache License 2.0.
LiveBench AI features and use cases
- Compares models on fresh questionsBecause questions are refreshed, scores are less affected by models that saw the test in training. Teams get a more current comparison than older static benchmarks. Scores stay meaningful for longer.
- Runs the benchmark on own endpointsThe code can call any model behind an OpenAI-compatible API. IT teams can test a self-hosted or fine-tuned model with the same questions as the public leaderboard. Results are directly comparable.
- Checks coding agentsAgentic coding tasks run in Docker and are scored automatically. Development teams see how well a model completes code tasks end to end. Docker must be installed, which the tool checks before these tasks start.
What sets LiveBench apart
- Scoring against objective answers without an LLM judge.
- Regularly refreshed questions to limit contamination.
Where LiveBench is strongest
Available internationally
LiveBench is an open benchmark with a public leaderboard that is used worldwide.
Questions to ask before you buy LiveBench
Running the full benchmark costs API tokens for every model tested, and agentic coding tasks need Docker. The README notes that local model inference is unmaintained, so serve local models behind an OpenAI-compatible API. Questions are in English. The results say little about German-language performance.
LiveBench reviews
No reviews yet. Be the first to share how your team uses it.
Every review shows its verification level. We read each one before it goes live. How we check reviews
Questions about LiveBench
How does LiveBench limit contamination?
It releases new questions over time and bases questions on recent datasets, arXiv papers, news articles and movie synopses.
Does it use an LLM as judge?
No. Each question has a verifiable ground-truth answer and is scored automatically.
Can we have our model evaluated?
Yes. The maintainers invite model developers to open an issue on GitHub or send an email to have a model evaluated.
LiveBench update historyno updates yet
No changes recorded for LiveBench yet.
Changes the vendor made and confirmed user reports, each checked against the vendor's own pages. We review every tool monthly.
Alternatives to LiveBench
Tools in the same category (Model leaderboards) with the most overlapping markets.
Contact for pricing · Free plan🇺🇸
Public AI arena, formerly LMArena, that ranks models for text, code, vision, search, images and video from anonymous side-by-side votes.
Free🇨🇦
Evaluation set of about 12,000 expert-reviewed multiple-choice questions with ten options each, used to compare language models.
Free🇨🇳
Open-source LLM evaluation platform from Shanghai with CompassRank leaderboards, the CompassHub benchmark catalog and VLMEvalKit.