Vectara Hallucination Leaderboard overview and key facts (October 2026)

The Vectara Hallucination Leaderboard ranks language models by how often they introduce unsupported facts when summarizing a document. It matters for teams building RAG systems that must stay close to their sources.

Each model summarizes the same set of documents, and Vectara's Hughes Hallucination Evaluation Model judges whether each summary is consistent with its source. The leaderboard uses HHEM-2.3, Vectara's commercial version; an open variant, HHEM-2.1-Open, is available on Hugging Face under Apache 2.0. The table lists hallucination rate, factual consistency rate, answer rate and average summary length; it was last updated on September 22, 2026. An interactive version runs as a Hugging Face Space. Vectara, Inc. in Palo Alto maintains the project.

• Chooses models for document summariesTeams that let a model summarize contracts, reports or tickets can prefer models with low hallucination rates. Fewer invented details means less checking by staff. The leaderboard lists the rate for each model tested.
• Checks own outputs with HHEMThe open variant HHEM-2.1-Open can score summaries from your own system. Quality teams can monitor factual consistency in production as well as at selection time. Its scores can differ from those of the commercial HHEM-2.3 used for the leaderboard.
• Balances accuracy and refusalThe answer rate shows how often a model declines to summarize. Teams can see whether a low hallucination rate comes with too many refusals for their use. Both figures appear side by side in the table.

• Focuses on one business-relevant risk: invented facts in summaries of given documents.
• An open variant of the evaluation model, HHEM-2.1-Open, is available under Apache 2.0.

Vectara, Inc. is based in Palo Alto, California, according to its privacy policy.

HHEM is itself a model and can misjudge summaries, so confirm results on samples of your own documents. The test uses English documents and short summaries; longer or German texts can behave differently. Vectara sells a RAG platform, which is worth knowing when reading the rankings.

No reviews yet. Be the first to share how your team uses it.

Every review shows its verification level. We read each one before it goes live. How we check reviews