HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
vectara
hallucination-leaderboard
Python2023–2025
vectara / hallucination-leaderboard

hallucination-leaderboard

Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents

11Distinct people
26Mentions
0Explicit recommenders
3.3KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 11:39:46 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month26 dated mentions

Observed mentions: 2023-11-06 19:11:51 UTC to 2025-12-18 16:18:15 UTC.

02462023-11-01 00:00:00 UTC to 2023-12-01 00:00:00 UTC (exclusive): 7 mentions; 2 submissions, 5 commentsNov 20232023-12-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2024-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-01 00:00:00 UTC to 2024-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-03-01 00:00:00 UTC to 2024-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-04-01 00:00:00 UTC to 2024-05-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-05-01 00:00:00 UTC to 2024-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-01 00:00:00 UTC to 2024-07-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-07-01 00:00:00 UTC to 2024-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-08-01 00:00:00 UTC to 2024-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-09-01 00:00:00 UTC to 2024-10-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2024-10-01 00:00:00 UTC to 2024-11-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-11-01 00:00:00 UTC to 2024-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsNov 20242024-12-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-01-01 00:00:00 UTC to 2025-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-02-01 00:00:00 UTC to 2025-03-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-03-01 00:00:00 UTC to 2025-04-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2025-04-01 00:00:00 UTC to 2025-05-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-05-01 00:00:00 UTC to 2025-06-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2025-06-01 00:00:00 UTC to 2025-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-07-01 00:00:00 UTC to 2025-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-08-01 00:00:00 UTC to 2025-09-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-09-01 00:00:00 UTC to 2025-10-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-10-01 00:00:00 UTC to 2025-11-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-11-01 00:00:00 UTC to 2025-12-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-12-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 commentsDec 2025

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Nov 20232023-11-01 00:00:00 UTC2023-12-01 00:00:00 UTC725
Dec 20232023-12-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
Jan 20242024-01-01 00:00:00 UTC2024-02-01 00:00:00 UTC000
Feb 20242024-02-01 00:00:00 UTC2024-03-01 00:00:00 UTC000
Mar 20242024-03-01 00:00:00 UTC2024-04-01 00:00:00 UTC000
Apr 20242024-04-01 00:00:00 UTC2024-05-01 00:00:00 UTC101
May 20242024-05-01 00:00:00 UTC2024-06-01 00:00:00 UTC000
Jun 20242024-06-01 00:00:00 UTC2024-07-01 00:00:00 UTC101
Jul 20242024-07-01 00:00:00 UTC2024-08-01 00:00:00 UTC000
Aug 20242024-08-01 00:00:00 UTC2024-09-01 00:00:00 UTC000
Sep 20242024-09-01 00:00:00 UTC2024-10-01 00:00:00 UTC202
Oct 20242024-10-01 00:00:00 UTC2024-11-01 00:00:00 UTC101
Nov 20242024-11-01 00:00:00 UTC2024-12-01 00:00:00 UTC000
Dec 20242024-12-01 00:00:00 UTC2025-01-01 00:00:00 UTC101
Jan 20252025-01-01 00:00:00 UTC2025-02-01 00:00:00 UTC000
Feb 20252025-02-01 00:00:00 UTC2025-03-01 00:00:00 UTC101
Mar 20252025-03-01 00:00:00 UTC2025-04-01 00:00:00 UTC211
Apr 20252025-04-01 00:00:00 UTC2025-05-01 00:00:00 UTC101
May 20252025-05-01 00:00:00 UTC2025-06-01 00:00:00 UTC202
Jun 20252025-06-01 00:00:00 UTC2025-07-01 00:00:00 UTC000
Jul 20252025-07-01 00:00:00 UTC2025-08-01 00:00:00 UTC000
Aug 20252025-08-01 00:00:00 UTC2025-09-01 00:00:00 UTC101
Sep 20252025-09-01 00:00:00 UTC2025-10-01 00:00:00 UTC101
Oct 20252025-10-01 00:00:00 UTC2025-11-01 00:00:00 UTC101
Nov 20252025-11-01 00:00:00 UTC2025-12-01 00:00:00 UTC101
Dec 20252025-12-01 00:00:00 UTC2026-01-01 00:00:00 UTC303

What Hacker News says

7 sources
This is just typical of so much work in the field. They pick and choose which models to compare against and on which benchmarks. If this model was truly great, they would be comparing against Claude 2 and GPT4 across a bunch of different benchmarks. Instead they compare against Palm 2, which in a lot of tests is a weak model (https://venturebeat.com/ai/google-bard-fails-to-deliver-on-i....) and prone to hallucination (https://github.com/vectara/hallucination-leaderboard).
In Inflection-2: the next step up
eskibars·Nov 6, 2023Submission

Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and some specific examples of these summarization hallucinations), and a GitHub repository containing our evaluation from the most popular generative LLMs available today. Links to all of them are referenced in the blog here, but for the technical audie
Read the rest
nce here, the most interesting additional links might be: - https://huggingface.co/vectara/hallucination_evaluation_mode... - https://github.com/vectara/hallucination-leaderboard We hope that releasing these under a truly open source license and detailing the methodology, we hope to increase the viability of anyone really quantitatively measuring and improving the generative LLMs they're publishing.
65 points · Open discussion

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.