HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
jlko
semantic_uncertainty
Python2024–2024
jlko / semantic_uncertainty

semantic_uncertainty

Codebase for reproducing the experiments of the semantic uncertainty paper (short-phrase and sentence-length experiments).

2Distinct people
2Mentions
0Explicit recommenders
431GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Codebase for reproducing the experiments of the semantic uncertainty paper (short-phrase and sentence-length experiments).

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 12:56:33 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by hour2 dated mentions

Observed mentions: 2024-06-24 06:47:58 UTC to 2024-06-24 20:46:10 UTC.

012024-06-24 06:00:00 UTC to 2024-06-24 07:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJun 24, 06:002024-06-24 07:00:00 UTC to 2024-06-24 08:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 08:00:00 UTC to 2024-06-24 09:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 09:00:00 UTC to 2024-06-24 10:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 10:00:00 UTC to 2024-06-24 11:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 11:00:00 UTC to 2024-06-24 12:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 12:00:00 UTC to 2024-06-24 13:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 13:00:00 UTC to 2024-06-24 14:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsJun 24, 13:002024-06-24 14:00:00 UTC to 2024-06-24 15:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 15:00:00 UTC to 2024-06-24 16:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 16:00:00 UTC to 2024-06-24 17:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 17:00:00 UTC to 2024-06-24 18:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 18:00:00 UTC to 2024-06-24 19:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 19:00:00 UTC to 2024-06-24 20:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 20:00:00 UTC to 2024-06-24 21:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJun 24, 20:00

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Jun 24, 06:002024-06-24 06:00:00 UTC2024-06-24 07:00:00 UTC101
Jun 24, 07:002024-06-24 07:00:00 UTC2024-06-24 08:00:00 UTC000
Jun 24, 08:002024-06-24 08:00:00 UTC2024-06-24 09:00:00 UTC000
Jun 24, 09:002024-06-24 09:00:00 UTC2024-06-24 10:00:00 UTC000
Jun 24, 10:002024-06-24 10:00:00 UTC2024-06-24 11:00:00 UTC000
Jun 24, 11:002024-06-24 11:00:00 UTC2024-06-24 12:00:00 UTC000
Jun 24, 12:002024-06-24 12:00:00 UTC2024-06-24 13:00:00 UTC000
Jun 24, 13:002024-06-24 13:00:00 UTC2024-06-24 14:00:00 UTC000
Jun 24, 14:002024-06-24 14:00:00 UTC2024-06-24 15:00:00 UTC000
Jun 24, 15:002024-06-24 15:00:00 UTC2024-06-24 16:00:00 UTC000
Jun 24, 16:002024-06-24 16:00:00 UTC2024-06-24 17:00:00 UTC000
Jun 24, 17:002024-06-24 17:00:00 UTC2024-06-24 18:00:00 UTC000
Jun 24, 18:002024-06-24 18:00:00 UTC2024-06-24 19:00:00 UTC000
Jun 24, 19:002024-06-24 19:00:00 UTC2024-06-24 20:00:00 UTC000
Jun 24, 20:002024-06-24 20:00:00 UTC2024-06-24 21:00:00 UTC101

What Hacker News says

1 sources
the method described by this paper does not > draw[ing] several samples and evaluating the entropy and/or disagreement between those draws the method from the paper (as I understand it): - samples multiple answers, (e.g. "music:0.8, musician:0.9, concert:0.7, actress:0.5, superbowl:0.6") - groups them by semantic similarity and gives them an id ([music, musician, concert] -> MUSIC, [actress] -> ACTING, [superbowl] -> SPORTS), note that they just use an integer or something for the id - sums the probability of those grouped answers and normalizes: (MUSIC:2.4, ACTING:0.5, SPORTS:0.6 -> MUSIC:0.686, SPORTS:0.171, ACTING:0.143) They also go to pains in the paper to clearly define what they are trying to prevent, which is confabulations. > We focus on a subset of hallucinations which we call ‘confabulations’ for which LLMs fluently make claims
Read the rest
that are both wrong and arbitrary—by which we mean that the answer is sensitive to irrelevant details such as random seed. Common misconceptions will still be strongly represented in the dataset. What this method does is it penalizes semantically isolated answers (answers dissimilar to other possible answers) with mediocre likelihood. Now technically, this paper only compares the effectiveness of "detecting" the confabulation to other methods - it doesn't offer an improved sampling method which utilizes that detection. And of course, if it were used as part of a generation technique it is subject to the extreme penalty of 10xing the number of model generations required. link to the code: https://github.com/jlko/semantic_uncertainty
In Detecting hallucinations in large language models using semantic entropy

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.