HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
ask-fini
paramount
TypeScript2024–2024
ask-fini / paramount

paramount

Agent accuracy measurements for LLMs

2Distinct people
4Mentions
0Explicit recommenders
202GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Agent accuracy measurements for LLMs

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 12:56:33 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week4 dated mentions

Observed mentions: 2024-05-03 14:22:37 UTC to 2024-06-13 23:18:19 UTC. Weeks start Monday, UTC.

0122024-04-29 00:00:00 UTC to 2024-05-06 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsApr 292024-05-06 00:00:00 UTC to 2024-05-13 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-05-13 00:00:00 UTC to 2024-05-20 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-05-20 00:00:00 UTC to 2024-05-27 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsMay 202024-05-27 00:00:00 UTC to 2024-06-03 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2024-06-03 00:00:00 UTC to 2024-06-10 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-10 00:00:00 UTC to 2024-06-17 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 commentsJun 10

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Apr 292024-04-29 00:00:00 UTC2024-05-06 00:00:00 UTC110
May 62024-05-06 00:00:00 UTC2024-05-13 00:00:00 UTC000
May 132024-05-13 00:00:00 UTC2024-05-20 00:00:00 UTC000
May 202024-05-20 00:00:00 UTC2024-05-27 00:00:00 UTC000
May 272024-05-27 00:00:00 UTC2024-06-03 00:00:00 UTC110
Jun 32024-06-03 00:00:00 UTC2024-06-10 00:00:00 UTC000
Jun 102024-06-10 00:00:00 UTC2024-06-17 00:00:00 UTC211

What Hacker News says

1 sources
hakimk·May 29, 2024Submission

Show HN: Paramount – OSS package for *Human* Evals of AI support

Hey HN, Hakim here from Fini (YC S22). We focus on providing support bots for Enterprises that have a high volume of support requests. Today, one of the largest use cases of LLMs is specifically for the purpose of automating support. As the space has evolved over the past year, there has subsequently been a need for evaluations of LLM outputs - and a sea of LLM Evals packages have been released. "LLM evals" refer to the evaluation of large language models, assessing how well these AI systems understand and generate human-like text. These packages have relied on "automatic evals," where algorithms (usually another LLM) automatically test and score AI responses without human intervention. In our day to day work, we have found that Automatic Evals are not enough to get the required 95% accuracy for our Enterprise customers. Automatic Evals a
Read the rest
re efficient, but still often miss nuances that only human expertise can catch. Automatic Evals can never replace the feedback of a trained human who is deeply knowledgeable on an organization's latest product releases, knowledgebase, policies and support issues. The key to solve this is to stop ignoring the business side of the problem, and start involving knowledgeable experts in the evaluation process. That is why we are releasing Paramount - an Open Source package which involves human feedback directly into the evaluation process. By simplifying the step of gathering feedback, ML Engineers can pinpoint and fix accuracy issues (prompts, knowledgebase issues) much faster. Paramount provides a framework for recording LLM function outputs (ground truth data) and facilitates agent evaluations through a simple UI, reducing the time to identify and correct errors. Developers can integrate Paramount with a python decorator that logs LLM interactions into a database, followed by a straightforward UI for expert review. This process aids the debugging and validation phase of launching accurate support bots.
2 points · Open discussion

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.