HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
vibrantlabsai
ragas
Python2023–2024
vibrantlabsai / ragas

ragas

Supercharge Your LLM Application Evaluations 🚀

AI & Machine Learning
Explore on GitHubRead Hacker News discussions
5Distinct people
7Mentions
0Explicit recommenders
16.3KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Supercharge Your LLM Application Evaluations 🚀

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month7 dated mentions

Observed mentions: 2023-08-02 04:06:41 UTC to 2024-03-21 19:23:31 UTC.

01232023-08-01 00:00:00 UTC to 2023-09-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsAug 20232023-09-01 00:00:00 UTC to 2023-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-01 00:00:00 UTC to 2023-11-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2023-11-01 00:00:00 UTC to 2023-12-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsNov 20232023-12-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2024-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-01 00:00:00 UTC to 2024-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-03-01 00:00:00 UTC to 2024-04-01 00:00:00 UTC (exclusive): 3 mentions; 2 submissions, 1 commentsMar 2024

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Aug 20232023-08-01 00:00:00 UTC2023-09-01 00:00:00 UTC101
Sep 20232023-09-01 00:00:00 UTC2023-10-01 00:00:00 UTC000
Oct 20232023-10-01 00:00:00 UTC2023-11-01 00:00:00 UTC211
Nov 20232023-11-01 00:00:00 UTC2023-12-01 00:00:00 UTC110
Dec 20232023-12-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
Jan 20242024-01-01 00:00:00 UTC2024-02-01 00:00:00 UTC000
Feb 20242024-02-01 00:00:00 UTC2024-03-01 00:00:00 UTC000
Mar 20242024-03-01 00:00:00 UTC2024-04-01 00:00:00 UTC321

What Hacker News says

7 sources
swyx·Mar 21, 2024
congrats on launching! i think my continuing struggle with looking at Ragas as a company/library rather than a very successful mental model is that the core of it is like 8 metrics (https://github.com/explodinggradients/ragas/tree/main/src/ra...) that are each 1-200 LOC. i can inline that easily in my app and retain full control, or model that in langchain or haystack or whatever. why is Ragas a library and a company, rather than an overall "standard" or philosophy (eg like Heroku's 12 Factor Apps) that could maybe be more universally adopted without using the library? (just giving an opp to pitch some underappreciated benefits of using this library)
In Show HN: Ragas – Open-source library for evaluating RAG pipelines
shahules·Mar 21, 2024Submission

Show HN: Ragas – Open-source library for evaluating RAG pipelines

Ragas is an open-source library for evaluating and testing RAG and other LLM applications. Github: https://docs.ragas.io/en/stable/, docs: https://docs.ragas.io/. Ragas provides you with different sets of metrics and methods like synthetic test data generation to help you evaluate your RAG applications. Ragas started off by scratching our own itch for evaluating our RAG chatbots last year. Problems Ragas can solve - How do you choose the best components for your RAG, such as the retriever, reranker, and LLM? - How do you formulate a test dataset without spending tons of money and time? We believe there needs to be an open-source standard for evaluating and testing LLM applications, and our vision is to build it for the community. We are tackling this challenge by evolving the ideas from the traditional ML lifecycle for LLM applications. M
Read the rest
L Testing Evolved for LLM Applications We built Ragas on the principles of metrics-driven development and aim to develop and innovate techniques inspired by state-of-the-art research to solve the problems in evaluating and testing LLM applications. We don't believe that the problem of evaluating and testing applications can be solved by building a fancy tracing tool; rather, we want to solve the problem from a layer under the stack. For this, we are introducing methods like automated synthetic test data curation, metrics, and feedback utilisation, which are inspired by lessons learned from deploying stochastic models in our careers as ML engineers. While currently focused on RAG pipelines, our goal is to extend Ragas for testing a wide array of compound systems, including those based on RAGs, agentic workflows, and various transformations. Try out Ragas here https://colab.research.google.com/github/shahules786/openai-... in Google Colab. Read our docs - https://docs.ragas.io/ to know more We would love to hear feedback from the HN community :)
121 points · Open discussion
shahules·Mar 20, 2024Submission

Show HN: Ragas – Open-source library for evals and testing RAG systems

Ragas is an open-source library designed for evaluating and testing RAG (Retrieval-Augmented Generation) and other LLM applications. It offers a diverse set of metrics and methods, including synthetic test data generation, to help you assess your RAG applications. Ragas was initially developed to address our own needs for evaluating RAG chatbots last year. ### Problems Ragas Can Solve: - How can you select the best components for your RAG, such as the retriever, reranker, and LLM? - How can you create a test dataset without incurring significant expenses and time? We believe there's a need for an open-source standard for evaluating and testing LLM applications. Our vision is to establish this standard for the community. We're addressing this challenge by adapting ideas from the traditional ML lifecycle for LLM applications. ### ML Testing
Read the rest
Evolved for LLM Applications Ragas is founded on the principles of metrics-driven development. Our goal is to develop and innovate techniques inspired by the latest research to address the challenges in evaluating and testing LLM applications. We don't think that merely building a sophisticated tracing tool will solve the evaluation and testing challenges. Instead, we aim to tackle these issues from a foundational level. To this end, we're introducing methods such as automated synthetic test data curation, metrics, and feedback utilization. These approaches are inspired by lessons learned from deploying stochastic models throughout our careers as machine learning engineers. While our current focus is on RAG pipelines, we intend to expand Ragas to test a broad spectrum of compound systems. This includes systems based on RAGs, agentic workflows, and various transformations. ### Try Ragas Experience Ragas by trying it out in Google Colab [here](https://colab.research.google.com/github/shahules786/openai-...). For more information, read our [documentation](https://docs.ragas.io/). We would love to hear feedback from the Hacker News community :)
15 points · Open discussion

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.