benchflow

Research infra for creating RL environments, post-training, and evals.

AI & Machine Learning Testing & Quality

Explore on GitHub Read Hacker News discussions

2Distinct people

2Mentions

0Explicit recommenders

367GitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Research infra for creating RL environments, post-training, and evals.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by day2 dated mentions

Observed mentions: 2025-03-21 21:16:11 UTC to 2025-03-30 07:29:11 UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions
Mar 21	2025-03-21 00:00:00 UTC	2025-03-22 00:00:00 UTC	1	1
Mar 22	2025-03-22 00:00:00 UTC	2025-03-23 00:00:00 UTC	0	0
Mar 23	2025-03-23 00:00:00 UTC	2025-03-24 00:00:00 UTC	0	0
Mar 24	2025-03-24 00:00:00 UTC	2025-03-25 00:00:00 UTC	0	0
Mar 25	2025-03-25 00:00:00 UTC	2025-03-26 00:00:00 UTC	0	0
Mar 26	2025-03-26 00:00:00 UTC	2025-03-27 00:00:00 UTC	0	0
Mar 27	2025-03-27 00:00:00 UTC	2025-03-28 00:00:00 UTC	0	0
Mar 28	2025-03-28 00:00:00 UTC	2025-03-29 00:00:00 UTC	0	0
Mar 29	2025-03-29 00:00:00 UTC	2025-03-30 00:00:00 UTC	0	0
Mar 30	2025-03-30 00:00:00 UTC	2025-03-31 00:00:00 UTC	1	1

What Hacker News says

2 sources

www_xiangyi_li·Mar 30, 2025Submission

Show HN: BenchFlow – Open-Source Benchmark Hub and Eval Infra for AI Devs

We just open-sourced [*BenchFlow*](https://github.com/benchflow-ai/benchflow) — an evaluation infrastructure and benchmark hub for AI developers and researchers. Whether you're building or running benchmarks, BenchFlow helps you do it fast and reproducibly. - Easy-to-use `BaseAgent`/`BaseBench` interfaces - Run tasks across agents & models with one command - Add your own benchmarks and agents - Collect results, compare runs, and iterate faster - Docker-ready benchmark deployment - Public Benchmark Hub: [benchflow.ai](https://benchflow.ai) We support OpenAI, HuggingFace, local models, and more. Would love feedback or contributions! GitHub: https://github.com/benchflow-ai/benchflow Docs: https://docs.benchflow.ai/introduction Discord: https://discord.gg/mZ9Rc8q8W3

1 points · Open discussion

xdotli·Mar 21, 2025Submission

Show HN: BenchFlow – run AI benchmarks as an API

I built BenchFlow, an open-source framework that lets you integrate and evaluate AI tasks using Docker-based benchmarks. You can try it out right now by cloning the repo and running a benchmark in minutes. As an AI researcher, I was frustrated with how much time my team spent setting up benchmark environments rather than actually improving our models. We'd spend weeks configuring environments, only to find inconsistencies when comparing results with other teams. BenchFlow started as an internal tool to standardize our evaluation process, and we decided to open-source it after seeing how much time it saved us. Unlike other benchmarking tools that focus on specific domains, BenchFlow provides a unified interface for any AI task. The Docker-based approach ensures consistent environments across different machines and teams. You don't need to

Read the rest

worry about dependency conflicts or environment setup - just implement a simple interface and you're ready to go. How to try it out? check our link but here's a preview of that 1. pip install benchflow 2. load a benchmark and define how to call your agents/models 3. run it and get the result Available benchmarks you can try today: - MMLU-PRO: Test your model's knowledge across 57 subjects - Bird: Evaluate business intelligence reasoning capabilities - WebArena: See how your agent performs on web-based tasks - MedQA-CS: Test medical question answering abilities The framework handles all the containerization, task distribution, and result collection, so you can focus on improving your models rather than managing infrastructure. I'd love to hear your feedback and see how you use it. What benchmarks would you like to see added next? Please give us a star if you can, thanks! GitHub: https://github.com/benchflow-ai/benchflow Website: https://benchflow.ai/ Benchmark Hub: https://benchflow.ai/benchmarks Inspo: https://github.com/ServiceNow/BrowserGym

24 points · Open discussion