HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
langwatch
scenario
TypeScript2025–2026
langwatch / scenario

scenario

Agentic testing for agentic codebases

4Distinct people
9Mentions
0Explicit recommenders
1KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Agentic testing for agentic codebases

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 1:56:48 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month9 dated mentions

Observed mentions: 2025-04-15 13:26:43 UTC to 2026-07-22 14:52:14 UTC.

012342025-04-01 00:00:00 UTC to 2025-05-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsApr 20252025-05-01 00:00:00 UTC to 2025-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-06-01 00:00:00 UTC to 2025-07-01 00:00:00 UTC (exclusive): 4 mentions; 1 submissions, 3 comments2025-07-01 00:00:00 UTC to 2025-08-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2025-08-01 00:00:00 UTC to 2025-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-09-01 00:00:00 UTC to 2025-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-10-01 00:00:00 UTC to 2025-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-11-01 00:00:00 UTC to 2025-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsNov 20252025-12-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-01-01 00:00:00 UTC to 2026-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-02-01 00:00:00 UTC to 2026-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-01 00:00:00 UTC to 2026-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-04-01 00:00:00 UTC to 2026-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-05-01 00:00:00 UTC to 2026-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-01 00:00:00 UTC to 2026-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-07-01 00:00:00 UTC to 2026-08-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsJul 2026

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Apr 20252025-04-01 00:00:00 UTC2025-05-01 00:00:00 UTC110
May 20252025-05-01 00:00:00 UTC2025-06-01 00:00:00 UTC000
Jun 20252025-06-01 00:00:00 UTC2025-07-01 00:00:00 UTC413
Jul 20252025-07-01 00:00:00 UTC2025-08-01 00:00:00 UTC303
Aug 20252025-08-01 00:00:00 UTC2025-09-01 00:00:00 UTC000
Sep 20252025-09-01 00:00:00 UTC2025-10-01 00:00:00 UTC000
Oct 20252025-10-01 00:00:00 UTC2025-11-01 00:00:00 UTC000
Nov 20252025-11-01 00:00:00 UTC2025-12-01 00:00:00 UTC000
Dec 20252025-12-01 00:00:00 UTC2026-01-01 00:00:00 UTC000
Jan 20262026-01-01 00:00:00 UTC2026-02-01 00:00:00 UTC000
Feb 20262026-02-01 00:00:00 UTC2026-03-01 00:00:00 UTC000
Mar 20262026-03-01 00:00:00 UTC2026-04-01 00:00:00 UTC000
Apr 20262026-04-01 00:00:00 UTC2026-05-01 00:00:00 UTC000
May 20262026-05-01 00:00:00 UTC2026-06-01 00:00:00 UTC000
Jun 20262026-06-01 00:00:00 UTC2026-07-01 00:00:00 UTC000
Jul 20262026-07-01 00:00:00 UTC2026-08-01 00:00:00 UTC110

What Hacker News says

4 sources
That's true, we have been trying to help customers doing evals for ages now, and it's super hard for everyone to build a really good dataset and define great quality metrics just wanted then to shameless plug this lib I've built recently for this very topic, because it's been much easier to sell that into our clients than evals really, because it's closer to e2e tests: https://github.com/langwatch/scenario instead of 100 examples, it's easier for people to think on just the anecdotal example where the problem happens and let AI expand it, or replicate a situation from prod and describe the criteria in simple terms or code
In Karpathy: "context engineering" over "prompt engineering"
Hello HN! tl;dr: We built Scenario, an open-source testing library for AI agents. It simulates real conversations with your agent, its code-driven, and lets you assert anything mid-dialogue. Repo: https://github.com/langwatch/scenario Docs: https://scenario.langwatch.ai/ I'm Rogerio, founder of LangWatch, I've been helping many customers building LLM applications in this past two years and worked with Alex on this. Most of the efforts for LLM quality so far were about evaluations, single-turn, there was nothing actually good to test agents, it all felt forced, but we believe we cracked it now, we have built an agent testing library that test your agent by simulating a user and playing a conversation back and forth with it. One of the key challenges there was that we had to make it compatible with all the 273+ AI frameworks (and counting)
Read the rest
there are. Luckliy AG-UI protocol popped up recently, standardizing agents frameworks and UI interactions, this is perfect, because at the end of the day, we want our user simulator to "see" just the same that the user sees. So we made Scenario in a way that is really easy to connect to any agent no matter the tech stack, from a simple string <-> string connection, to openai standard messages format, to AG-UI. The other key challenge was to balance testing the open-endedness of agents vs having reliable cases you want to test, so we worked a lot on thinking through the autopilot simulation vs the fully scripted one, and here again, the goal was complete interoperability. At the end of the day, the design we achieved was simply having lambdas, that you can call at any point of the test, so it's just code, where you can connect any other evaluation or assertion tool you want, we are not restrictive. Check out the repo and the docs, we would love to get some feedback in here! Repo: https://github.com/langwatch/scenario Docs: https://scenario.langwatch.ai/
In Show HN: Simulation-Based Testing for Agents Using AG-UI Protocol

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.