REretrovrv·Jul 22, 2025
there's portkey that we've been working on: https://github.com/portkey-AI/gateway
In Show HN: Any-LLM – Lightweight router to access any LLM ProviderPortkey-AI
gateway
TypeScript2024–2025
Portkey-AI / gateway
gateway
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
3Distinct people
8Mentions
0Explicit recommenders
13.4KGitHub stars
What to know before exploring
What it is
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by month8 dated mentions
Observed mentions: 2024-01-08 13:35:09 UTC to 2025-07-22 20:00:44 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Jan 2024 | 2024-01-01 00:00:00 UTC | 2024-02-01 00:00:00 UTC | 1 | 1 | 0 |
| Feb 2024 | 2024-02-01 00:00:00 UTC | 2024-03-01 00:00:00 UTC | 1 | 1 | 0 |
| Mar 2024 | 2024-03-01 00:00:00 UTC | 2024-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2024 | 2024-04-01 00:00:00 UTC | 2024-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2024 | 2024-05-01 00:00:00 UTC | 2024-06-01 00:00:00 UTC | 1 | 0 | 1 |
| Jun 2024 | 2024-06-01 00:00:00 UTC | 2024-07-01 00:00:00 UTC | 0 | 0 | 0 |
| Jul 2024 | 2024-07-01 00:00:00 UTC | 2024-08-01 00:00:00 UTC | 1 | 0 | 1 |
| Aug 2024 | 2024-08-01 00:00:00 UTC | 2024-09-01 00:00:00 UTC | 2 | 2 | 0 |
| Sep 2024 | 2024-09-01 00:00:00 UTC | 2024-10-01 00:00:00 UTC | 0 | 0 | 0 |
| Oct 2024 | 2024-10-01 00:00:00 UTC | 2024-11-01 00:00:00 UTC | 0 | 0 | 0 |
| Nov 2024 | 2024-11-01 00:00:00 UTC | 2024-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2024 | 2024-12-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2025 | 2025-01-01 00:00:00 UTC | 2025-02-01 00:00:00 UTC | 0 | 0 | 0 |
| Feb 2025 | 2025-02-01 00:00:00 UTC | 2025-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2025 | 2025-03-01 00:00:00 UTC | 2025-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2025 | 2025-04-01 00:00:00 UTC | 2025-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2025 | 2025-05-01 00:00:00 UTC | 2025-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 2025 | 2025-06-01 00:00:00 UTC | 2025-07-01 00:00:00 UTC | 0 | 0 | 0 |
| Jul 2025 | 2025-07-01 00:00:00 UTC | 2025-08-01 00:00:00 UTC | 2 | 0 | 2 |
What Hacker News says
8 sourceswe essentially built the gateway as a service rather than an SDK: https://github.com/portkey-AI/gateway
In Show HN: Any-LLM – Lightweight router to access any LLM ProviderShow HN: Created a notebook to compare the top LMSYS vision models easily
2 points · Open discussionShow HN: An open-source AI Gateway with integrated guardrails
Hi HN,
I've been developing Portkey Gateway, an open-source AI gateway that's now processing billions of tokens daily across 200+ LLMs. Today, we're launching a significant update: integrated Guardrails at the gateway level.
Key technical features:
1. Guardrails as middleware: We've implemented a hooks architecture that allows guardrails to act as middleware in the request/response flow. This enables real-time LLM output evaluation and transformation.
2. Flexible orchestration: The gateway can now route requests based on guardrail verdicts. This allows for complex logic like fallbacks to different models or prompts based on output quality.
3. Plugin system: We've designed a modular plugin system that allows integration of various guardrail implementations (e.g., anthropic/constrained-llm, microsoft/guidance).
4. Stateless design: The guar
Read the rest
drails implementation maintains the gateway's stateless nature, ensuring scalability and allowing for easy horizontal scaling.
5. Unified API: Despite the added complexity, we've maintained our unified API across different LLM providers, now extended to include guardrail configurations.
Implementation details:
* The guardrails are implemented as async functions in the request pipeline.
* We use a combination of regex and LLM-based evaluation for output validation.
* The system supports both pre-processing (input modification) and post-processing (output filtering/transformation) guardrails.
Performance impact:
* Latency increase is minimal (<50ms) for most deterministic guardrails.
* We've implemented caching mechanisms to reduce repeated evaluations.
* Since the gateway lives on the edge, it avoids longer roundtrips
Challenges we're still tackling:
* Balancing strict guardrails with maintaining model creativity
* Standardizing evaluation metrics across different types of guardrails
* Handling guardrail false positives/negatives effectively
We believe this approach of integrating guardrails at the gateway level provides a powerful tool for managing LLM behavior in production environments.
The code is open-source, and we welcome contributions and feedback. We're particularly interested in hearing about specific use cases or challenges you've faced in implementing reliable LLM systems.
Detailed documentation: https://portkey.wiki/guardrails
What are your thoughts on this approach? Are there specific guardrail implementations or orchestration patterns you'd like to see added?
there's an open source ai gateway - https://github.com/Portkey-AI/gateway
In Ask HN: Devs using LLMs, how are you keeping costs low for LLM calls locally?Great to know this!
I have come across Portkey's Open-source AI Gateway which kind of does the same.
https://github.com/portkey-ai/gateway
It looks like with more LLM adoption, resiliency and cost related aspects take off sooner than expected unlike other technological trends in the past.
I'm also thinking that there is a chance if something like this could help build a better RAG pipeline or evals for the GenAI App. Because end of the day you want to reduce hallucinations but still get good generative responses.
In Show HN: Route your prompts to the best LLMAdding a streaming run function to the Assistants API
1 points · Open discussionShow HN: A lightweight AI gateway to 100+ models, in TS
We are Rohit & Ayush, we created Portkey this year March to help tackle some challenges we had seen while building apps based on GPT3, 3.5, 4, and the DevOps principles we brought to the scene to help tackle them.
We believe, a solid, performant, and reliable gateway lays the foundation to help build the next level of LLM apps. It decreases excessive reliance on any one company and takes the focus back to building instead of spending time fixing the nitty gritties of different providers and making them work together.
Features:
Blazing fast (9.9x faster) with a tiny footprint (~45kb installed)
Load balance across multiple models, providers, and keys
Fallbacks make sure your app stays resilient
Automatic Retries with exponential fallbacks come by default
Plug-in middleware as needed
Battle tested over 100B tokens and millions of
Read the rest
requests
For the folks serious about gateway, separation of concerns, TS developers.. I'd love to hear your thoughts and we're hungry for feedback!
Reach out to us at hello@portkey.ai or explore the project: https://github.com/portkey-ai/gateway