HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
BerriAI
litellm
Python2023–2026
BerriAI / litellm

litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

39Distinct people
53Mentions
0Explicit recommenders
63.2KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 1:56:48 AM UTC.

Anthropic IPO direct repository linkDebated

Commenters are divided on whether Anthropic's IPO represents a necessary step for scaling infrastructure or an opportunistic cash-out driven by inflated valuations and looming commoditization. While some argue that switching between LLM providers is technically trivial due to standardized APIs, others contend that enterprise moats are sustained by deep integration, governance stability, and corporate contracts.

887 readable replies · Supporting reply 1 · Supporting reply 2 · Supporting reply 3 · Supporting reply 4 · Supporting reply 5 · Supporting reply 6 · Supporting reply 7 · Supporting reply 8 · Supporting reply 9 · Supporting reply 10 · Supporting reply 11 · Supporting reply 12 · Supporting reply 13 · Supporting reply 14 · Supporting reply 15 · Supporting reply 16

Mentions over time

HN mentions by year53 dated mentions

Observed mentions: 2023-07-27 01:31:35 UTC to 2026-10-21 16:40:34 UTC.

051015202023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 17 mentions; 3 submissions, 14 comments20232024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 9 mentions; 0 submissions, 9 comments20242025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 7 mentions; 2 submissions, 5 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 20 mentions; 5 submissions, 15 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC17314
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC909
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC725
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC20515

What Hacker News says

7 sources
What a train wreck, there are thousand more apps in store that do exactly this because its the easiest way to use openAI without having to host your own backend/proxy. I have spend quite some time protecting my apps from this scenario and found a couple of open source projects that do a good job as proxys (no affiliation I just used them in the past): - https://github.com/BerriAI/litellm - https://github.com/KenyonY/openai-forward/tree/main but they still lack other abuse protection mechanism like rate limitting, device attestation etc. so I started building my own open source SDK - https://github.com/brahyam/Gateway
In Exploiting the IKKO Activebuds “AI powered” earbuds (2024)
cauchyk·Jun 29, 2025Submission

Show HN: A modern C++20 AI SDK (GPT‑4o, Claude 3.5, tool‑calling)

Hi all, I’m hacking on new features for the ClickHouse native client and wanted the same “just call the model” ergonomics JavaScript and Python now enjoy. It didn’t exist for modern C++, so I wrote one. ai‑sdk‑cpp (Apache‑2.0) gives you: - Unified calls to OpenAI (GPT‑4o) and Anthropic (Claude 3.5) with a single C++20 API. - Streaming, multi‑turn chat, error handling—all std::optional/std::variant, no macros. - Tool calling (function‑calling) so the model can hit real APIs; sync or async, runs in parallel. The tricky bit: C++ still lacks real reflection, so mapping plain functions → JSON schemas isn’t as automatic as, say, TypeScript decorators. I’d love fresh eyes on that part. Try the examples and tell me where it feels clunky. This is inspired by Vercel's AI SDK [1], and litellm [2]. Repo live here: https://github.com/ClickHouse/ai-s
Read the rest
56 points · Open discussion
Confession time: litellm still doesn't support parallel tool calls with Gemini models [https://github.com/BerriAI/litellm/issues/9686] so we wrote our own "parallel tool calls" on top of Structured Output. It did take a few iterations on the prompt design but all of it was "yeah I can see why that was ambiguous" kinds of things, no real complaints. GP2.5 does have a different flavor than S3.7 but it's hard to say that one is better or worse than the other [edit: at tool calling -- GP2.5 is definitely smarter in general]. GP2.5 is I would say a bit more aggressive at doing "speculative" tool execution in parallel with the architect, e.g. spawning multiple search agent calls at the same time, which for Brokk is generally a good thing but I could see use cases where you'd want to dial that back.
In The unreasonable effectiveness of an LLM agent loop with tool use
I think LLM's plugin architecture is a better bet for supporting model providers than the way LiteLLM does it. The problem with LiteLLM's approach is that every model provider needs to be added to the core library - in https://github.com/BerriAI/litellm/tree/main/litellm/llms - and then shipped as a new release. LLM uses plugins because then there's no need to sync new providers with the core tool. When a new Gemini feature comes out I ship a new release of https://github.com/simonw/llm-gemini - no need for a release of core. I can wake up one morning and LLM grew support for a bunch of new models overnight because someone else released a plugin. I'm not saying "LLM is better than LiteLLM" here - LiteLLM is a great library with a whole lot more contributors than LLM, and it's also been fully focused on being a great Python library - LLM h
Read the rest
as also had more effort invested in the CLI aspect than the Python library aspect so far. I am confident that a plugin system is a better way to solve this problem generally though.
In Smartfunc: Turn Docstrings into LLM-Functions

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.