XCXCSme·Mar 15, 2024
Just found this: https://github.com/zilliztech/GPTCache which seems to address this idea/issue.
In Ask HN: What are the drawbacks of caching LLM responses?zilliztech
GPTCache
Python2023–2024
zilliztech / GPTCache
GPTCache
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
12Distinct people
24Mentions
0Explicit recommenders
8.2KGitHub stars
What to know before exploring
What it is
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:03:16 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by month24 dated mentions
Observed mentions: 2023-04-05 19:59:52 UTC to 2024-03-15 21:30:08 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Apr 2023 | 2023-04-01 00:00:00 UTC | 2023-05-01 00:00:00 UTC | 13 | 13 | 0 |
| May 2023 | 2023-05-01 00:00:00 UTC | 2023-06-01 00:00:00 UTC | 3 | 2 | 1 |
| Jun 2023 | 2023-06-01 00:00:00 UTC | 2023-07-01 00:00:00 UTC | 2 | 1 | 1 |
| Jul 2023 | 2023-07-01 00:00:00 UTC | 2023-08-01 00:00:00 UTC | 5 | 4 | 1 |
| Aug 2023 | 2023-08-01 00:00:00 UTC | 2023-09-01 00:00:00 UTC | 0 | 0 | 0 |
| Sep 2023 | 2023-09-01 00:00:00 UTC | 2023-10-01 00:00:00 UTC | 0 | 0 | 0 |
| Oct 2023 | 2023-10-01 00:00:00 UTC | 2023-11-01 00:00:00 UTC | 0 | 0 | 0 |
| Nov 2023 | 2023-11-01 00:00:00 UTC | 2023-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2023 | 2023-12-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2024 | 2024-01-01 00:00:00 UTC | 2024-02-01 00:00:00 UTC | 0 | 0 | 0 |
| Feb 2024 | 2024-02-01 00:00:00 UTC | 2024-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2024 | 2024-03-01 00:00:00 UTC | 2024-04-01 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
24 sourcesCheck this out. Built on a vector database (https://github.com/milvus-io/milvus) and a semantic cache (https://github.com/zilliztech/GPTCache)
https://osschat.io/
In Show HN: Danswer – Open-source question answering across all your docsAsk HN: Is LLM caching necessary?
With the proliferation of large models, an increasing number of enterprises and individual developers are now developing applications based on these models. As such, it is worth considering whether large model caching is necessary during the development process.
Our project: https://github.com/zilliztech/GPTCache
3 points · Open discussionMaybe [GPTCache](https://github.com/zilliztech/GPTCache) can make it more attractive, because similar problems can be less expensive, and can also be responded to faster. Of course, the specific configuration needs to be based on real usage scenarios.
In Gorilla-CLI: LLMs for CLI including K8s/AWS/GCP/Azure/sed and 1500 APIsLimited budget or machine resources, how to achieve a decent LLM experience?
2 points · Open discussionMaybe GPTCache can reduce the count of llm request for lower cost.
detail: https://github.com/zilliztech/GPTCache
In Show HN: DB-GPT, an LLM tool for databaseHow GPTCache works with llama index
5 points · Open discussionShow HN: Reduce your GPT bill via semantic caching
Hey folks,
As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite.
GPTCache provides several benefits:
1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM service
2) enhanced performance by fetching cached query results directly
3) improved scalability and availability by avoiding rate limits, and
4) a flexible development environment that allows developers to verify their application's features without connecting to LLM APIs or network.
Come check it out! https://github.com/zilliztech/gptcache
1 points · Open discussionReduce your GPT bill via semantic caching
3 points · Open discussionShowing the 15 most recent of 24 sources. Enable JavaScript to search and browse every indexed mention.