HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
finic-ai
rag-stack
TypeScript2023–2024
finic-ai / rag-stack

rag-stack

🤖 Deploy a private ChatGPT alternative hosted within your VPC. 🔮 Connect it to your organization's knowledge base and use it as a corporate oracle. Supports open-source LLMs like Llama 2, Falcon, and GPT4All.

ProductivityAI & Machine LearningDevOps & Cloud
Explore on GitHubRead Hacker News discussions
5Distinct people
6Mentions
1Explicit recommenders
1.6KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

🤖 Deploy a private ChatGPT alternative hosted within your VPC. 🔮 Connect it to your organization's knowledge base and use it as a corporate oracle. Supports open-source LLMs like Llama 2, Falcon, and GPT4All.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

I love the idea of a Modelfile but wonder if it can be setup to also retrieve information from other systems and insert them into the LLM’s context window via a prompt. This…

skanga used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 12:54:37 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week6 dated mentions

Observed mentions: 2023-07-20 17:11:08 UTC to 2024-02-16 05:15:52 UTC. Weeks start Monday, UTC.

01232023-07-17 00:00:00 UTC to 2023-07-24 00:00:00 UTC (exclusive): 3 mentions; 1 submissions, 2 commentsJul 172023-07-24 00:00:00 UTC to 2023-07-31 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-07-31 00:00:00 UTC to 2023-08-07 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2023-08-07 00:00:00 UTC to 2023-08-14 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-08-14 00:00:00 UTC to 2023-08-21 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2023-08-21 00:00:00 UTC to 2023-08-28 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-08-28 00:00:00 UTC to 2023-09-04 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-09-04 00:00:00 UTC to 2023-09-11 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-09-11 00:00:00 UTC to 2023-09-18 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-09-18 00:00:00 UTC to 2023-09-25 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-09-25 00:00:00 UTC to 2023-10-02 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-02 00:00:00 UTC to 2023-10-09 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-09 00:00:00 UTC to 2023-10-16 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-16 00:00:00 UTC to 2023-10-23 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-23 00:00:00 UTC to 2023-10-30 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-30 00:00:00 UTC to 2023-11-06 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsOct 302023-11-06 00:00:00 UTC to 2023-11-13 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-11-13 00:00:00 UTC to 2023-11-20 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-11-20 00:00:00 UTC to 2023-11-27 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-11-27 00:00:00 UTC to 2023-12-04 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-12-04 00:00:00 UTC to 2023-12-11 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-12-11 00:00:00 UTC to 2023-12-18 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-12-18 00:00:00 UTC to 2023-12-25 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-12-25 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2024-01-08 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-08 00:00:00 UTC to 2024-01-15 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-15 00:00:00 UTC to 2024-01-22 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-22 00:00:00 UTC to 2024-01-29 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-29 00:00:00 UTC to 2024-02-05 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-05 00:00:00 UTC to 2024-02-12 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-12 00:00:00 UTC to 2024-02-19 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsFeb 12

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Jul 172023-07-17 00:00:00 UTC2023-07-24 00:00:00 UTC312
Jul 242023-07-24 00:00:00 UTC2023-07-31 00:00:00 UTC000
Jul 312023-07-31 00:00:00 UTC2023-08-07 00:00:00 UTC110
Aug 72023-08-07 00:00:00 UTC2023-08-14 00:00:00 UTC000
Aug 142023-08-14 00:00:00 UTC2023-08-21 00:00:00 UTC110
Aug 212023-08-21 00:00:00 UTC2023-08-28 00:00:00 UTC000
Aug 282023-08-28 00:00:00 UTC2023-09-04 00:00:00 UTC000
Sep 42023-09-04 00:00:00 UTC2023-09-11 00:00:00 UTC000
Sep 112023-09-11 00:00:00 UTC2023-09-18 00:00:00 UTC000
Sep 182023-09-18 00:00:00 UTC2023-09-25 00:00:00 UTC000
Sep 252023-09-25 00:00:00 UTC2023-10-02 00:00:00 UTC000
Oct 22023-10-02 00:00:00 UTC2023-10-09 00:00:00 UTC000
Oct 92023-10-09 00:00:00 UTC2023-10-16 00:00:00 UTC000
Oct 162023-10-16 00:00:00 UTC2023-10-23 00:00:00 UTC000
Oct 232023-10-23 00:00:00 UTC2023-10-30 00:00:00 UTC000
Oct 302023-10-30 00:00:00 UTC2023-11-06 00:00:00 UTC000
Nov 62023-11-06 00:00:00 UTC2023-11-13 00:00:00 UTC000
Nov 132023-11-13 00:00:00 UTC2023-11-20 00:00:00 UTC000
Nov 202023-11-20 00:00:00 UTC2023-11-27 00:00:00 UTC000
Nov 272023-11-27 00:00:00 UTC2023-12-04 00:00:00 UTC000
Dec 42023-12-04 00:00:00 UTC2023-12-11 00:00:00 UTC000
Dec 112023-12-11 00:00:00 UTC2023-12-18 00:00:00 UTC000
Dec 182023-12-18 00:00:00 UTC2023-12-25 00:00:00 UTC000
Dec 252023-12-25 00:00:00 UTC2024-01-01 00:00:00 UTC000
Jan 12024-01-01 00:00:00 UTC2024-01-08 00:00:00 UTC000
Jan 82024-01-08 00:00:00 UTC2024-01-15 00:00:00 UTC000
Jan 152024-01-15 00:00:00 UTC2024-01-22 00:00:00 UTC000
Jan 222024-01-22 00:00:00 UTC2024-01-29 00:00:00 UTC000
Jan 292024-01-29 00:00:00 UTC2024-02-05 00:00:00 UTC000
Feb 52024-02-05 00:00:00 UTC2024-02-12 00:00:00 UTC000
Feb 122024-02-12 00:00:00 UTC2024-02-19 00:00:00 UTC101

What Hacker News says

6 sources
kordlessagain·Aug 14, 2023Submission

Ask HN: Are you training and running custom LLMs and how are you doing it?

I have been researching methods and projects built around training and running LLMs locally. I'm interested in what others have been using on this front (including straight up Pytorch/Transformers). Here's what I've gathered so far: Engines/APIs: - vllm: Inference and serving engine for LLMs (none quantized models only?) [1] - ollama: Go project to run, create and share LLMs [2] - llama.cpp: Inference of LLaMA models in C/C++ w/UI (including quantized models) [3] - llama-cpp-python: run OpenAI compatible API bindings for llama.cpp [4] - llm-engine: engine for fine-tuning and serving LLMs [5] - Lamini: hosted? closed source? solution for training LLMs [6] - GPT4All: free, locally run chatbot [7] - SkyPilot: framework for running LLMs, AI, and batch jobs [8] - HuggingFace Transformers: APIs and tools to download and train
Read the rest
models (via Pytorch) [9] - RAGStack: Deploy a private ChatGPT alternative hosted within your VPC [14] UI/Interface: - Simon's `llm` tool [15] Quantization Bits: - AutoGPTQ: GPTQ algorithm based quantization package [10] - QLoRA: finetuning of quantized LLMs [11] - bitsandbytes: CUDA functions for PyTorch [12] - SkyPilot QLoRA [13] Video Guides: - https://www.youtube.com/watch?v=eeM6V5aPjhk - https://www.youtube.com/watch?v=TYgtG2Th6fI (jawerty's example) Reference: - [1] https://github.com/vllm-project/vllm - [2] https://github.com/jmorganca/ollama - [3] https://github.com/ggerganov/llama.cpp - [4] https://github.com/abetlen/llama-cpp-python - [5] https://github.com/scaleapi/llm-engine - [6] https://www.lamini.ai/ - [7] https://github.com/nomic-ai/gpt4all - [8] https://github.com/skypilot-org/skypilot - [9] https://github.com/huggingface/transformers/releases - [10] https://github.com/PanQiWei/AutoGPTQ - [11] https://github.com/artidoro/qlora - [12] https://github.com/TimDettmers/bitsandbytes - [13] https://github.com/artidoro/qlora/pull/132 - [14] https://github.com/psychic-api/rag-stack - [15] https://simonwillison.net/2023/Aug/1/llama-2-mac/
15 points · Open discussion
ayanb9440·Aug 3, 2023Submission

Show HN: ChatMyFiles, Open Source ChatPDF

ChatMyFiles is an open-source alternative to ChatPDF: you upload any PDF or Microsoft Office document and ask questions about it. Unlike other "chat with your documents" solutions, ChatMyFiles can be self hosted using a one-click deploy script that uses Terraform to deploy a - Vector database - Server - (Optional) Open-source LLM such as Falcon, Llama, or GPT4All to your virtual private cloud (VPC). We used Langchain to interface with open source LLMs and Ragstack to deploy to Google Cloud: https://github.com/psychic-api/rag-stack
6 points · Open discussion
skanga·Jul 21, 2023Recommendation phrase
I love the idea of a Modelfile but wonder if it can be setup to also retrieve information from other systems and insert them into the LLM’s context window via a prompt. This gives LLMs information beyond what was provided in their training data, which is necessary for almost every enterprise use case. Examples include data from current web pages, data from SaaS apps like Confluence or Salesforce, and data from documents like sales contracts and PDFs. See https://github.com/psychic-api/rag-stack for an example. Note that most of the above verbiage also comes from there!
In Show HN: Ollama – Run LLMs on your Mac
ayanb9440·Jul 20, 2023Submission

Show HN: RAGstack – private ChatGPT for enterprise VPCs, built with Llama 2

Hey hacker news, We’re the cofounders at Psychic.dev (http://psychic.dev) where we help companies connect LLMs to private data. With the launch of Llama 2, we think it’s finally viable to self-host an internal application that’s on-par with ChatGPT, so we did exactly that and made it an open source project. We also included a vector DB and API server so you can upload files and connect Llama 2 to your own data. The RAG in RAGstack stands for Retrieval Augmented Generation, a technique where the capabilities of a large language model (LLM) are augmented by retrieving information from other systems and inserting them into the LLM’s context window via a prompt. This gives LLMs information beyond what was provided in their training data, which is necessary for almost every enterprise application. Examples include data from current web pages,
Read the rest
data from SaaS apps like Confluence or Salesforce, and data from documents like sales contracts and PDFs. RAG works better than fine-tuning the model because it’s cheaper, it’s faster, and it’s more reliable since the provenance of information is attached to each response. While there are quite quite a few “chat with your data” apps at this point, most have external dependencies to APIs like OpenAI or Pinecone. RAGstack, on the other hand, only has open-source dependencies and lets you run the entire stack locally or on your cloud provider. This includes: - Containerizing LLMs like Falcon, Llama2, and GPT4all with Truss - Vector search with Qdrant. - File parsing and ingestion with Langchain, PyMuPDF, and Unstructured.io - Cloud deployment with Terraform If you want to dive into it yourself, we also published a couple of tutorials on how to deploy open source LLMs for your organization, and optionally give it access to internal documents without any data ever leaving your VPC. - How to deploy Llama 2 to Google Cloud (GCP): https://www.psychic.dev/post/how-to-deploy-llama-2-to-google... - How to connect Llama 2 to your own data using RAGstack: https://www.psychic.dev/post/how-to-self-host-llama-2-and-co... Let a thousand private corporate oracles bloom!
84 points · Open discussion

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.