localGPT

Nice, adding these to my list. Here's a list that I put together, it has active GitHub projects for LLM UIs, ordered by stars: - https://github.com/nomic-ai/gpt4all - https://github.com/imartinez/privateGPT - https://github.com/oobabooga/text-generation-webui - https://github.com/FlowiseAI/Flowise - https://github.com/lobehub/lobe-chat - https://github.com/PromtEngineer/localGPT - https://github.com/h2oai/h2ogpt - https://github.com/huggingface/chat-ui - https://github.com/SillyTavern/SillyTavern - https://github.com/ollama-webui/ollama-webui - https://github.com/Chainlit/chainlit - https://github.com/LostRuins/koboldcpp - https://github.com/ParisNeo/lollms-webui/

On the note of youtube channels, I can wholeheartedly recommend Prompt Engineering [1]. He has great videos and hands-on tutorials, not just about RAG, and also maintains the LocalGPT repo [2] with 17k stars. [1] https://www.youtube.com/@engineerprompt/videos [2] https://github.com/PromtEngineer/localGPT

I think local LLMs are great for tinkerers, and with quantization can run on most modern PCs. I am not comfortable sending over my personal data over to OpenAI/Anthropic, so I've been playing around with https://github.com/PromtEngineer/localGPT/, GPT4All, etc. which keep the data all local. Sliding window chunking, RAG, etc. seem more sophisticated than the other document LLM tools, so I would love to try this out if you ever add the ability to run LLMs locally!

I've used https://github.com/PromtEngineer/localGPT for this and thought it was nice. So I packaged it in a docker container for easy use. docker run -itd --gpus all -p $(PORT):5111 --name llm-local-wizardlm-7b obald/llm-launcher:0.0.2 just use localhost:port in the browser and upload docs then ask questions in the gui. Really nice for easy lookup of rules in boardgames and such. As it provides the relevant text from the docs in addition to the query answer. https://gitlab.com/PeterHedman/llm-local