Yeah, the dream is to have a "reasoning kernel" that you can plug specific knowledge and skills into ("I know kung fu"). I'm not aware of any progress on the dream though. Plenty of people are making LoRAs for general-purpose open models and there are companies like Thinking Machines that will help you. I'm not sure how you'd choose which base model to start with. I'm also not sure how to draw the boundary between LoRA and RAG.
Some links I found but I can't vouch for them:
https://github.com/oobabooga/textgen/wiki/05-%E2%80%90-Train...
https://thinkingmachines.ai/tinker/
In Ask HN: How can I tell which LLM's have been trained on what?oobabooga
textgen
Python2023–2026
oobabooga / textgen
textgen
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.
99Distinct people
132Mentions
0Explicit recommenders
47.7KGitHub stars
What to know before exploring
What it is
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
- ggml-org/llama.cpp · linked in 4 of the same discussions
- mudler/localai · linked in 3 of the same discussions
- automatic1111/stable-diffusion-webui · linked in 2 of the same discussions
Mentioned in the same Hacker News threads. That is not proof they are substitutes.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year132 dated mentions
Observed mentions: 2023-02-26 02:38:28 UTC to 2026-08-08 04:41:38 UTC.
* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 108 | 6 | 102 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 19 | 1 | 18 |
| 2025 | 2025-01-01 00:00:00 UTC | 2026-01-01 00:00:00 UTC | 4 | 0 | 4 |
| 2026 (partial) | 2026-01-01 00:00:00 UTC | 2027-01-01 00:00:00 UTC | 1 | 0 | 1 |
From the project README
An excerpt from the repository, fetched Nov 12, 2026.
Read the project’s introduction
TextGen
A desktop app for local LLMs. Open source, no telemetry. Text, vision, tool-calling, web search. UI + API.
Get started in 1 minute
Download, unzip, double-click textgen. A window opens.
https://github.com/oobabooga/textgen/releases
Portable builds for Linux, Windows, and macOS with CUDA, Vulkan, ROCm, and CPU-only options. All dependencies included. Compatible with GGUF (llama.cpp) models.
For additional backends (ExLlamaV3, Transformers), training, image generation, and extensions, see Installation.
Features
Chat & generation
instruct mode for instruction-following (like ChatGPT), and chat-instruct/chat modes for talking to custom characters. Prompts are automatically formatted with Jinja2 templates.
Vision (multimodal): Attach images to messages for visual understanding (tutorial).
File attachments: Upload text files, PDF documents, and .docx documents to talk about their contents.
Edit messages, navigate between message versions, and branch conversations at any point.
Notebook tab for free-form text generation outside of chat turns.
Backends & API
Multiple backends: llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM. Switch between backends and models without restarting.
OpenAI/Anthropic-compatible API: Chat, Completions, and Messages endpoints with tool-calling support. Use as a local drop-in replacement for the OpenAI/Anthropic APIs (examples).
Tool-calling: Models can call custom functions during chat, including web search, page fetching, and math. Each tool is a single .py file. MCP servers are also supported (tutorial).
Training & image generation
Training: Fine-tune LoRAs on multi-turn chat or raw text datasets. Supports resuming interrupted runs (tutorial).
Image generation: A dedicated tab for diffusers models like Z-Image-Turbo. Features 4-bit/8-bit quantization and a persistent gallery with image metadata (tutorial).
Privacy & interface
100% offline and private, with zero telemetry, external resources, or remote update requests.
Dark/light themes, syntax highlighting for code blocks, and LaTeX rendering for mathematical expressions.
Built-in and community extensions including TTS, voice input, and translation. See the extensions directory for the full list.
Downloading models
Download a GGUF model file from Hugging Face.
Place it in the user_data/models folder.
That's it. The UI will detect it automatically.
For recommended GGUF quants, check out LocalBench. To estimate how much memory a model will use, try the GGUF Memory Calculator.
Other model types (Transformers, EXL3)
Models that consist of multiple files (like 16-bit Transformers models and EXL3 models) should be placed in a subfolder inside user_data/models:
textgen
└── user_data
└── models
└── Qwen_Qwen3-8B
├── config.json
├── generation_config.json
├── model-00001-of-00004.safetensors
├── ...
├── tokenizer_config.json
└── tokenizer.json
These formats require the full installation (not the portable build).
Installation
For the desktop app, see the portable builds. The options below run the web UI in your browser instead.
Manual portable install with venv
Fast setup on any Python 3.9+:
# Clone repository
git clone https://github.com/oobabooga/textgen
cd textgen
# Create virtual environment
python -m venv venv
# Activate virtual environment
# On Windows:
venv\Scripts\activate
# On macOS/Linux:
source venv/bin/activate
# Install dependencies (choose appropriate file under requirements/portable for your hardware)
pip install -r requirements/portable/requirements.txt --upgrade
# Launch server (basic command)
python server.py --portable --api --auto-launch
# When done working, deactivate
deactivate
Full installation
For users who need additional backends (ExLlamaV3, Transformers), training, image generation, or extensions like TTS, voice input, and translation. Requires ~10GB disk space and downloads PyTorch.
Installation details
One-click installer
Clone the repository, or download its source code and extract it.
Run the startup script for your OS: start_windows.bat, start_linux.sh, or start_macos.sh.
When prompted, select your GPU vendor.
After installation, open http://127.0.0.1:7860 in your browser.
After installation:
Restart: run the same start_ script.
Pass command-line flags: directly (e.g., ./start_linux.sh --help), or persist them in user_data/CMD_FLAGS.txt (e.g., --api to enable the API).
Update: run the update script for your OS (update_wizard_windows.bat, update_wizard_linux.sh, or update_wizard_macos.sh).
Reinstall from scratch: delete the installer_files folder and run the start_ script again.
Install extension requirements: use the update wizard's "Install/update extensions requirements" option. It reinstalls the main project requirements at the end to ensure they take precedence over conflicting extension dependencies.
Notes:
These scripts (start_, update_wizard_, cmd_) don't need to run as admin/root.
For automated installation, set the GPU_CHOICE, LAUNCH_AFTER_INSTALL, and INSTALL_EXTENSIONS environment variables. Example: GPU_CHOICE=A LAUNCH_AFTER_INSTALL=FALSE INSTALL_EXTENSIONS=TRUE ./start_linux.sh.
Under the hood, the script uses Miniforge to set up a Conda environment in installer_files/. To run anything manually in this environment, launch an interactive shell using cmd_linux.sh, cmd_windows.bat, or cmd_macos.sh.
Full installation with Conda
0. Install Conda
https://github.com/conda-forge/miniforge
On Linux or WSL, Miniforge can be automatically installed with these two commands:
curl -sL "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh" > "Miniforge3.sh"
bash Miniforge3.sh
For other platforms, download from: https://github.com/conda-forge/miniforge/releases/latest
1. Create a new conda environment
conda create -n textgen python=3.13
conda activate textgen
2. Install Pytorch
System
GPU
Command
Linux/WSL
NVIDIA
p
Read the full README on GitHubWhat Hacker News says
132 sourcesI've been looking for this too. It seems to me as though all the ide's are trying to sell the llms as a service or trying to lock you in by downloading llms through their ide. I have been downloading llm's from huggingface as gguf files and would like to use those downloads (and running them through https://github.com/oobabooga/text-generation-webui). It is possible to run those llms as a local api using something like llama-cpp-python (https://pypi.org/project/llama-cpp-python/) and would prefer to use something like that method. Zed (https://zed.dev/), which is now available on windows might be able to do it, but i'd rather use something (foss) that doesn't have a pricing model (the development focus will always be upon those who pay). tbh i'm getting a bit sick of changing ide's, as their support changes, and really would prefer not to
Read the rest
use (microsoft) visual studio code which seems to be cornering the market. Starting to think i'm going to try to learn emacs, with https://github.com/karthink/gptel looking as if it would meet my needs.
There are also things like Oobabooga's text-generation-webui[0] which can present a similar interface to ChatGPT for local models.
I've had great success in running Qwen3-8B-GGUF[1] on my RTX 2070 SUPER (8GB VRAM) using Oobabooga (everyone just calls it via the author's name, it's much catchier) so this is definitely doable on consumer hardware. Specifically, I run the Q4_K_M model as Oobabooga loads all of its layers into the GPU by default, making it nice and snappy. (Testing has shown that I can actually load up to the Q6_K model before some layers have to be loaded into the CPU, but I have to manually specify that all those layers should be loaded into the GPU, as opposed to leaving it auto-determined.)
It does obviously hallucinate more often than ChatGPT does, so care should be taken. That said, it's really nice to have something lo
Read the rest
cal.
There's a subreddit for running text gen models locally that people might be interested in: https://www.reddit.com/r/LocalLlama
[0] https://github.com/oobabooga/text-generation-webui
[1] https://huggingface.co/Qwen/Qwen3-8B-GGUF
In my case just CPU (it's a Hetzner server, checked in /proc/cpuinfo and it said "AMD EPYC 9454P 48-Core Processor"). I apparently had still in terminal backlog some stats, so I pasted below.
It's not a speed demon but enough to mess around and test things out. Thinking can sometimes be pretty long so it can take a while to get responses, even if 6 tokens/sec is pretty good considering pure CPU setup.
---
prompt eval time = 133.55 ms / 1 tokens ( 133.55 ms per token, 7.49 tokens per second)
eval time = 392205.46 ms / 2220 tokens ( 176.67 ms per token, 5.66 tokens per second)
total time = 392339.02 ms / 2221 tokens
And my exact command was:
llama-server --model DeepSeek-R1-UD-Q2_K_XL-00001-of-00005.gguf --temp 0.6 -c 9000 --min-p 0.1 --top-k 0 --top-p 1 --timeout 3600 --slot-save-path ~/llama_kv_path --po
Read the rest
rt 8117 -ctk q8_0
(IIRC slot save path argument does absolutely nothing unless and is superfluous, but I have been pasting a similar command around and been too lazy to remove it). -ctk q8_0 reduces memory use a bit for context.
I think my 256gb is right at the limit of spilling a bit into swap, so I'm pushing the limits :)
The --min-p 0.1 was a recommendation from Unsloth page; I think because the quant is going so low in bits, some things may start to misbehave and it is a mitigation. But I haven't messed around enough to say how true that is, or any nuance about it. I think I put --temp 0.6 for the same reason.
To explain to anyone not aware of llama-server: it exposes (a somewhat) OpenAI-compatible API and then you can use it with any software that speaks that. llama-server itself also has a UI, but I haven't used it.
I had some SSH tunnels set up to use the server interface with https://github.com/oobabooga/text-generation-webui where I hacked an "OpenAI" client to it (that UI doesn't have it natively). The only reason I use the oobabooga UI is out of habit so I don't recommend this setup to others.
Can't this kind of repetition be dealt with at the ~~decoder~~ (edit: sampler) level, like for any models? (see DRY ~~decoder~~ sampler for instance: https://github.com/oobabooga/text-generation-webui/pull/5677)
In DeepSeek-R1 with Dynamic 1.58-bit QuantizationStill nothing better than oobabooga (https://github.com/oobabooga/text-generation-webui) in terms of maximalism/"Pro"/"Prosumer" LLM UI/UX ALA Blender, Photoshop, Final Cut Pro, etc.
Embarrassing and any VCs reading this can contact me to talk about how to fix that. lm-studio is today the closest competition (but not close enough) and Adobe or Microsoft could do it if they fired their current folks which prevent this from happening.
If you're not using Oobabooga, you're likely not playing with the settings on models, and if you're not playing with your models settings, you're hardly even scratching the surface on its total capabilities.
In How I run LLMs locallyRunning a Python application using a Windows batch file is not a special task at all. Oobabooga and AUTOMATIC1111 work in the same way. They also have the same issues regarding Windows Defender.
https://github.com/oobabooga/text-generation-webui
https://github.com/AUTOMATIC1111/stable-diffusion-webui
In Show HN: Voice-Pro – AI Voice CloningAfaiu “sampling” here, it is controlled with (not only?) topk and temp parameters in e.g. “text generation web ui”. You may find these in other frontends probably too.
This ofc implies local models and that you have a decent cpu + min 64gb of ram to run above 7b-sized model.
https://github.com/oobabooga/text-generation-webui
https://huggingface.co/models?pipeline_tag=text-generation&s...
In OpenAI, Google and Anthropic are struggling to build more advanced AII understand your situation. It sounds super simple to me now but I remember having to spend at least a week trying to get the concepts and figuring out what prerequisite knowledge I would need between a continium of just using chatgpt and learning relevant vector math etc. It is much closer to the chatgpt side fortunately. I don't like ollama per se (because i can't reuse its models with other frontends due to it compressing them in its own format) but it's still a very good place to start. Any interface that lets you download models as gguf from huggingface will do just fine. Don't be turned off by the roleplaying/waifu sounding frontend names. They are all fine. This is what I mostly prefer: https://github.com/oobabooga/text-generation-webui
In Yi-Coder: A Small but Mighty LLM for CodeUsually it can't be reopened because you can't even really get someone to look at it, because issues are wrongfully closed so frequently that they don't pay attention to complaints about the closures.
Take a look at this issue to see what it takes to keep something open: https://github.com/oobabooga/text-generation-webui/issues/41...
(not especially proud of my reactions there, but I hate being abused, even by robots.)
In I've built my first successful side project, and I hate itXTC: An LLM sampler that boosts creativity, breaks writing clichés
2 points · Open discussionWhy do people recommend this instead of the much better oobabooga text-gen-webui?
https://github.com/oobabooga/text-generation-webui
It's like you hate settings, features, and access to many backends!
In Codestral MambaI’m using a local frontend: https://github.com/oobabooga/text-generation-webui
With local 8x7b: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G... (Q4_K_M)
Requires 35gb+ of regular RAM, RTX card is optional but speeds it up a little. Can’t tell if cloud llms ever exposed this “notebook” mode, I’m in a sanctioned country.
Notebook is basically a raw interface to an llm. No system prompt, no bubbles. You give it a starter text and hit generate, it generates until you hit stop or some system limit triggers. You then hit generate again.
your workflow is to basically just transiently cohabitate in the "dream-state" of the LLM, starting and stopping at arbitrary points
I basically act like voices in a mind of an llm which induce thoughts and direct it without it even noticing, for the most part.
Sometimes, a dialog is still more p
Read the rest
referable. E.g. I can think in a person-help-required mode and then it feels more natural. But I still dialog it there in a notebook tab, where I can adjust both my and its replies at any point, as if I was at the same time one level above. Chat is just a text with A:… B:… lines.
I’m not ND btw, found it by experimenting with writing stories.
Some of the tools offer a path to doing tool use (fetching URLs and doing things with them) or RAG (searching your documents). I think Oobabooga https://github.com/oobabooga/text-generation-webui offers the latter through plugins.
Our tool, https://github.com/transformerlab/transformerlab-app also supports the latter (document search) using local llms.
In Ask HN: What is the current (Apr. 2024) gold standard of running an LLM locally?You can use webui https://github.com/oobabooga/text-generation-webui
Once you get a version up and running I make a copy before I update it as several times updates have broken my working version and caused headaches.
a decent explanation of parameters outside of reading archive papers: https://github.com/oobabooga/text-generation-webui/wiki/03-%...
a news ai website:
https://www.emergentmind.com/
Reddit locallamma and how to prompt an llm:
https://old.reddit.com/r/LocalLLaMA/comments/1atyxqz/better_...
Since you mention silly text generation there is also sillytavern which runs on top of another llm software such as webui.
https://docs.sillytavern.app/
In Ask HN: How to get started with local language models?Showing the 15 most recent of 132 sources. Enable JavaScript to search and browse every indexed mention.