HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
oobabooga
textgen
Python2023–2026
oobabooga / textgen

textgen

Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

99Distinct people
132Mentions
0Explicit recommenders
47.7KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 8:59:02 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year132 dated mentions

Observed mentions: 2023-02-26 02:38:28 UTC to 2026-08-08 04:41:38 UTC.

02550751002023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 108 mentions; 6 submissions, 102 comments20232024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 19 mentions; 1 submissions, 18 comments20242025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 4 mentions; 0 submissions, 4 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC1086102
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC19118
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC404
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC101

From the project README

An excerpt from the repository, fetched Nov 12, 2026.

Read the project’s introduction
TextGen A desktop app for local LLMs. Open source, no telemetry. Text, vision, tool-calling, web search. UI + API. Get started in 1 minute Download, unzip, double-click textgen. A window opens. https://github.com/oobabooga/textgen/releases Portable builds for Linux, Windows, and macOS with CUDA, Vulkan, ROCm, and CPU-only options. All dependencies included. Compatible with GGUF (llama.cpp) models. For additional backends (ExLlamaV3, Transformers), training, image generation, and extensions, see Installation. Features Chat & generation instruct mode for instruction-following (like ChatGPT), and chat-instruct/chat modes for talking to custom characters. Prompts are automatically formatted with Jinja2 templates. Vision (multimodal): Attach images to messages for visual understanding (tutorial). File attachments: Upload text files, PDF documents, and .docx documents to talk about their contents. Edit messages, navigate between message versions, and branch conversations at any point. Notebook tab for free-form text generation outside of chat turns. Backends & API Multiple backends: llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM. Switch between backends and models without restarting. OpenAI/Anthropic-compatible API: Chat, Completions, and Messages endpoints with tool-calling support. Use as a local drop-in replacement for the OpenAI/Anthropic APIs (examples). Tool-calling: Models can call custom functions during chat, including web search, page fetching, and math. Each tool is a single .py file. MCP servers are also supported (tutorial). Training & image generation Training: Fine-tune LoRAs on multi-turn chat or raw text datasets. Supports resuming interrupted runs (tutorial). Image generation: A dedicated tab for diffusers models like Z-Image-Turbo. Features 4-bit/8-bit quantization and a persistent gallery with image metadata (tutorial). Privacy & interface 100% offline and private, with zero telemetry, external resources, or remote update requests. Dark/light themes, syntax highlighting for code blocks, and LaTeX rendering for mathematical expressions. Built-in and community extensions including TTS, voice input, and translation. See the extensions directory for the full list. Downloading models Download a GGUF model file from Hugging Face. Place it in the user_data/models folder. That's it. The UI will detect it automatically. For recommended GGUF quants, check out LocalBench. To estimate how much memory a model will use, try the GGUF Memory Calculator. Other model types (Transformers, EXL3) Models that consist of multiple files (like 16-bit Transformers models and EXL3 models) should be placed in a subfolder inside user_data/models: textgen └── user_data └── models └── Qwen_Qwen3-8B ├── config.json ├── generation_config.json ├── model-00001-of-00004.safetensors ├── ... ├── tokenizer_config.json └── tokenizer.json These formats require the full installation (not the portable build). Installation For the desktop app, see the portable builds. The options below run the web UI in your browser instead. Manual portable install with venv Fast setup on any Python 3.9+: # Clone repository git clone https://github.com/oobabooga/textgen cd textgen # Create virtual environment python -m venv venv # Activate virtual environment # On Windows: venv\Scripts\activate # On macOS/Linux: source venv/bin/activate # Install dependencies (choose appropriate file under requirements/portable for your hardware) pip install -r requirements/portable/requirements.txt --upgrade # Launch server (basic command) python server.py --portable --api --auto-launch # When done working, deactivate deactivate Full installation For users who need additional backends (ExLlamaV3, Transformers), training, image generation, or extensions like TTS, voice input, and translation. Requires ~10GB disk space and downloads PyTorch. Installation details One-click installer Clone the repository, or download its source code and extract it. Run the startup script for your OS: start_windows.bat, start_linux.sh, or start_macos.sh. When prompted, select your GPU vendor. After installation, open http://127.0.0.1:7860 in your browser. After installation: Restart: run the same start_ script. Pass command-line flags: directly (e.g., ./start_linux.sh --help), or persist them in user_data/CMD_FLAGS.txt (e.g., --api to enable the API). Update: run the update script for your OS (update_wizard_windows.bat, update_wizard_linux.sh, or update_wizard_macos.sh). Reinstall from scratch: delete the installer_files folder and run the start_ script again. Install extension requirements: use the update wizard's "Install/update extensions requirements" option. It reinstalls the main project requirements at the end to ensure they take precedence over conflicting extension dependencies. Notes: These scripts (start_, update_wizard_, cmd_) don't need to run as admin/root. For automated installation, set the GPU_CHOICE, LAUNCH_AFTER_INSTALL, and INSTALL_EXTENSIONS environment variables. Example: GPU_CHOICE=A LAUNCH_AFTER_INSTALL=FALSE INSTALL_EXTENSIONS=TRUE ./start_linux.sh. Under the hood, the script uses Miniforge to set up a Conda environment in installer_files/. To run anything manually in this environment, launch an interactive shell using cmd_linux.sh, cmd_windows.bat, or cmd_macos.sh. Full installation with Conda 0. Install Conda https://github.com/conda-forge/miniforge On Linux or WSL, Miniforge can be automatically installed with these two commands: curl -sL "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh" > "Miniforge3.sh" bash Miniforge3.sh For other platforms, download from: https://github.com/conda-forge/miniforge/releases/latest 1. Create a new conda environment conda create -n textgen python=3.13 conda activate textgen 2. Install Pytorch System GPU Command Linux/WSL NVIDIA p
Read the full README on GitHub

What Hacker News says

4 sources
I've been looking for this too. It seems to me as though all the ide's are trying to sell the llms as a service or trying to lock you in by downloading llms through their ide. I have been downloading llm's from huggingface as gguf files and would like to use those downloads (and running them through https://github.com/oobabooga/text-generation-webui). It is possible to run those llms as a local api using something like llama-cpp-python (https://pypi.org/project/llama-cpp-python/) and would prefer to use something like that method. Zed (https://zed.dev/), which is now available on windows might be able to do it, but i'd rather use something (foss) that doesn't have a pricing model (the development focus will always be upon those who pay). tbh i'm getting a bit sick of changing ide's, as their support changes, and really would prefer not to
Read the rest
use (microsoft) visual studio code which seems to be cornering the market. Starting to think i'm going to try to learn emacs, with https://github.com/karthink/gptel looking as if it would meet my needs.
In Is there an IDE that can use the local open-source model?
There are also things like Oobabooga's text-generation-webui[0] which can present a similar interface to ChatGPT for local models. I've had great success in running Qwen3-8B-GGUF[1] on my RTX 2070 SUPER (8GB VRAM) using Oobabooga (everyone just calls it via the author's name, it's much catchier) so this is definitely doable on consumer hardware. Specifically, I run the Q4_K_M model as Oobabooga loads all of its layers into the GPU by default, making it nice and snappy. (Testing has shown that I can actually load up to the Q6_K model before some layers have to be loaded into the CPU, but I have to manually specify that all those layers should be loaded into the GPU, as opposed to leaving it auto-determined.) It does obviously hallucinate more often than ChatGPT does, so care should be taken. That said, it's really nice to have something lo
Read the rest
cal. There's a subreddit for running text gen models locally that people might be interested in: https://www.reddit.com/r/LocalLlama [0] https://github.com/oobabooga/text-generation-webui [1] https://huggingface.co/Qwen/Qwen3-8B-GGUF
In When you're asking AI chatbots for answers, they're data-mining you
In my case just CPU (it's a Hetzner server, checked in /proc/cpuinfo and it said "AMD EPYC 9454P 48-Core Processor"). I apparently had still in terminal backlog some stats, so I pasted below. It's not a speed demon but enough to mess around and test things out. Thinking can sometimes be pretty long so it can take a while to get responses, even if 6 tokens/sec is pretty good considering pure CPU setup. --- prompt eval time = 133.55 ms / 1 tokens ( 133.55 ms per token, 7.49 tokens per second) eval time = 392205.46 ms / 2220 tokens ( 176.67 ms per token, 5.66 tokens per second) total time = 392339.02 ms / 2221 tokens And my exact command was: llama-server --model DeepSeek-R1-UD-Q2_K_XL-00001-of-00005.gguf --temp 0.6 -c 9000 --min-p 0.1 --top-k 0 --top-p 1 --timeout 3600 --slot-save-path ~/llama_kv_path --po
Read the rest
rt 8117 -ctk q8_0 (IIRC slot save path argument does absolutely nothing unless and is superfluous, but I have been pasting a similar command around and been too lazy to remove it). -ctk q8_0 reduces memory use a bit for context. I think my 256gb is right at the limit of spilling a bit into swap, so I'm pushing the limits :) The --min-p 0.1 was a recommendation from Unsloth page; I think because the quant is going so low in bits, some things may start to misbehave and it is a mitigation. But I haven't messed around enough to say how true that is, or any nuance about it. I think I put --temp 0.6 for the same reason. To explain to anyone not aware of llama-server: it exposes (a somewhat) OpenAI-compatible API and then you can use it with any software that speaks that. llama-server itself also has a UI, but I haven't used it. I had some SSH tunnels set up to use the server interface with https://github.com/oobabooga/text-generation-webui where I hacked an "OpenAI" client to it (that UI doesn't have it natively). The only reason I use the oobabooga UI is out of habit so I don't recommend this setup to others.
In Questions censored by DeepSeek

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.