Llama-macOS

This MacOS app used to be called LlamaBarn. Really excellent to see the fast progress being made. Official repo, also has documentation how to configure server parameters: https://github.com/ggml-org/Llama-macOS Small tip, install llama.cpp with brew before llama.app, which will pick up the existing llama.cpp. That way it's easier to stay up to date with llama.cpp, since llama.app is on a slower release cadence. Also, models installed with the hugging face CLI (hf) are picked up by llama.app automatically. The CLI will keep the model cache updated, e.g. when models get updated. Llama.cpp became part of Huggingface recently.

This is partly why we're building LlamaBarn. It's a lightweight macOS menu bar app that runs llama-server under the hood, with models stored as standard GGUFs in your Hugging Face cache — the same location llama-server uses by default. No separate model store, no lock-in. https://github.com/ggml-org/LlamaBarn

I switched to using LlamaBarn to manage local models on macOS. https://github.com/ggml-org/llamabarn

it is called llama-barn https://github.com/ggml-org/LlamaBarn

Yeah, but it can be a bit of a tight squeeze if you don't have at least 24gb (preferably 32gb+) of memory. Especially if you want other apps to run at the same time, I think it's safer to stick with something more like 9b. You can see a table with quantized sizes here [0] -- yes, there are smaller quants than Q4_K_XL, but then you're down in the weeds with nickel-and-diming things, and if you want to even keep something like a (memory-hungry) instance of VSCode running, good luck. IMO -- if 9b is doing the job, stick with 9b. 0 - https://github.com/ggml-org/LlamaBarn/pull/63

So great to see my two favorite Open Source AI projects/companies joining forces. Since I don't see it mentioned here, LlamaBarn is an awesome little—but mighty—MacOS menubar program, making access to llama.cpp's great web UI and downloading of tastefully curated models easy as pie. It automatically determines the available model- and context-sizes based on available RAM. https://github.com/ggml-org/LlamaBarn Downloaded models live in: ~/.llamabarn Apart from running on localhost, the server address and port can be set via CLI: # bind to all interfaces (0.0.0.0) defaults write app.llamabarn.LlamaBarn exposeToNetwork -bool YES # or bind to a specific IP (e.g., for Tailscale) defaults write app.llamabarn.LlamaBarn exposeToNetwork -string "100.x.x.x" # disable (default) defaults delete app.llamabarn.LlamaBarn exposeToNetwork

If you're on a Mac and want a simple and open-source way to run models locally, check out our app LlamaBarn: https://github.com/ggml-org/LlamaBarn

Note that there's also "LlamaBarn" (macOS app): https://github.com/ggml-org/LlamaBarn