jan

GitHub links: - https://github.com/janhq/jan - https://github.com/danny-avila/LibreChat - https://github.com/Mintplex-Labs/anything-llm

Exactly: https://github.com/menloresearch/jan/issues/5474 Can't make it work with ollama endpoint this seems to be the problem but they're not focusing on it: https://github.com/menloresearch/jan/issues/5474#issuecommen...

If you're on Mac you can download LM Studio and get the MLX version (edit - links below). I am running it on 64 gb M1 and it takes about ~30 gb ram. I've been on the hunt for a local orchestrator model that interprets input with speech to text (STT) from WhisperX, then can decide what to do. I have only been running it for a day, but it may be overkill for my setup. For simple tasks it can quickly respond and then understand to use MCP servers for tasks and other things, but offloading all the heavy lifting to claude code via sdk and cli, then bringing the results back in a summary or with clarifying questions as text to speech (TTS). I'm playing with Kyutai TTs b/c have great models that sound real and can do conversational streaming with VAD (though my mbp is too slow with it for now but see https://unmute.sh/ for demo). I am looking fo

If you like simple, try out Jan as well. https://github.com/menloresearch/jan

If you want privacy, use a local models and an open-source chat interface such as OpenWeb-UI or Jan. (avoid proprietary systems such as Msty or LM Studio). https://github.com/janhq/jan https://github.com/open-webui/open-webui

Given how much data (personal and otherwise) that you're likely going to feeding into it, I would HIGHLY recommend using an open-source chat interface like Jan. It's cross platform, works with most LLMs, and has extension support. https://github.com/janhq/jan Another alternative is LibreChat, it's more fully featured but its heavier weight and spins up quite a few docker instances. https://github.com/danny-avila/LibreChat If you're going through the trouble of running models locally, it doesn't make sense to couple that with a proprietary closed source interface (like LM Studio).

Jan [0] is MacOS native, open source, similar feel to the ChatGPT frontend, very polished, and offers Anthropic integration (all Claude models). It also features one-click installation, OpenAI integration, a hub for downloading and running local models, a spec-compatible API server, global "quick answer" shortcut, and more. Really can't recommend it enough! [0] https://github.com/janhq/jan

LibreChat is also one of the few LLM chat interfaces that works with both external APIs (OpenAI, Anthropic, etc) and local ones (hosted via Ollama, etc) out of the box. If it has a downside, I'd say its a little more involved to get setup, lots of docker containers, etc then a more batteries-included approach like Jan. https://github.com/janhq/jan

Have you tried Jan? https://github.com/janhq/jan

A better question would be over something like Jan or LibreChat. Ollama's is CLI/API/backend for easily downloading and running models. https://github.com/janhq/jan https://github.com/danny-avila/LibreChat Jan's probably the closest thing to a open-source LLM chat interface that is relatively easy to get started with. I personally prefer Librechat (which supports integration with image generation) but it does have to spin up some docker stuff and that can make it a bit more complicated.

It's not a perfect solution but you might consider using something like Jan and connecting an API key to GPT4. https://github.com/janhq/jan