HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
jundot
omlx
Python2026–2026
jundot / omlx

omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

9Distinct people
15Mentions
0Explicit recommenders
23.6KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 2:19:09 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week15 dated mentions

Observed mentions: 2026-02-13 17:18:43 UTC to 2026-06-29 18:10:43 UTC. Weeks start Monday, UTC.

0122026-02-09 00:00:00 UTC to 2026-02-16 00:00:00 UTC (exclusive): 2 mentions; 2 submissions, 0 commentsFeb 92026-02-16 00:00:00 UTC to 2026-02-23 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-02-23 00:00:00 UTC to 2026-03-02 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-03-02 00:00:00 UTC to 2026-03-09 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2026-03-09 00:00:00 UTC to 2026-03-16 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-16 00:00:00 UTC to 2026-03-23 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-23 00:00:00 UTC to 2026-03-30 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-30 00:00:00 UTC to 2026-04-06 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-04-06 00:00:00 UTC to 2026-04-13 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-04-13 00:00:00 UTC to 2026-04-20 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-04-20 00:00:00 UTC to 2026-04-27 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsApr 202026-04-27 00:00:00 UTC to 2026-05-04 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2026-05-04 00:00:00 UTC to 2026-05-11 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-05-11 00:00:00 UTC to 2026-05-18 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-05-18 00:00:00 UTC to 2026-05-25 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-05-25 00:00:00 UTC to 2026-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-01 00:00:00 UTC to 2026-06-08 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-08 00:00:00 UTC to 2026-06-15 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-06-15 00:00:00 UTC to 2026-06-22 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-22 00:00:00 UTC to 2026-06-29 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-29 00:00:00 UTC to 2026-07-06 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJun 29

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Feb 92026-02-09 00:00:00 UTC2026-02-16 00:00:00 UTC220
Feb 162026-02-16 00:00:00 UTC2026-02-23 00:00:00 UTC110
Feb 232026-02-23 00:00:00 UTC2026-03-02 00:00:00 UTC110
Mar 22026-03-02 00:00:00 UTC2026-03-09 00:00:00 UTC211
Mar 92026-03-09 00:00:00 UTC2026-03-16 00:00:00 UTC000
Mar 162026-03-16 00:00:00 UTC2026-03-23 00:00:00 UTC000
Mar 232026-03-23 00:00:00 UTC2026-03-30 00:00:00 UTC000
Mar 302026-03-30 00:00:00 UTC2026-04-06 00:00:00 UTC101
Apr 62026-04-06 00:00:00 UTC2026-04-13 00:00:00 UTC000
Apr 132026-04-13 00:00:00 UTC2026-04-20 00:00:00 UTC101
Apr 202026-04-20 00:00:00 UTC2026-04-27 00:00:00 UTC101
Apr 272026-04-27 00:00:00 UTC2026-05-04 00:00:00 UTC202
May 42026-05-04 00:00:00 UTC2026-05-11 00:00:00 UTC000
May 112026-05-11 00:00:00 UTC2026-05-18 00:00:00 UTC110
May 182026-05-18 00:00:00 UTC2026-05-25 00:00:00 UTC110
May 252026-05-25 00:00:00 UTC2026-06-01 00:00:00 UTC000
Jun 12026-06-01 00:00:00 UTC2026-06-08 00:00:00 UTC000
Jun 82026-06-08 00:00:00 UTC2026-06-15 00:00:00 UTC101
Jun 152026-06-15 00:00:00 UTC2026-06-22 00:00:00 UTC000
Jun 222026-06-22 00:00:00 UTC2026-06-29 00:00:00 UTC000
Jun 292026-06-29 00:00:00 UTC2026-07-06 00:00:00 UTC101

What Hacker News says

1 sources
I've been working with local models for the past year. There's so many possibilities, but I don't think coding is one. Coding requires so many layers beyond inference; I spent so much time trying to replicate what Claude Code does end to end locally. Understanding all the layers and keeping up with the advancements feels like a slog. Even this article messes up and misunderstands what some of the settings are doing. Qwen in particular seems to work at first, then often gets stuck in thought loops when used for actual work. However, text-to-speech, speech-to-text, and non-code LLM use cases are so useful to have local, and don't require big hardware. Having a universal reliable inference engine interface, I think, is the big unlock that needs to happen before app devs can ship these features. Personal concrete use case: meeting recording a
Read the rest
pp. This uses Parakeet + Qwen to create local transcriptions and post-cleanup, respectively. Right now this app has to download and manage all these models, then bundle an inference engine to run them. It's a lot of code that probably should belong to the OS, or at least a standard interface. While apps can offload some of this to llama.cpp or a similar process over http, that's another set of setup for the user to do before they can have a useful app. Anyway, if you're getting started on a Mac, I'd suggest trying out oMLX (https://github.com/jundot/omlx) before messing with llama.cpp. In particular they have community benchmarks so you can see what kind of performance you're likely to get: https://omlx.ai/benchmarks. I wished each one had more configuration details though.
In Qwen 3.6 27B is the sweet spot for local development

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.