HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
jundot
omlx
Python2026–2026
jundot / omlx

omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

9Distinct people
15Mentions
0Explicit recommenders
23.6KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 6:34:31 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week15 dated mentions

Observed mentions: 2026-02-13 17:18:43 UTC to 2026-06-29 18:10:43 UTC. Weeks start Monday, UTC.

0122026-02-09 00:00:00 UTC to 2026-02-16 00:00:00 UTC (exclusive): 2 mentions; 2 submissions, 0 commentsFeb 92026-02-16 00:00:00 UTC to 2026-02-23 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-02-23 00:00:00 UTC to 2026-03-02 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-03-02 00:00:00 UTC to 2026-03-09 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2026-03-09 00:00:00 UTC to 2026-03-16 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-16 00:00:00 UTC to 2026-03-23 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-23 00:00:00 UTC to 2026-03-30 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-30 00:00:00 UTC to 2026-04-06 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-04-06 00:00:00 UTC to 2026-04-13 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-04-13 00:00:00 UTC to 2026-04-20 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-04-20 00:00:00 UTC to 2026-04-27 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsApr 202026-04-27 00:00:00 UTC to 2026-05-04 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2026-05-04 00:00:00 UTC to 2026-05-11 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-05-11 00:00:00 UTC to 2026-05-18 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-05-18 00:00:00 UTC to 2026-05-25 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-05-25 00:00:00 UTC to 2026-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-01 00:00:00 UTC to 2026-06-08 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-08 00:00:00 UTC to 2026-06-15 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-06-15 00:00:00 UTC to 2026-06-22 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-22 00:00:00 UTC to 2026-06-29 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-29 00:00:00 UTC to 2026-07-06 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJun 29

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Feb 92026-02-09 00:00:00 UTC2026-02-16 00:00:00 UTC220
Feb 162026-02-16 00:00:00 UTC2026-02-23 00:00:00 UTC110
Feb 232026-02-23 00:00:00 UTC2026-03-02 00:00:00 UTC110
Mar 22026-03-02 00:00:00 UTC2026-03-09 00:00:00 UTC211
Mar 92026-03-09 00:00:00 UTC2026-03-16 00:00:00 UTC000
Mar 162026-03-16 00:00:00 UTC2026-03-23 00:00:00 UTC000
Mar 232026-03-23 00:00:00 UTC2026-03-30 00:00:00 UTC000
Mar 302026-03-30 00:00:00 UTC2026-04-06 00:00:00 UTC101
Apr 62026-04-06 00:00:00 UTC2026-04-13 00:00:00 UTC000
Apr 132026-04-13 00:00:00 UTC2026-04-20 00:00:00 UTC101
Apr 202026-04-20 00:00:00 UTC2026-04-27 00:00:00 UTC101
Apr 272026-04-27 00:00:00 UTC2026-05-04 00:00:00 UTC202
May 42026-05-04 00:00:00 UTC2026-05-11 00:00:00 UTC000
May 112026-05-11 00:00:00 UTC2026-05-18 00:00:00 UTC110
May 182026-05-18 00:00:00 UTC2026-05-25 00:00:00 UTC110
May 252026-05-25 00:00:00 UTC2026-06-01 00:00:00 UTC000
Jun 12026-06-01 00:00:00 UTC2026-06-08 00:00:00 UTC000
Jun 82026-06-08 00:00:00 UTC2026-06-15 00:00:00 UTC101
Jun 152026-06-15 00:00:00 UTC2026-06-22 00:00:00 UTC000
Jun 222026-06-22 00:00:00 UTC2026-06-29 00:00:00 UTC000
Jun 292026-06-29 00:00:00 UTC2026-07-06 00:00:00 UTC101

What Hacker News says

2 sources
Hi HN, I'm the developer. I built oMLX because every Mac LLM server I tried fell apart with coding agents. The core problem: tools like Claude Code and OpenClaw send requests where the prompt prefix keeps shifting. Most servers invalidate the KV cache after each turn, forcing a full re-prefill. On large contexts (50-100K tokens), that means 30-90 seconds of waiting per request. After a few turns, it's practically unusable. oMLX solves this with paged SSD caching. Every KV cache block is persisted to disk. When a previous prefix comes back, it's restored from SSD instead of being recomputed. In practice, TTFT drops from 30-90s to 1-3s on follow-up requests. Cache blocks survive server restarts too. What's under the hood: - Continuous batching via mlx-lm (multiple concurrent requests) - Multi-model serving (LLM + VLM + Embedding + Reranker
Read the rest
simultaneously, LRU eviction) - OpenAI + Anthropic compatible APIs Tool calling support(JSON, Qwen, Gemma, MiniMax, GLM formats + MCP) - Native macOS menubar app (PyObjC, signed DMG) — download, drag, done As of v0.2.0: Vision-Language Model support with the same tiered caching Benchmarks on M3 Ultra 512GB with Qwen3-Coder-Next-8bit: 58.7 tok/s single request, 243 tok/s at 8x batch. Full results on the README. It reuses LM Studio's model directory, so you don't need to re-download anything. 100% open source (Apache 2.0): https://github.com/jundot/omlx
In Show HN: oMLX – SSD-backed KV cache cuts coding agent TTFT from 90s to 1s on Mac

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.