HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
magnitudedev
magnitude
Rust2025–2026
magnitudedev / magnitude

magnitude

Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU.

3Distinct people
13Mentions
0Explicit recommenders
7.8KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 4:43:50 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month13 dated mentions

Observed mentions: 2025-04-25 17:00:35 UTC to 2026-10-01 18:15:34 UTC.

024682025-04-01 00:00:00 UTC to 2025-05-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 commentsApr 20252025-05-01 00:00:00 UTC to 2025-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-06-01 00:00:00 UTC to 2025-07-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2025-07-01 00:00:00 UTC to 2025-08-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-08-01 00:00:00 UTC to 2025-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-09-01 00:00:00 UTC to 2025-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-10-01 00:00:00 UTC to 2025-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-11-01 00:00:00 UTC to 2025-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-12-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-01-01 00:00:00 UTC to 2026-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsJan 20262026-02-01 00:00:00 UTC to 2026-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-01 00:00:00 UTC to 2026-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-04-01 00:00:00 UTC to 2026-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-05-01 00:00:00 UTC to 2026-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-01 00:00:00 UTC to 2026-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-07-01 00:00:00 UTC to 2026-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-08-01 00:00:00 UTC to 2026-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-09-01 00:00:00 UTC to 2026-10-01 00:00:00 UTC (exclusive): 8 mentions; 1 submissions, 7 comments2026-10-01 00:00:00 UTC to 2026-11-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsOct 2026

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Apr 20252025-04-01 00:00:00 UTC2025-05-01 00:00:00 UTC211
May 20252025-05-01 00:00:00 UTC2025-06-01 00:00:00 UTC000
Jun 20252025-06-01 00:00:00 UTC2025-07-01 00:00:00 UTC110
Jul 20252025-07-01 00:00:00 UTC2025-08-01 00:00:00 UTC101
Aug 20252025-08-01 00:00:00 UTC2025-09-01 00:00:00 UTC000
Sep 20252025-09-01 00:00:00 UTC2025-10-01 00:00:00 UTC000
Oct 20252025-10-01 00:00:00 UTC2025-11-01 00:00:00 UTC000
Nov 20252025-11-01 00:00:00 UTC2025-12-01 00:00:00 UTC000
Dec 20252025-12-01 00:00:00 UTC2026-01-01 00:00:00 UTC000
Jan 20262026-01-01 00:00:00 UTC2026-02-01 00:00:00 UTC000
Feb 20262026-02-01 00:00:00 UTC2026-03-01 00:00:00 UTC000
Mar 20262026-03-01 00:00:00 UTC2026-04-01 00:00:00 UTC000
Apr 20262026-04-01 00:00:00 UTC2026-05-01 00:00:00 UTC000
May 20262026-05-01 00:00:00 UTC2026-06-01 00:00:00 UTC000
Jun 20262026-06-01 00:00:00 UTC2026-07-01 00:00:00 UTC000
Jul 20262026-07-01 00:00:00 UTC2026-08-01 00:00:00 UTC000
Aug 20262026-08-01 00:00:00 UTC2026-09-01 00:00:00 UTC000
Sep 20262026-09-01 00:00:00 UTC2026-10-01 00:00:00 UTC817
Oct 20262026-10-01 00:00:00 UTC2026-11-01 00:00:00 UTC101

What Hacker News says

8 sources
Thanks for reporting this issue - assessing is not supposed to take more than a minute or so. This is not strictly necessary but filters out models that don't fit in memory and gives speed estimates. This should ideally be a very short step so skipping hopefully wouldn't feel necessary if we patch this. Could you share your hardware and OS details to help us identify what might be the issue here? There's also a github issue open on this topic if you want to leave a comment there: https://github.com/magnitudedev/magnitude/issues/142
In Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents
Thanks for reporting the issue. Currently we don't support multi-GPU setups, that is on our near-term roadmap. It saying the model is too big for that GPU might be a bug - would you be willing to open a github issue with more detail on your setup? https://github.com/magnitudedev/magnitude/issues As for performance, there may be some variability still depending on the model and backend. We have room for improvement for various setups that we are closing as we work out some details with our kernels and tuning system, so appreciate the data point and will look into that combination.
In Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents
The benchmark we cited here is a simple prose-repetition task. We put the content of Moby Dick up to 64k context in the request, and then ask it to repeat the last section. For llama.cpp, we try to make the comparison as fair as possible by using similar settings. No speculative decoding, default prefill batch sizes, flash attention on. We tried also quantizing the KV cache to 8-bit keys and 4-bit values like we do in Magnitude, but this bombed decode speed for llama.cpp in our testing. Since it seems llama.cpp did not optimize that path, we used 16-bit KV instead. The source for the benchmark is available here also: https://github.com/magnitudedev/magnitude/tree/main/inferenc...
In Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents
anerli·Sep 30, 2026Submission

Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents

Hey HN, Anders and Tom here. We're building Magnitude, an inference engine for agents that optimizes itself to run as fast as possible on your hardware. It works on Mac, Linux, and Windows on any hardware and is up to 2x faster than llama.cpp. We're both software engineers and previously built an open source browser agent to 4k+ GH stars and 100k+ downloads. We increasingly wanted to run it on local models, but found that no inference engine worked for our use case. Inference engines today all make a performance tradeoff. They are either: - Built for batched inference on datacenter hardware at the cost of single-session performance (vLLM, SGLang) - Designed for broad compatibility instead of optimizing for specific hardware (llama.cpp, Ollama) - Specialized for specific hardware or models but lacking engine completeness (oMLX, ds4) Plus n
Read the rest
one of them are designed for running agents locally. Sessions are long, several often run at once, and you still want to use your computer for other things. Magnitude is built for maximum performance on your hardware and running local agents: - On-device compilation and tuning: Kernels are written with flexible parameters that are tuned on your actual device before the model runs. This gives you broad hardware compatibility with the same performance ceiling as hardware-specific kernels. - Focus on best architectures: We write our tunable, highly efficient kernels for the most popular open-weights families. This allows us to achieve and surpass the performance of hardware or model specialized engines, without forcing ourselves to over-generalize at the cost of performance. - Dynamic memory allocation: Magnitude reserves only enough memory up front to hold model weights. As your agent sessions grow, the memory heap dynamically increases, and frees itself when agents stop. Your hardware can still be used for other stuff while agents run. - Hybrid paged attention: We borrow the best ideas from engines like SGLang to allow concurrent sessions to share prefix caches, but optimize placement for memory-adjacency so single-session performance doesn't suffer. Magnitude is fully open source (Apache 2.0). We built it in Rust, including a custom GPU kernel runtime and autotuner. We take inspiration from the best innovations in inference from academics (e.g. FlashAttention, FlashInfer, TurboQuant) as well as other engines (e.g. SGLang radix attention) to reach the performance ceiling. Benchmarked against llama.cpp with Qwen 3.6 35B A3B (4 bit), 64k context, no speculative decoding: Metal (Mac M4 Pro 48 GB) - 92% faster decode (30 tok/s → 57 tok/s) - 9% faster prefill (466 tok/s → 507 tok/s) - 28% less per-agent memory usage CUDA (DGX Spark) - 19% faster decode (49 tok/s → 58 tok/s) - 23% faster prefill (2,033 tok/s → 2,507 tok/s) - 27% less per-agent memory usage Magnitude ships as a desktop app that you can easily connect with whatever agents you already use (Pi, OpenCode, Hermes, Codex, and more). It automatically runs models on demand when these agents actually need them, and shuts them down after inactivity. Here's what it looks like: https://www.youtube.com/watch?v=0qE8BWEZu7o We're excited to push Magnitude further to let you run bigger models on the same hardware while continuing to improve performance. Our plans include: - Expert streaming: store experts on RAM or disk and load them just-in-time. This lets you run models bigger than what otherwise would fit on your GPU. - Kernel compiler: our current kernels tune a few parameters to fit your hardware. We can take this further with a fully custom compiler that automatically chooses how to fuse kernels and which implementations to use, to make it fit to your hardware even better. - Multi-device utilization: Make the best possible use of all hardware on a system (CPU, GPUs, RAM, disk) by detecting these and automatically solving for the best model layout. We'd love for more people to try it out and give us feedback. Feel free to comment here, we'll be around all day!
194 points · Open discussion

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.