ds4

Then check out those: https://github.com/prabanta-dev/janas <- this, focuses on Qwen3-Next-80B-A3B. https://github.com/Niko1221/Strata <- Qwen3.8-Flash-Next. I use this (tuned) and I cannot understate how satisfied I am. https://github.com/antirez/ds4 <- DeepSeek 4 family + more.

Projects like DwarfStar https://github.com/antirez/ds4 really lower the hardware bar a lot so Deepseek 4.1 flash and other mixture of expert models can run on consumer hardware. There are also other inference providers who make their money serving openweight models. Services like OpenRouter make it all too easy to utilize these models. Access to these models isn't hard. The hardware moat is becoming pretty easy to bridge.

Dwarfstar already supports this, curious how it compares, but I use the q4 quant daily and it works really well. https://github.com/antirez/ds4/blob/main/docs/MODELS.md#qwen...

from the project https://github.com/antirez/ds4 it was said that: > It must be more a working template for the biggest use cases, without trying to cover every possible setup. this is such a great quote about building software in the AI age. given that everyone has a different use case, the best way for open source software to be built is to build it for the general use case and specialization could be done afterwards.

https://github.com/antirez/ds4 The project GitHub page is a much better introduction for the hn crowd.

It is pretty nifty. I spend some time over last weekend implementing fused TQ to allow for 1m context lengths on a 128 gb MacBook M5 Max when using Qwen 3.8 flash next (https://github.com/antirez/ds4/pull/1115 if you are interested). If I get bored I might port over the Metal kernels from oMLX -- the speed increase they have for the v0.7.0 release is amazeballs.

Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent. Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9... Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of the

Antirez just landed a commit[1] to the github repo for DwarfStar that adds support for his heavily quantized variant of DeepSeek V4.1 Flash. He claims it runs with SSD streaming on a macbook pro M5-Max 128 GB laptop. The code was just checked in literally hours ago, so take YMMV quite literally. [1] https://github.com/antirez/ds4/commit/bd66c402070042bf0a79ad6ece8242de4c93680c

Hey there! I do the same but I use dwarfstar at a 2-bit quant: https://github.com/antirez/ds4 I'm curious if you've tried dwarfstar and decided to move to llama.cpp and 3 bit quants or what made you go that route instead? I've been using ds4 for months now and it's already got support for the new vision model, haven't tried it yet, still on 0731 but it's been very solid for me.

Something Cherny said in early December last year is clearly not relevant anymore. By late December he said: "100% of my contributions to Claude Code were written by Claude Code"[1]. That's production software shipping to millions of people. Antirez's Dwarfstar is also mostly AI written: > This software is developed with strong assistance from GPT 5.5, 5.6, Claude Fable and with humans leading the ideas, testing, and debugging. We say this openly because it shaped how the project was built. If you are not happy with AI-developed code, this software is not for you. [2] I'm actually pretty shocked anyone would claim otherwise. In January 2026, sure, but the world has changed since then. Many, many places are doing 100% AI code now, and yes for production code. https://www.businessinsider.com/ai-writing-all-startup-code-... [1] https://x.com

Primarily used ds4[0] by antirez [0]https://github.com/antirez/ds4

I agree about getting the Ultra if you're interested in AI (LLM) inference speed. I'm a little perplexed as to why there isn't a RAM option in between 96GB and 256GB, though. For instance, I believe Deepseek v4 flash runs a lot faster on (https://github.com/antirez/ds4) with 128GB than 96. I assume the higher SKUs have low enough sales that Apple doesn't want to be manufacturing too many different RAM configs as that would eat into their margins. (I say this as someone interested in AI hardware in general, even the 96GB is out of my price range).

Your analysis is inclonclusive, you are not considering high-optimized inference engine, SSD-streaming approach and other optimization https://github.com/antirez/ds4