warp

imi K3 on a laptop https://news.ycombinator.com/item?id=49098966 [ok] 29 Jul 2026 19:54:13 UTC Kimi K3 for local use (1.56TB > 594GB) compressed by Unsloth https://huggingface.co/unsloth/Kimi-K3-GGUF https://news.ycombinator.com/item?id=49102192 [ok] 29 Jul 2026 20:02:40 UTC Kimi k3 now runs on one consumer GPU https://twitter.com/Akashi203/status/2082555972380401852 https://news.ycombinator.com/item?id=49102291 [ok] 30 Jul 2026 06:12:55 UTC Show HN: Run Full Kimi K3 with 29 GB of RAM https://github.com/sqliteai/waste/ https://news.ycombinator.com/item?id=49106591 [ok] 30 Jul 2026 13:05:09 UTC Kimi k3 run on RTX 5090 https://github.com/RightNow-AI/local-kimi https://news.ycombinator.com/item?id=49109455 [ok] 30 Jul 2026 14:01:44 UTC A new inference engine to run Kimi K3 2.78T parameter with 29GB of RAM https://marcobambini.substack.com/p/the-waste-inference-engi... https://news.ycombinator.com/item?id=49110183 [ok] 30 Jul 2026 16:52:58 UTC Running Kimi K3 on a local computer https://github.com/sqliteai/waste https://news.ycombinator.com/item?id=49112587 [ok]

second, with a measured minimum memory requirement of approximately 29 GB at a 4K context. That is obviously not interactive performance yet. But the result we found interesting is that it works at all: this is the full open-weights model, not a distillation, a pruned version, or a smaller model using the Kimi name. The engine is written in C and has no BLAS, CUDA, ONNX, or Python dependency in the inference path. The same code can be used through the CLI, embedded as a library, or exposed through the included OpenAI-compatible server. Correctness was the first constraint. Every layer was validated against a PyTorch reference, with final logits matching within 3.6e-06. The vision tower is supported as well and matches its reference within 2.3e-06. The current bottleneck is understood: K3 needs roughly 17 GB of expert data per token, and more than half of the decode time is spent reading experts from disk. The engine is already operating close to the measured throughput limit of the laptop’s internal SSD. The next improvements therefore need to reduce the number of bytes read per token and increase useful expert reuse without pushing the operating system into paging. K3 is deliberately the extreme case. The same engine runs Kimi-Linear 48B from a 19 GB container at 8.92 tokens per second with an 8 GB memory budget. The broader goal is to make models that are much larger than available RAM usable locally, without sending private data to an API and without requiring specialized accelerator hardware. We have published the engine, container format, conversion tools, benchmarks, validation suite, and also the experiments that failed rather than quietly removing them. Everything is fully open source. Feedback on the storage layout, quantization, caching strategy, direct I/O, portability, and potential optimizations would be very welcome. Contributions of any kind — code, benchmarks, testing on different hardware, documentation, bug reports, or new ideas — are more than appreciated. Repo: https://github.com/sqliteai/waste