krasis

> We've all heard people say that local Qwen 27B or 35-A3B is "near-Opus level" Uh, so, yeah. Im running local Qwen, but Qwen3.5-122B using Krasis https://github.com/brontoguana/krasis Its by far better than Opus. In fact with a phone migration, I was using an OLD android 2fa app "andOTP". Backup files it emitted were JSON but not any sort of standard. I needed the standards version using otpauth:// to upload in my current 2fa. And gave it to my local qwen3.5-122b. It responded with a scary "you uploaded credentials to a public instance LLM! And, it emitted standards compliant URLs. The new app "Tokn" ingested just fine. When side by side was tested, everything was 100% correct. I coukd have did it myself, but it was a one-off. And asking local Qwen worked perfectly. Took like 6 minutes. Would have taken me 1h.

Check out Krasis https://github.com/brontoguana/krasis It enables something similar to unified memory. Ive got a 5060 (16GB) card and 96 GB ddr5. I can run qwen3.5-122b int4 at 25tok/sec.And now even does image ingestion! Ive been bulk transliterating and translating foreign language books into english. And all completely local.

https://github.com/brontoguana/krasis On my desktop RTX 5060 TI (16GB) and 96GB ram, I routinely get 25-30 tokens/sec using an 80B model quantized to int8. Uses 65GB system ram and 15GB gfx ram. And its plenty fast for many of my purposes. I could easily run a 30B model bf16 (full) and do like 50tok/s