samosa-chat

It's hard to believe 16GB unified memory will give you 5 tok/sec unless you are ignoring the thermal warnings. I am running Qwen3.6-35B-A3B on my 16GB M3 and get 7-8 tokens/sec with all the optimizations while keeping the peak memory and thermal warnings at check. https://github.com/deepanwadhwa/samosa-chat

Can't wait to try this. Ornith1 (9B) was a really nice model. I have been running it locally using - https://github.com/deepanwadhwa/samosa-chat

> I am running Qwen3.6-35B-A3B locally on my 16GB mac with 7-9 tokens/second. Link - https://github.com/deepanwadhwa/samosa-chat It's quite telling you didn't use Qwen3.6-35B-A3B locally to build that, seems there was another collaborator ;) Show something you've built with the model+tooling instead, truly dogfood it. I'm sure you'll discover things along the way too!

I have a prediction. By the mid of 2027, we will have >200B MoE models running on basic consumer hardware. I am running Qwen3.6-35B-A3B locally on my 16GB mac with 7-9 tokens/second. Link - https://github.com/deepanwadhwa/samosa-chat This is a GPT4 level model running locally with a decent speed on a 16gb ram macbook air.

>>One reason I am not running local models on my Mac now a days, is My Mac book pro is getting heated more Samosa was optimized for keeping mac's temp in check. This is one of the important reasons for creating this - no excessive wear/tear to the machine. https://github.com/deepanwadhwa/samosa-chat#the-three-princi... maybe give it a try and let me know if your macbook is getting heated while running this?