q38rocm

There's plenty of optimisations still possible if you optimise for specific GPU families. Eg see Hipfire[1] and q38rocm/HaloPFX[2], they're so much more faster than llama.cpp for consumer AMD GPUs. [1] https://github.com/warpfront/hipfire [2] https://github.com/julianmb/q38rocm

The Strix Halo and DGX Spark are pretty danged slow, relatively speaking. I don't recall exact numbers, but with MoE models in this size ballpark (Laguna S 2.1), I seem to recall I was seeing about 20-25 t/s with a big context, which is close to usable. Qwen 3.8 27B crawls on this hardware, though, at 10-16 t/s, definitely not comfortable for interactive use. (Though this makes it seem like you can cook pretty good with a 4-bit ROCmFP4 quantization: https://github.com/julianmb/q38rocm the model does get notably dumber below six bits.) A model similar in size to Laguna S 2.1, but with only 6B active parameters, should be a notable amount faster, so I would imagine 25-30 t/s would be a reasonable guess for where Qwen 3.8 Flash Next will land. DFlash2 might improve all these numbers. It wasn't available last I was testing new models on the S