deltafin

They mention it here: https://github.com/argonautlabsai/deltafin/blob/main/k3-publ... >device bytes read during the prefill window, all four drives (arm csv) 8,977 GB at 24.1 GB/s aggregate I.e. low memory bandwidth.

LLMs tend to care for sequential bandwidth more than random access. From that perspective, Optane probably doesn't offer much of an advantage vs other NVMe drives even from the same era while losing out greatly to newer drives with faster busses to fill. There could be something special to this particular instance but https://github.com/argonautlabsai/deltafin/blob/main/k3-publ... and the overall tok/s numbers seem to suggest it's just the sequential that's playing in.

SSDs are connected via Thunderbolt 5 enclosures. I have one Gen4 and three Gen5 ssds inside enclosures. You can see specs here https://github.com/argonautlabsai/deltafin/tree/main/k3-publ...

Thank you! Here is the short version: Kimi K3, 2.78T parameters, ~1.45 TB of MXFP4 experts streamed from four SSDs on an M5 Max / 128 GB. 1.00 tok/s steady over 512 tokens, 1.13 over 128, ~6.3 min to first token on a 512-token prompt. Output token-identical drafter on/off on a given drive layout; the int8 trunk is non-weight-exact per upstream. The useful bits: one drive gives ≈52% of four, two ≈73%, three ≈90%; and prefill is slow because of ~9 TB of reads for a 1.4 TB model — a scheduling bug with a planned fix. README with per-run logs: github.com/argonautlabsai/deltafin — a fork of gavamedia/deltafin, who built the engine.