Kimi-Linear

k at every word every time” trick (full attention) with a clever shortcut called linear attention. The shortcut is packaged into something they call Kimi Delta Attention (KDA). What the numbers mean in plain English 51.0 on MMLU-Pro: on a 4 000-word school-test set, the shortcut scores about as well as the old, slow method. 84.3 on RULER at 128 000 words: on a much longer test it keeps the quality high while running almost four times faster. 6 × faster TPOT: when the AI is writing its reply, each new word appears up to six times sooner than with the previous best shortcut (MLA). 75 % smaller KV cache: the scratch paper is only one-quarter the usual size, so you can fit longer conversations in the same memory. Key pieces explained Full attention: the old, accurate but slow “look back at every word” method. KV cache: the scratch paper that stores which words were already seen. Linear attention: a faster but traditionally weaker way of summarising what was read. Gated DeltaNet: an improved linear attention trick that keeps the most useful bits of the summary. Kimi Delta Attention (KDA): Moonshot’s even better version of Gated DeltaNet. Hybrid 3:1 mix: three layers use the fast KDA shortcut, one layer still uses the old reliable full attention, giving speed without losing smarts. 48 B total, 3 B active: the model has 48 billion total parameters but only 3 billion “turn on” for any given word, saving compute. Context length 1 M: it can keep track of about 1 000 000 words in one go—longer than most novels. Bottom line Kimi Linear lets an AI read very long documents or hold very long conversations with far less memory and much less waiting time, while still giving answers as good as—or better than—the big, slow models we use today.