luminal

aries in that we ahead-of-time compile everything, generate a large search space of logically-equivalent kernels, and search through it to find the fastest kernels. This allows us to leverage the Bitter Lesson to discover complex optimizations like Flash Attention entirely automatically without needing manual heuristics. The best rule is no rule, the best heuristic is no heuristic, just search everything. We’re working on bringing CUDA support up to parity with Metal, adding more flexibility to the search space, adding full-model examples (like Llama), and adding very exotic hardware backends. We aim to radically simplify the ML ecosystem while improving performance and hardware utilization. Please check out our repo: https://github.com/luminal-ai/luminal and I’d love to hear your thoughts!

ck in depending on which backend we're running on. Luminal takes this approach to the extreme, supporting only 11 primitive operations (primops): - Unary - Log2, Exp2, Sin, Sqrt, Recip - Binary - Add, Mul, Mod, LessThan - Other - SumReduce, MaxReduce, Contiguous Every complex operation boils down to these primitive operations, so when you do a - b for instance, add(a, mul(b, -1)) gets written to the graph. Or when you do a.matmul(b), what actually gets put on the graph is sum_reduce(mul(reshape(a), reshape(b))). Once the graph is built, iterative compiler passes can modify it to replace primops with more efficient ops, depending on the device it's running on. On Nvidia cards, for instance, efficient Cuda kernels are written on the fly to replace these ops, and specialized cublas kernels are swapped in for supported operations. This approach leads to a simple library, and performance is only limited by the creativity of the compiler programmer, not the model programmer. Luminal has a number of other neat features, check out the repo here: https://github.com/jafioti/luminal