shainet

uning, combining, and doing inference on existing models (and weights). People like to criticize PyTorch, but in fairness, with it I can do useful things with recent models (which matters at the speed of things) pretty much the day they have their open source release (thanks to the transformers library). There exists an equivalent to transformers for Elixir (https://github.com/elixir-nx/bumblebee), but it doesn’t support the top 10 trending models on HuggingFace, which is a problem if you want to use them. Qwen 3, for instance, arguably the most useful open source model family at the moment, isn’t supported at all. And this is the issue all of these libraries face: they are not useful for the majority of people needing libraries like this out there. Also, PyTorch is really well optimized, and there is a good reason that vLLM (yes, it’s not the fastest but pretty fast, supports almost everything, and is the de facto standard for open source LLM inference) is written in Python and PyTorch. Things like Flash Attention 2 are crucial for LLM inference and still hard to find. Yes, OpenXLA is technically very sophisticated, but I never got it working every time I wanted to use it, and JAX seems more comfortable if you want to use TPUs. It’s still a very strange development that Python runs the AI revolution. Multi-node training might be a lot of fun with Elixir, but here performance (especially making sure the CPU is not stalling the GPU) is vital, which might make BEAM not an ideal choice. There is hope though. Recently there have been some interesting developments in the Rust world (https://luminalai.com), and of course no HN comment about training and inference is complete without mentioning TinyGrad. But Elixir simply doesn’t make sense to me (for neural networks). I’m sorry.