mlx

I thought this seemed significant at the time: https://github.com/ml-explore/mlx/pull/1983

Hi HN, I ported the Python MLX bindings to Ruby (https://github.com/ml-explore/mlx). GitHub: https://github.com/skryl/mlx-ruby MLX-Ruby is a native C++ extension that wraps the upstream MLX runtime, giving Ruby full access to the array framework, neural network layers, optimizers, and Metal GPU acceleration on Apple silicon. What’s included: ∙Lazy arrays with dynamic graph construction ∙Function transforms: grad, value_and_grad, vmap, jvp, vjp, compile ∙Full NN module system: Conv2d, Linear, Embedding, Transformer layers, RNNs, etc. ∙Optimizers: Adam, AdamW, SGD, and more ∙A Ruby DSL for declarative model definition, training loops, and checkpointing ∙CPU and Metal GPU support Working examples (https://github.com/skryl/mlx-ruby-examples): ∙LLaMA inference ∙Stable Diffusion ∙Whisper ∙Transformer language model ∙LoRA fine-tuning

Sure, I directly and explicitly talked about Apple's version of tensor cores in the GPU. But the ANE is by every definition a neural accelerator. Yes, I'm aware of Apple's weird branding for their tensor cores. "In fact MLX does not even support ANE yet" I didn't say otherwise. The ANE is a fantastic unit for small, power-efficient models, like extracting text from images, doing depth modelling, etc. It's not made for LLMs, or the other sorts of experimental stuff MLX is intended for. Though note that MLX's author's reason for not supporting the ANE is that it has a "closed-source" API (https://github.com/ml-explore/mlx/issues/18#issuecomment-184...), making it unsuitable for an open-source project, and given that MLX didn't want to just lean on CoreML. But anyways, the ANE is fantastically fast at what it does, while sipping juice. In an

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083

Interesting post, but the last bit of logic pointing to the Neural Engine for MLX doesn’t hold up. MLX supports running on CPU, Apple GPU via Metal, and NVIDIA GPU via CUDA: https://github.com/ml-explore/mlx/tree/main/mlx/backend

Noteworthy: this is powered by a new ultra-low-latency userspace TB5 driver offering an ibverbs/RDMA interface over thunderbolt 5[1]. Non-inference HPC workloads could see a massive benefit as well. [1]: See MLX integration PR at https://github.com/ml-explore/mlx/pull/2808

MLX does not support the ANE. https://github.com/ml-explore/mlx/issues/18

They're working—almost done—on a CUDA backend for their Apple Silicon framework: https://github.com/ml-explore/mlx/pull/1983

Apple have certainly taken a couple steps back re: overall reliability, but if you think that the grass is greener on the other side…pray tell how that goes Plus, you can now deploy [MLX projects on CUDA](https://github.com/ml-explore/mlx/pull/1983)

MLX github: https://github.com/ml-explore/mlx get started: https://developer.apple.com/videos/play/wwdc2025/315/ details: https://developer.apple.com/videos/play/wwdc2025/298/

How does this work when one of the key features of MLX is using a unified memory architecture? (see bullets on repo readme: https://github.com/ml-explore/mlx ) I would think that bringing that to all UMA APUs (of any vendor) would be interesting, but discreet GPU's definitely would need a different approach? edit: reading the PR comments, it appears that CUDA supports a UMA API directly, and will transparently copy as needed.

I wonder if Apple ever followed up with this: https://github.com/apple/ml-ane-transformers They claim their ANE-optimized models achieve "up to 10 times faster and 14 times lower peak memory consumption compared to baseline implementations." AFAIK, neither MLX nor llama.cpp support ANE. Though llama.cpp started exploring this idea [0]. What's weird is that MLX is made by Apple and yet, they can't support ANE given its closed-source API! [1] [0]: https://github.com/ggml-org/llama.cpp/issues/10453 [1]: https://github.com/ml-explore/mlx/issues/18#issuecomment-184...

For learning internals of ML frameworks I recommend reading the source code of MLX: https://github.com/ml-explore/mlx . It is a modern and clean codebase without legacies, and I could understand most things without seeking external articles.