neural-engine

You can help with the reverse engineering of Apple Silicon done by a dozen people worldwide, that is how we find out the GPU and NPU instructions[1-4]. There is over 43 trillion float operations per second to unlock at 8 terabit per second 'unified' memory bandwidth and 270 gigabits per second networking (less on the smaller chips).... [1] https://github.com/AsahiLinux/gpu [2] https://github.com/dougallj/applegpu [3] https://github.com/antgroup-skyward/ANETools/tree/main/ANEDi... [4] https://github.com/hollance/neural-engine You can use a high level APIs like MLX, Metal or CoreML to compute other things on the GPU and NPU. Shadama [5] is an example programming language that translates (with Ometa) matrix calculations into WebGPU or WebGL APIs (I forget which). You can do exactly the same with the MLX, Metal or CoreML APIs and only pay ar

> Then we'll see somewhat simpler, less general, and cheaper devices that do "AI" operations with as little excess hardware baggage as possible. Nice. Apple has already been doing this for a few years now. The NPU is totally different from the GPU or CPU on the die itself[1]. Nvidia is likely working on this as well, but I think a device that's a gaming/entertainment/crypto/AI bundle (i.e. sticking with the video card) is probably a better business move. [1] https://github.com/hollance/neural-engine/blob/master/docs/a...

~38 TOPS at fp16 is amazing, if the quoted number if fp16 (ANE is fp16 according to this [1] but that honestly seems like a bad choice when people are going smaller and smaller even at the higher level datacenter cards so not sure why apple would use it instead of fp8 natively) [1]: https://github.com/hollance/neural-engine/blob/master/docs/1...

yep. they have a neural engine that is separate from the CPU and GPU that does really fast matmuls https://github.com/hollance/neural-engine. it's basically completely undocumented and for programs like blas it has more horsepower than all the cores combined. meteorlake and zen4 (8000) also both have npus which are similar. https://www.pcmag.com/news/the-meteor-lake-npu-meet-intels-d...