Gaius.jl

Julia's SIMD programming model is still very much a work in progress; I think we have a way to go in providing the kind of flexibility and control that languages such as ISPC, Halide, TVM, etc... provide. That being said, packages such as SIMD.jl [0], and LoopVectorization.jl [1] are making fantastic progress, to the point that LoopVectorization forms the basis of a legitimate BLAS contender, in pure Julia [2]. It's not totally there yet, but it's close enough that real work is being done in LV at OpenBLAS-like speeds. As an aside, I find it incredible that these kinds of extensions can be built in packages thanks to the fact that Julia's compiler is extensible enough to allow for direct manipulation of the LLVM intrinsics being emitted by user code. [0] https://github.com/eschnett/SIMD.jl [1] https://github.com/chriselrod/LoopVectorizat

Note that a lot of Julia are using Julia-based BLAS implementations now (of which there are multiple) https://discourse.julialang.org/t/ann-paddedmatrices-jl-juli... https://github.com/MasonProtter/Gaius.jl https://github.com/mcabbott/Tullio.jl These libraries have been incorporated in various spots in the machine learning, scientific machine learning, differential equation, and other libraries due to how they often outperform BLAS's like OpenBLAS. While the core language still links to OpenBLAS by default, this is up for debate for future versions of Julia given these developments (and movements in the standard library are being done to better incorporate the use of different BLAS implementations).

You could just use LoopVectorization on the CPU side. It's been shown to match well-tuned C++ BLAS implementations, for example with the pure Julia Gaius.jl (https://github.com/MasonProtter/Gaius.jl), so you can follow that as an example for getting BLAS-speed CPU side kernels. For the GPU side, there's CUDAnative.jl and KernelAbstractions.jl, and indeed benchmarks from NVIDIA show that it at least rivals directly writing CUDA (https://devblogs.nvidia.com/gpu-computing-julia-programming-...), so you won't be missing anything just by learning Julia and sticking to using just Julia for researching new kernel implementations.

A lot of the wrappers are largely legacy at this point. BLAS1 and BLAS2 operations are handled in pure Julia through the broadcast system, there's no performance degradation there and the fusion of operations actually makes for a lot of performance advantages over simple BLAS calls. For BLAS3, tools like LoopVectorization let you write a pure BLAS3 in Julia and get good performance, Gaius.jl is a good example of this by matching OpenBLAS with pure Julia code. Of course you still have to write out how to do blocking, but that's about it. At this point it's mostly needing more coverage before deprecating BLAS bindings in a Julia 2.0 down the line. https://github.com/MasonProtter/Gaius.jl

Actaully, julia for loops can be made to perform at BLAS levels with this package: https://github.com/chriselrod/LoopVectorization.jl. Here are my early experiments at making a pure-julia multi-threaded BLAS using LoopVectorization.jl https://github.com/MasonProtter/Gaius.jl. It absolutely blows a naive triple for loop out of the water and is quite competitive against OpenBLAS until you get to very big sizes.