Tullio.jl

https://github.com/mcabbott/Tullio.jl is my favorite idea for extending einsum notation to more stuff. Really hope numpy/torch get something comparable!

https://github.com/mcabbott/Tullio.jl: - better syntax (because Julia has proper macro/metaprogramming) - faster - automatically works with GPU arrays.

It reads to me like ATL is actually more of a tensor compiler than a general-purpose language. In fact, I'd actually be curious if we could lower (subsets of) Julia to ATL, let it optimize the tensor expressions and transpile everything back into Julia/LLVM IR for further optimizations. In Julia, there is already Tullio.jl [1], which is basically a front end for tensor expressions, which can target both CPUs and GPUs with AD support and automatically parallelizes the computation. It doesn't really optimize the generated code much right now though, so something like this could be interesting. [1] https://github.com/mcabbott/Tullio.jl

>Higher level abstractions like these? https://github.com/JuliaGPU/KernelAbstractions.jl https://github.com/mcabbott/Tullio.jl https://github.com/JuliaFolds/FoldsCUDA.jl https://juliagpu.gitlab.io/CUDA.jl/usage/array/

Julia ships with OpenBLAS, in some cases there are pure-Julia "blas-like" routine that can be as fast: https://github.com/mcabbott/Tullio.jl

Ask and ye shall receive (more than ye bargained for): https://github.com/mcabbott/Tullio.jl https://github.com/Jutho/TensorOperations.jl https://github.com/under-Peter/OMEinsum.jl Tullio.jl is the most powerful and interesting to me of the above libraries, but they all have cool strengths.

He probably means Tullio.jl (which also seems to integrate with Julia's source to source differentiation library Zygote.jl, the main competitor to Swift for Tensorflow): https://discourse.julialang.org/t/realistically-how-close-is... https://github.com/mcabbott/Tullio.jl Regardless if it can consistently beat Fortran/BLAS in every area, in general JIT languages have more opportunities for optimizations than AoT languages, so it's interesting to see what comes out of a language that focuses on leveraging this to get the most performance.

Note that a lot of Julia are using Julia-based BLAS implementations now (of which there are multiple) https://discourse.julialang.org/t/ann-paddedmatrices-jl-juli... https://github.com/MasonProtter/Gaius.jl https://github.com/mcabbott/Tullio.jl These libraries have been incorporated in various spots in the machine learning, scientific machine learning, differential equation, and other libraries due to how they often outperform BLAS's like OpenBLAS. While the core language still links to OpenBLAS by default, this is up for debate for future versions of Julia given these developments (and movements in the standard library are being done to better incorporate the use of different BLAS implementations).

>Writing a fast CUDA kernel requires quite a bit of low level GPU knowledge False. Not in Julia: https://juliagpu.gitlab.io/KernelAbstractions.jl/ https://github.com/mcabbott/Tullio.jl