Zygote.jl

Since the blog post does not have many code samples, this non-trivial AD example with Zygote.jl is worth sharing (it's from their readme): julia> using Zygote julia> fs = Dict("sin" => sin, "cos" => cos); julia> derivative(x -> fs[readline()](x), 1.0) cos -0.8414709848078965 julia> -sin(1.0) # 'true' derivative in 1.0 -0.8414709848078965 So Zygote can apply AD to an anonymous function that looks up a function in a hash table from user input. https://github.com/FluxML/Zygote.jl

This is really cool, and I'm excited to see how this goes. When people first hear about automatic differentiation, their first reaction is "that's cool, why isn't it everywhere?", and the answer has been "it's really hard to use in practice". Until recently, AD systems only worked on restricted subsets of languages, or required extremely cumbersome template usage throughout the code. ML frameworks like TensorFlow pushed AD into the mainstream: they are essentially AD-compatible runtimes, but they are cumbersome to use (you have to use their functions instead of the language-provided ones) and they miss potential optimisations. So it's become increasingly clear that the next step was to actually embed AD as part of a language. I work on Julia, so I hope Zygote.jl (https://github.com/FluxML/Zygote.jl) gets there first, but it's always good