ORorbifold·Feb 19, 2019
I get that for scientific/numeric applications Julia is probably at a sweet spot and equally well suited if not better suited than many of its competitors (in many ways it feels like Matlab on steroids). Clearly the stuff you're doing with XLA.jl and the ease of implementation of Neural Differential Equations demonstrate the power of Julia's approach to deep learning.
As you probably know it is hard (and in a precise sense actually theoretically impossible) to design type systems that are both convenient to use (have automatic type inference) and incorporate dependent types (such as shapes of tensors). Julia's solution of using multiple dispatch is not a substitute for such a static type system.
Running into an exception, if there is no method for the particular combination of parameter types you have accidentally produced in a 1000 LOC p
Read the rest
roject might be fine, it will make it pretty hard to refactor anything in a larger project. It results in exactly the same challenges that large python / javascript projects have. Those challenges are well documented and are the reason why large companies move away from those languages to statically typed languages.
This is why I believe a statically typed language with good support for numerical computing will eventually win as the language of choice for large production machine learning applications. Leaving (for now) exotic examples such as Neural Differential Equations aside, current machine learning models would probably actually profit more from some of the type safety features let's say Swift has (using structs for example instead of Tensors of certain shapes) than from more comprehensive support of numerical features. For the most part we are talking about very basic linear algebra after all.
The Swift for Tensorflow design documents argue this pretty convincingly as well.
Regarding the package and module system you might be right, but looking at https://github.com/JuliaTPU/XLA.jl/blob/master/src/XLA.jl and comparing this to how you would assemble a user facing module in lets say OCaml https://github.com/janestreet/core/blob/master/src/core_unix... I personally feel like the later gives me a lot more information how I would use the module and a clear separation from the implementation at the same time, similar things can be said about how this would look like in Swift or even a well written C/C++ header. As far as I can tell package management is still roughly at the level of pip in sophistication, not comparable to solutions like cargo or opam.