SESemanticStrenghĀ·May 4, 2022
The most next gen autodiff library probably is https://github.com/breandan/kotlingrad because of its features, ergonomy and type safety
In Jax vs. Julia (Vs PyTorch)breandan
kotlingrad
Kotlin2019ā2022
breandan / kotlingrad
kotlingrad
š§© Shape-Safe Symbolic Differentiation with Algebraic Data Types
4Distinct people
13Mentions
0Explicit recommenders
545GitHub stars
What to know before exploring
What it is
š§© Shape-Safe Symbolic Differentiation with Algebraic Data Types
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7āNov 14 UTC Ā· Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year13 dated mentions
Observed mentions: 2019-01-18 18:25:36 UTC to 2022-05-04 20:52:00 UTC.
Counts come from the repositoryās dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 6 | 2 | 4 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 1 | 1 | 0 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 4 | 1 | 3 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 2 | 0 | 2 |
What Hacker News says
13 sourcesHey! Thanks for the link. We decided to postpone our coroutine-based implementation until native support for multi-shot delimited continuations becomes more stable. The linked section may be found here [1], for posterity.
[1]: https://github.com/breandan/kotlingrad/tree/c02ac55325c05a2e...
In Trade-Offs in Automatic Differentiation: TensorFlow, PyTorch, Jax, and JuliaNote however that Kotlin is backed by Facebook https://ai.facebook.com/blog/paving-the-way-for-software-20-...
and that there is a mature library for autodiff https://github.com/breandan/kotlingrad
In Trade-Offs in Automatic Differentiation: TensorFlow, PyTorch, Jax, and JuliaShow HN: Kotlinā: Shape-Safe Symbolic Differentiation with Algebraic Data Types
15 points Ā· Open discussionI really don't see how Scala is not fit for this. Swift for Tensorflow was mainly revolving around the Differentiable protocol.
There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1]
DeepLearning.scala that does compile-time reverse AD. [2]
Compute.scala for operator fusion at runtime (JITifying) [3]
Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time.
Moreover, Kotlin also has a library that does type-safe AD. [5]
nVidia and Oracle want better interopabiltiy between CUDA and GraalVM through grCuda. [6]
[1] Demystifying Differentiable Programming:
Shift/Reset the Penultimate Backpropagator - https://arxiv.org/pdf/1803.10228.pdf
[2] https://github.com/ThoughtWorksInc/DeepLearning.scal
Read the rest
Programming Tools for Intelligent Systems [pdf]
3 points Ā· Open discussionAuthor here, thank you for your kind remarks! Youāre right, the backticks surrounding our type level integers are too distracting. The Kotlin grammar [1] requires class names must either start with a letter, or be enclosed by back-ticks. Unless the Kotlin language designers reconsider the identifier scheme [2] (?), we would be better off replacing these with something like T1, T2, ..., TN for readability and to minimize confusion.
Re: Shape Inference. To properly understand how this works, the best place to start is by looking at ToyVectorExample.kt [3]. Once youāre comfortable with that, ToyMatrixExample.kt is a fairly direct generalization. The best way to explain our type system is that we stick to equality checking and perform manual type conversion for tensor contractions and expansions. Havenāt figures out a good way to do concatena
Read the rest
tion, slicing, or convolution which require type-level arithmetic. This may be feasible for a fixed number set of predefined shapes, but seems to be generally intractable in the Java/Kotlin type system without a significant overhead. But if someone has an idea, I would be very happy to be proven wrong about this!
[1]: https://kotlinlang.org/docs/reference/grammar.html
[2]: https://github.com/Kotlin/kotlin-spec/blob/master/grammar/sr...
[3]: https://github.com/breandan/kotlingrad/blob/master/src/main/...
[4]: https://github.com/breandan/kotlingrad/blob/master/src/main/...
Thanks for checking out our project! Completely understand where you're coming from, I also struggled with imagining how differentiable programs could work, after all, computer programs implement discrete functions! When building it, I learned two surprising facts:
(1) We can perform gradient descent on integer functions! [1] There are rigorous assumptions for differentiability (e.g. Lipschitz-continuity, Ī²-smoothness), but most often, these don't apply in the real world. Although many discrete optimization problems can relaxed to semi-differentiable ones, even if a function is not strictly differentiable, we can still derive an approximately correct expression for the gradient.
(2) It is possible to symbolically derive functions in discrete spaces! There are many, many examples of this in various settings (e.g. Brzonski's derivative [2]
Read the rest
, Lambda calculus [3], Boolean algebra [4], et al. [5]). A number of researchers are studying generalized differentiation, or "change propagation" in other topological spaces [6]. Although still very abstract and hard to interpret, these ideas have important applications for incremental computation and together with probabilistic programming [7], may one day lead to an analogous mechanism to gradient descent on a much broader class of functions.
With this in mind, Kotlinš implements symbolic differentiation. Whenever the user writes "+" or "*" in their program and later asks for the derivative, we apply the sum and product rules, and do not consider the question of domain validity. For floating point numbers, we try to perform some consistency checks using finite differences, but ultimately, leave this up to the user to verify whether their their domain is a valid one.
Shameless plug, but if you find this interesting, you can learn more about this in Chapter 3 of my M.S. thesis [8]. Please don't hesitate to get in touch if you have any questions or would like to chat!
[1]: https://openreview.net/pdf?id=HJGXzmspb
[2]: http://maveric.uwaterloo.ca/reports/1964_JACM_Brzozowski.pdf
[3]: https://core.ac.uk/download/pdf/82396223.pdf
[4]: https://pdfs.semanticscholar.org/a9c8/7568463a5ce8c2cd15ec17...
[5]: https://en.wikipedia.org/wiki/Generalizations_of_the_derivat...
[6]: https://arxiv.org/pdf/1902.05465.pdf
[7]: https://www.cs.ubc.ca/~fwood/CS532W-539W/
[8]: https://github.com/breandan/kotlingrad/blob/master/latex/the...
Hello HN, author here! Been working on this DSL for differentiable programming for a while now. The API is starting to stabilize and it would be great to collect feedback from real users. You can find some toy examples, including the core of our implementation here:
https://github.com/breandan/kotlingrad/tree/master/src/main/...
Please feel free to reach out if you have any questions or comments. Thanks so much for your feedback!
In Kotlingrad: A shape-safe DSL for differentiable programmingKotlingrad: A shape-safe DSL for differentiable programming
76 points Ā· Open discussionIf you're interested in these kinds of features but aren't up to learning Haskell, we've been writing a DSL for type-safe array programming in a language called Kotlin. It is possible to express certain type-level operations like shape checking and inference for multidimensional arrays (under certain constraints). You can read more about our approach here: https://github.com/breandan/kotlingrad
In Dex: A research language for array processing in the Haskell/ML familyKotlinš: Differentiable Functional Programming with Algebraic Data Types
1 points Ā· Open discussion