FAfanf2·Jun 23, 2019Submission
The simple essence of automatic differentiation
8 points · Open discussionconal
talk-2018-essence-of-ad
TeX2018–2019
conal / talk-2018-essence-of-ad
talk-2018-essence-of-ad
The simple essence of automatic differentiation
2Distinct people
2Mentions
0Explicit recommenders
201GitHub stars
What to know before exploring
What it is
The simple essence of automatic differentiation
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by month2 dated mentions
Observed mentions: 2018-10-26 07:09:02 UTC to 2019-06-23 22:43:04 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Oct 2018 | 2018-10-01 00:00:00 UTC | 2018-11-01 00:00:00 UTC | 1 | 0 | 1 |
| Nov 2018 | 2018-11-01 00:00:00 UTC | 2018-12-01 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2018 | 2018-12-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 0 | 0 | 0 |
| Jan 2019 | 2019-01-01 00:00:00 UTC | 2019-02-01 00:00:00 UTC | 0 | 0 | 0 |
| Feb 2019 | 2019-02-01 00:00:00 UTC | 2019-03-01 00:00:00 UTC | 0 | 0 | 0 |
| Mar 2019 | 2019-03-01 00:00:00 UTC | 2019-04-01 00:00:00 UTC | 0 | 0 | 0 |
| Apr 2019 | 2019-04-01 00:00:00 UTC | 2019-05-01 00:00:00 UTC | 0 | 0 | 0 |
| May 2019 | 2019-05-01 00:00:00 UTC | 2019-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 2019 | 2019-06-01 00:00:00 UTC | 2019-07-01 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
2 sourcesThere's more information here:
https://github.com/conal/talk-2018-essence-of-ad
That said, has anyone had a look at the paper? What I can't figure out is the assertion that, "In contrast to commonly used RAD implementations, the algorithms defined here involve no graphs, tapes, variables, partial derivatives, or mutation. They are inherently parallel-friendly, correct by construction, and usable directly from an existing programming language with no need for new data types or programming style, thanks to use of an AD-agnostic compiler plugin." That would be fantastic, but that's not immediately obvious to me. Generally, reverse mode is extremely non-parallel friendly because we're essentially traversing a computation graph. That traversal can be done in parallel, but figuring out how to partition that graph is probably more expensive
Read the rest
that traversing it in serial. Now, the paper makes a big deal about not building that graph and passing along the derivative at the same time as the function. Fine. But does that method really produce a more parallel friendly code and is that as efficient as a more traditional graph or tape based method? Unless I'm missing something, I don't see any benchmarks of the method and what would be really helpful would be a ratio of the run time of a routine versus the run time of the routine with the AD method instrumented and the gradient calculated. The big question is how does this ratio hold as the method is scaled up in terms of variables. If the parallel claim holds true, then we should be able to also calculate the resulting parallel efficiencies.
Anyway, I'm genuinely curious if someone has additional information about the computational viability.