SMSmerity·Jun 5, 2017
With all due respect, this is a quite random reading list, and appears more a result of the 'if "deep learning" in post.title: post.upvote()' trend on Hacker News ...
I will pick on two under the "Classics" section simply as I know the author and have used their work, so am not in any way saying the work isn't useful (it certainly can be in the right spot!), but it isn't "classic" or what I'd recommend for early readers at all.
"Uncertainty in Deep Learning" and "Dropout as a Bayesian Approximation" were both published within the last year and a half and is a PhD thesis + paper on interpretations of neural networks in a Bayesian fashion. "Classic" for a paper + thesis less than two years old is quite a stretch even for the fast moving field of deep learning.
The same holds true for many of the other papers in "Classics" such as batch norm
Read the rest
which is (a) recent and (b) certainly not the only of such techniques (see layer norm, recurrent batch norm, ...) and (c) has complications in implementation[1].
As the simplest example, why is the original dropout paper[7] not under classics? It's an elegant paper, fundamentally important for current neural networks, and is more classic than "Dropout as a Bayesian Approximation" or "Dropout Rademacher Complexity of Deep Neural Networks" which are both listed.
I'm also highly dubious of the noted neuroscience connection - most deep learning researchers use very little from neuroscience.
Again, this list may be helpful to the creator of the repo and tailored toward their specific research direction but it is not useful for readers from Hacker News or those aiming to get their start in deep learning. Why so many upvotes? Zero comments? Zero discussion?
If you want a book, check out the Deep Learning book[2]. If you want a course for RNNs, check out CS224d[3]. If you want a course for CNNs, check out CS231n[4]. If you want to get down and dirty in a practical software engineering way, check out Fast AI[5]. If you want summaries of select recent deep learning papers in GitHub format, check out Denny Britz's notes[8]. There are many other starting points but those are my default suggestions.
If you really want to start learning, this isn't the right list for you and I'd really like to suggest a more sane and potentially tailored path. Seriously. If you reply with what you want, I'll do my best to suggest a starting point.
Background: I'm a deep learning researcher who publishes papers and articles[6].
[1]: http://www.alexirpan.com/2017/04/26/perils-batch-norm.html
[2]: http://www.deeplearningbook.org/
[3]: http://cs224d.stanford.edu/
[4]: http://cs231n.github.io/
[5]: http://course.fast.ai/
[6]: http://smerity.com/articles/2016/google_nmt_arch.html
[7]: https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf
[8]: https://github.com/dennybritz/deeplearning-papernotes