XIxianshou·Dec 14, 2023
Not a complete answer, but here are the most helpful resources for understanding transformer basics in particular:
Original transformer paper: https://arxiv.org/abs/1706.03762
Illustrated transformer: http://jalammar.github.io/illustrated-transformer/
Transformer visualization: https://bbycroft.net/llm
minGPT (Karpathy): https://github.com/karpathy/minGPT
---
Next, some foundational textbooks for general ML and deep learning:
Elements of Statistical Learning (aka the bible): https://hastie.su.domains/ElemStatLearn/
Probabilistic ML: https://probml.github.io/pml-book/book2.html
Deep Learning Book (Goodfellow/Bengio): https://www.deeplearningbook.org/
Understanding Deep Learning:
https://udlbook.github.io/udlbook/
---
Finally, assorted tutorials/resources/intro courses:
Beyond the Illustrated Transformer: https://news.ycombinator.com/item?
Read the rest
id=35712334
AI Zero to Hero: https://karpathy.ai/zero-to-hero.html
AI Canon: https://a16z.com/2023/05/25/ai-canon/
LLM University by Cohere: https://llm.university/
Practical Guide to LLMs: https://github.com/Mooler0410/LLMsPracticalGuide
Practical Deep Learning for Coders: https://course.fast.ai/Lessons/part2.html
---
Hope that helps!