rasbt

LLMs-from-scratch

Jupyter Notebook2024–2026

rasbt / LLMs-from-scratch

LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

AI & Machine Learning Learning & Reference

Explore on GitHub Read Hacker News discussions

22Distinct people

33Mentions

0Explicit recommenders

108.3KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 5:05:26 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year33 dated mentions

Observed mentions: 2024-01-27 16:19:42 UTC to 2026-10-26 21:55:12 UTC.

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
2024	2024-01-01 00:00:00 UTC	2025-01-01 00:00:00 UTC	14	9	5
2025	2025-01-01 00:00:00 UTC	2026-01-01 00:00:00 UTC	14	8	6
2026 (partial)	2026-01-01 00:00:00 UTC	2027-01-01 00:00:00 UTC	5	1	4

What Hacker News says

5 sources

dtf·Oct 26, 2026

There's a note in Sebastian Raschka's LLMs-from-scratch github repository - "Recommendations for Getting the Most Out of a Technical Book". He recommends reading offline and code-free first (although chapter-by-chapter). https://github.com/rasbt/LLMs-from-scratch/blob/cdbd33e6d57a...

In Sebastian Raschka's from-scratch video course on building reasoning models

boredumb·Aug 24, 2026

I agree with mostly all of this, but personally I wrote a toy LLM almost 5 years ago and while it never saw much use outside of boring my wife with a shitty command line demo with glee it did help me understand how they worked and how to apply them, played a lot with JAX and pytorch, ended up building a ghetto version of MCP and an LLM-Pool to proxy requests to my baby local models and so I didn't struggle to see the evolution of openrouter and MCP agentic workflows. The same way i'm really glad when I was younger I built a bad webserver by myself, a really painful SQLx type database, etc etc etc - none of these things led me to developing for Nginx or Oracle nor will knowing JAX get me a job at an AI research lab, but I do have a lot of depth in understanding how the technology works so that the flavors on top of them are easy to digest

Read the rest

and make more use of immediately, and I think the same can be said for engineers coming into the field - if it's a spooky LLM box you aren't going to be squeezing the same amount of juice as the guy that knows how they work inside and out so having at least the understanding of a _babys first LLM_ is going to get you miles ahead of people who don't. For anyone who wants to dork around there is https://github.com/rasbt/LLMs-from-scratch which is something amazing that I think anyone who wants to engineer things around LLMs should at least blast through and read.

In I were 17, I'd learn how to build LLMs from scratch

eigenBasis·May 24, 2026Submission

DeepSeek Sparse Attention

2 points · Open discussion

JoeDaDude·May 5, 2026

Coincidentally, I just started on Build a Large Language Model (From Scratch), a repo/book/course by Sebastian Raschka [0][1][2]. Maybe it is a good problem to have to have to decide which learning resource to use. [0] https://github.com/rasbt/LLMs-from-scratch [1] https://www.manning.com/books/build-a-large-language-model-f... [2] https://magazine.sebastianraschka.com/p/coding-llms-from-the...

In Train Your Own LLM from Scratch

Propelloni·Apr 6, 2026

Great work! I still think that [1] does a better job of helping us understand how GPT and LLM work, but yours is funnier. Then, some criticism. I probably don't get it, but I think the HN headline does your project a disservice. Your project does not demystify anything (see below) and it diverges from your project's claim, too. Furthermore, I think you claim too much on your github. "This project exists to show that training your own language model is not magic." and then just posts a few command line statements to execute. Yeah, running a mail server is not magic, just apt-get install exim4. So, code. Looking at train_guppylm.ipynb and, oh, it's PyTorch again. I'm better off reading [2] if I'm looking into that (I know, it is a published book, but I maintain my point). So, in short, it does not help the initiated or the uninitiated. For

Read the rest

the initiated it needs more detail for it to be useful, the uninitiated more context for it to be understood. Still a fun project, even if oversold. [1] https://spreadsheets-are-all-you-need.ai/ [2] https://github.com/rasbt/LLMs-from-scratch

In Show HN: I built a tiny LLM to demystify how language models work