HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
KellerJordan
modded-nanogpt
Python2024–2026
KellerJordan / modded-nanogpt

modded-nanogpt

NanoGPT (124M) in 90 seconds

16Distinct people
21Mentions
1Explicit recommenders
6KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

NanoGPT (124M) in 90 seconds

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

> Performance of BDH-GPU and GPTXL versus model size on a translation task. [...] On the other hand, GPTXL [...] required Dropout [...] The model architecture follows GPT2 I…

kouteiheika used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 6:28:55 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month21 dated mentions

Observed mentions: 2024-10-12 20:29:05 UTC to 2026-09-29 01:47:21 UTC.

012342024-10-01 00:00:00 UTC to 2024-11-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsOct 20242024-11-01 00:00:00 UTC to 2024-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-12-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-01-01 00:00:00 UTC to 2025-02-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2025-02-01 00:00:00 UTC to 2025-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-03-01 00:00:00 UTC to 2025-04-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-04-01 00:00:00 UTC to 2025-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-05-01 00:00:00 UTC to 2025-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-06-01 00:00:00 UTC to 2025-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-07-01 00:00:00 UTC to 2025-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-08-01 00:00:00 UTC to 2025-09-01 00:00:00 UTC (exclusive): 4 mentions; 0 submissions, 4 comments2025-09-01 00:00:00 UTC to 2025-10-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsSep 20252025-10-01 00:00:00 UTC to 2025-11-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2025-11-01 00:00:00 UTC to 2025-12-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025-12-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-01-01 00:00:00 UTC to 2026-02-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2026-02-01 00:00:00 UTC to 2026-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-01 00:00:00 UTC to 2026-04-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-04-01 00:00:00 UTC to 2026-05-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-05-01 00:00:00 UTC to 2026-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-01 00:00:00 UTC to 2026-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-07-01 00:00:00 UTC to 2026-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-08-01 00:00:00 UTC to 2026-09-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2026-09-01 00:00:00 UTC to 2026-10-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsSep 2026

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Oct 20242024-10-01 00:00:00 UTC2024-11-01 00:00:00 UTC110
Nov 20242024-11-01 00:00:00 UTC2024-12-01 00:00:00 UTC000
Dec 20242024-12-01 00:00:00 UTC2025-01-01 00:00:00 UTC000
Jan 20252025-01-01 00:00:00 UTC2025-02-01 00:00:00 UTC202
Feb 20252025-02-01 00:00:00 UTC2025-03-01 00:00:00 UTC000
Mar 20252025-03-01 00:00:00 UTC2025-04-01 00:00:00 UTC101
Apr 20252025-04-01 00:00:00 UTC2025-05-01 00:00:00 UTC000
May 20252025-05-01 00:00:00 UTC2025-06-01 00:00:00 UTC101
Jun 20252025-06-01 00:00:00 UTC2025-07-01 00:00:00 UTC000
Jul 20252025-07-01 00:00:00 UTC2025-08-01 00:00:00 UTC000
Aug 20252025-08-01 00:00:00 UTC2025-09-01 00:00:00 UTC404
Sep 20252025-09-01 00:00:00 UTC2025-10-01 00:00:00 UTC101
Oct 20252025-10-01 00:00:00 UTC2025-11-01 00:00:00 UTC303
Nov 20252025-11-01 00:00:00 UTC2025-12-01 00:00:00 UTC101
Dec 20252025-12-01 00:00:00 UTC2026-01-01 00:00:00 UTC000
Jan 20262026-01-01 00:00:00 UTC2026-02-01 00:00:00 UTC202
Feb 20262026-02-01 00:00:00 UTC2026-03-01 00:00:00 UTC000
Mar 20262026-03-01 00:00:00 UTC2026-04-01 00:00:00 UTC110
Apr 20262026-04-01 00:00:00 UTC2026-05-01 00:00:00 UTC101
May 20262026-05-01 00:00:00 UTC2026-06-01 00:00:00 UTC000
Jun 20262026-06-01 00:00:00 UTC2026-07-01 00:00:00 UTC000
Jul 20262026-07-01 00:00:00 UTC2026-08-01 00:00:00 UTC000
Aug 20262026-08-01 00:00:00 UTC2026-09-01 00:00:00 UTC211
Sep 20262026-09-01 00:00:00 UTC2026-10-01 00:00:00 UTC110

What Hacker News says

3 sources
Scaling applies to multiple dimensions simultaneously over time. A frontier model today could be replicated a year later with a model half the size, with a quarter of the FLOPS, etc. I don’t know the real numbers for optimization scaling, but you could check out NanoGPT speedrun [1] as an example. The best solution in the meantime is giving the LLM a harness that allows tool use like what coding agents have. I suspect current models are fully capable of solving arbitrary complexity artificial reasoning problems here, provided that they’re used in the context of a coding agent tool. [1] https://github.com/KellerJordan/modded-nanogpt
In Reasoning models reason well, until they don't
kouteiheika·Oct 22, 2025Recommendation phrase
> Performance of BDH-GPU and GPTXL versus model size on a translation task. [...] On the other hand, GPTXL [...] required Dropout [...] The model architecture follows GPT2 I love when a new architecture comes out, but come on, it's 2025, can we please stop comparing fancy new architectures to the antiquated GPT2? This makes the comparison practically, well, useless. Please pick something more modern! Even the at-this-point ubiquitous Llama would be a lot better. I don't want to have to spend days of my time doing my own benchmarks to see how it actually compares to a modern transformer (and realistically, I was burned so many times now that I just stopped bothering). Modern LLMs are very similar to GPT2, but those architectural tweaks do matter and can make a big difference. For example, take a look at the NanoGPT speedrun[1] and look at
Read the rest
how many training speedups they got by tweaking the architecture. Honestly, everyone who publishes a paper in this space should read [2]. The post talks about optimizers, but this is also relevant to new architectures too. Here's the relevant quote: > With billions of dollars being spent on neural network training by an industry hungry for ways to reduce that cost, we can infer that the fault lies with the research community rather than the potential adopters. That is, something is going wrong with the research. Upon close inspection of individual papers, one finds that the most common culprit is bad baselines [...] > I would like to note that the publication of new methods which claim huge improvements but fail to replicate / live up to the hype is not a victimless crime, because it wastes the time, money, and morale of a large number of individual researchers and small labs who run and are disappointed by failed attempts to replicate and build on such methods every day. Sure, a brand new architecture is most likely not going to compare favorably to a state-of-art transformer. That is fine! But at least it will make the comparison actually useful. [1] -- https://github.com/KellerJordan/modded-nanogpt [2] -- https://kellerjordan.github.io/posts/muon/#discussion-solvin...
In The Dragon Hatchling: The missing link between the transformer and brain models
> nanochat is also inspired by modded-nanoGPT Nice synergy here, the lineage is: Karpathy's nano-GPT -> Keller Jordan's modded-nanoGPT (a speedrun of training nanoGPT) -> NanoChat modded-nanoGPT [1] is a great project, well worth checking out, it's all about massively speeding up the training of a small GPT model. Notably it uses the author's Muon optimizer [2], rather than AdamW, (for the linear layers). [1] https://github.com/KellerJordan/modded-nanogpt [2] https://kellerjordan.github.io/posts/muon/
In NanoChat – The best ChatGPT that $100 can buy

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.