GEgeorgehill·Apr 11, 2024
Please change the title; it seems very misleading. It has nothing to do with LLMs.
https://github.com/bennyschmidt/next-token-prediction/blob/d...
In Show HN: Next-token prediction in JavaScriptbennyschmidt
next-token-prediction
JavaScript2024–2024
bennyschmidt / next-token-prediction
next-token-prediction
Next-token prediction in JavaScript - build fast language and diffusion models.
4Distinct people
11Mentions
0Explicit recommenders
145GitHub stars
What to know before exploring
What it is
Next-token prediction in JavaScript - build fast language and diffusion models.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
- bennyschmidt/llimo · linked in 2 of the same discussions
Mentioned in the same Hacker News threads. That is not proof they are substitutes.
This week on Hacker News
Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 4:00:59 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by week11 dated mentions
Observed mentions: 2024-04-10 21:27:07 UTC to 2024-09-03 04:41:06 UTC. Weeks start Monday, UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Apr 8 | 2024-04-08 00:00:00 UTC | 2024-04-15 00:00:00 UTC | 2 | 1 | 1 |
| Apr 15 | 2024-04-15 00:00:00 UTC | 2024-04-22 00:00:00 UTC | 1 | 0 | 1 |
| Apr 22 | 2024-04-22 00:00:00 UTC | 2024-04-29 00:00:00 UTC | 2 | 0 | 2 |
| Apr 29 | 2024-04-29 00:00:00 UTC | 2024-05-06 00:00:00 UTC | 1 | 0 | 1 |
| May 6 | 2024-05-06 00:00:00 UTC | 2024-05-13 00:00:00 UTC | 1 | 1 | 0 |
| May 13 | 2024-05-13 00:00:00 UTC | 2024-05-20 00:00:00 UTC | 0 | 0 | 0 |
| May 20 | 2024-05-20 00:00:00 UTC | 2024-05-27 00:00:00 UTC | 0 | 0 | 0 |
| May 27 | 2024-05-27 00:00:00 UTC | 2024-06-03 00:00:00 UTC | 0 | 0 | 0 |
| Jun 3 | 2024-06-03 00:00:00 UTC | 2024-06-10 00:00:00 UTC | 0 | 0 | 0 |
| Jun 10 | 2024-06-10 00:00:00 UTC | 2024-06-17 00:00:00 UTC | 0 | 0 | 0 |
| Jun 17 | 2024-06-17 00:00:00 UTC | 2024-06-24 00:00:00 UTC | 0 | 0 | 0 |
| Jun 24 | 2024-06-24 00:00:00 UTC | 2024-07-01 00:00:00 UTC | 0 | 0 | 0 |
| Jul 1 | 2024-07-01 00:00:00 UTC | 2024-07-08 00:00:00 UTC | 0 | 0 | 0 |
| Jul 8 | 2024-07-08 00:00:00 UTC | 2024-07-15 00:00:00 UTC | 0 | 0 | 0 |
| Jul 15 | 2024-07-15 00:00:00 UTC | 2024-07-22 00:00:00 UTC | 0 | 0 | 0 |
| Jul 22 | 2024-07-22 00:00:00 UTC | 2024-07-29 00:00:00 UTC | 0 | 0 | 0 |
| Jul 29 | 2024-07-29 00:00:00 UTC | 2024-08-05 00:00:00 UTC | 0 | 0 | 0 |
| Aug 5 | 2024-08-05 00:00:00 UTC | 2024-08-12 00:00:00 UTC | 0 | 0 | 0 |
| Aug 12 | 2024-08-12 00:00:00 UTC | 2024-08-19 00:00:00 UTC | 0 | 0 | 0 |
| Aug 19 | 2024-08-19 00:00:00 UTC | 2024-08-26 00:00:00 UTC | 0 | 0 | 0 |
| Aug 26 | 2024-08-26 00:00:00 UTC | 2024-09-02 00:00:00 UTC | 2 | 0 | 2 |
| Sep 2 | 2024-09-02 00:00:00 UTC | 2024-09-09 00:00:00 UTC | 2 | 0 | 2 |
What Hacker News says
2 sourcesShow HN: Next-token prediction in JavaScript
What inspired this project today was watching this amazing video by 3Blue1Brown called "But what is a GPT?" on Youtube (https://www.youtube.com/watch?v=wjZofJX0v4M - I highly recommend watching it). I added it to the repo for reference.
When it clicked in my head that "knowing a fact" is nearly synonymous with predicting a word (or series of words), I wanted to put it to the test, because it seemed so simple. I chose JavaScript because I can exploit the way it structures objects to aid in the modeling of language.
For example:
"I want to be at the beach",
"I will do it later",
"I want to know the answer",
...
becomes:
{
I: {
want: {
to: {
be: { ... },
know: { ... }
}
},
will: { ... }
},
...
}
in JavaScript. You can exploit the language's fast object lookup speed
Read the rest
to find known sentences this way, rather than recursively searching text - which is the convention and would take forever or not work at all considering there are several full books loaded in by default (and it could support many more).
Accompanying research yielded learnings about what "tokens" and "embeddings" are, what is meant by "training", and most of the rest - though I'm still learning jargon. I wrote a script to iterate over every single word of every single book to rank how likely it is that word will appear next, if given a cursor, and extended that to rank entire phrases. The base decoder started out what I'll call "token-agnostic" - didn't care if you were looking for the next letter... word... pixel... it's the same logic. But actually it's not, and it soon evolved into a text (language) model. But I have plans to get into image generation next (next-pixel prediction), using this. Overall the concepts are similar, but there are differences primarily around extraction and formatting.
Goals of the project:
- Demystify LLMs for people, show that it's just regular code that does normal stuff
- Actually make a pretty good LLM in JavaScript, with a version at least capable of running in a browser tab