HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
benhoyt
countwords
Rust2021–2024
benhoyt / countwords · Archived

countwords

Playing with counting word frequencies (and performance) in various languages.

10Distinct people
17Mentions
0Explicit recommenders
307GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Playing with counting word frequencies (and performance) in various languages.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 10:50:02 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year17 dated mentions

Observed mentions: 2021-03-15 10:08:21 UTC to 2024-06-01 19:40:22 UTC.

02468102021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 10 mentions; 0 submissions, 10 comments20212022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 6 mentions; 0 submissions, 6 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC10010
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC606
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC101

What Hacker News says

10 sources
I don't know. But Ben Hoyt got a similar time as me too. I saw your GitHub PR. Moreover, looking at your program, I would expect it to be slower than the other optimized versions (Rust, C, Go). Because it doesn't actually implement the optimizations in those programs! So not only is your code slow on two different compilers, but it also matches my prior. You're the one who came out of the woodwork and said a bunch of nonsense. Maybe put in a little effort here to try to figure things out. Like, oh, I don't know, run the benchmarks on the other programs to compare relative timings. Like I did. Or show the commands you used to compile. Like I did. Or show the commands you used to run the program. Like I did. You want me to hold your beer? Fine. But put in the work. My bet is that you're being just as sloppy now as you were in your comments.
Read the rest
In Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
The source code of the trie program I wrote is here: https://github.com/benhoyt/countwords/blob/master/rust/optim... The comments explain why I think it's slower. The TL;DR is that using a trie requires more memory accesses than a hash table (per byte of input), and those memory accesses slow the whole enterprise down. But, not all memory accesses are created equal. So perhaps a more clever representation that exploits locality better would do better. "Obvious" techniques for shrinking memory usage made a big difference and brought the performance of the trie program closer to C: https://github.com/benhoyt/countwords/pull/2
In Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
Yeah, I noticed that too: https://github.com/benhoyt/countwords/blob/9d81d13711e56c250... The constraints do say that it is okay to assume lines are reasonable length. But yes, if you were making GNU wordfreq, that might not be something you want to assume. But at that point, it just depends on what you want your failure mode to be I suppose. greps for example will happily just gobble up as much memory as they can to fit a line into memory. :-)
In Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
You've traded hashing for a memory access per byte. I did something similar in the 'optimized-trie' program for Rust: https://github.com/benhoyt/countwords/blob/master/rust/optim... --- It ended up being slower. But, it is very much impacted by cache effects. So the next step here would indeed be to figure out how to reduce cache misses. But it's non-trivial. Like, maybe your idea works. Maybe. I don't know. You'd have to try it. But it's not clear to me that it will. And it doesn't support "is far from what an experienced C programmer would write if performance was paramount" IMO.
In Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.