HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
BurntSushi
rebar
Python2023–2026
BurntSushi / rebar

rebar

A biased barometer for gauging the relative speed of some regex engines on a curated set of tasks.

6Distinct people
44Mentions
1Explicit recommenders
327GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

A biased barometer for gauging the relative speed of some regex engines on a curated set of tasks.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

I love the flourish of "in the world." I had never thought about it that way. Which makes me wonder if there are any regex engines that aren't in rebar that could conceivably…

burntsushi used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 2:19:09 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year44 dated mentions

Observed mentions: 2023-07-05 13:48:49 UTC to 2026-05-30 15:54:58 UTC.

051015202023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 22 mentions; 2 submissions, 20 comments20232024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 14 mentions; 0 submissions, 14 comments20242025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 5 mentions; 0 submissions, 5 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 3 mentions; 1 submissions, 2 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC22220
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC14014
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC505
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC312

What Hacker News says

22 sources
burntsushi·Dec 31, 2023Recommendation phrase
I love the flourish of "in the world." I had never thought about it that way. Which makes me wonder if there are any regex engines that aren't in rebar that could conceivably be competitive with the top engines in rebar. I do maintain a WANTED list of engines[1], but none of them jump out to me except for maybe Nim's engine. Of course, there's also the question of whether the benchmarks are representative enough to make such extrapolations. I don't have a good answer for that one. All models are wrong, but, some are useful. [1]: https://github.com/BurntSushi/rebar/blob/96c6779b7e1cdd850b8...
In C# Regex engine is now 3rd fastest in the world
I don't disagree. It's why I brought this up. It's tricky to use Hyperscan, as-is, as a regex engine in a grep tool for these reasons. I don't mean to claim it is impossible, but there are non-trivial issues you'll need to solve. It's hard to learn too much from hypergrep. It still has some rough spots: $ hgrep -o 'foo.*bar' foobarbar.txt foobarbar.txt 1:[Omitted long line with 1 matches] $ hgrep -M0 -o 'foo.*bar' foobarbar.txt Too few arguments For more information try --help $ hgrep -M 0 -o 'foo.*bar' foobarbar.txt foobarbar.txt 1:[Omitted long line with 1 matches] $ hgrep -M 0 'foo.*bar' foobarbar.txt foobarbar.txt 1:[Omitted long line with 1 matches] $ hgrep -M0 'foo.*bar' foobarbar.txt terminate called after throwing an instance of 'std::invalid_argument' what(): patt
Read the rest
ern not found zsh: IOT instruction (core dumped) hgrep -M0 'foo.*bar' foobarbar.txt Another issue with Hyperscan is that if you enable HS_FLAG_UTF8[1], which hypergrep does[2,3], and then search invalid UTF-8, then the result is UB. > This flag instructs Hyperscan to treat the pattern as a sequence of UTF-8 characters. The results of scanning invalid UTF-8 sequences with a Hyperscan library that has been compiled with one or more patterns using this flag are undefined. That's another issue you'll need to grapple with if you use Hyperscan. PCRE2 used to have this issue[4], but they've since defined the semantics of searching invalid UTF-8 with Unicode mode enabled. ripgrep 14 uses that new mode, but I haven't updated that FAQ answer yet. Hyperscan isn't alone. Many regex engines do not support searching arbitrary byte sequences[5]. And this is why many/most regex engines are awkward to use in a fast grep implementation. Because you really do not want your grep to fall over when it comes across invalid UTF-8. And the overhead of doing UTF-8 checking in the first place (and perhaps let you just skip over lines that contain invalid UTF-8) would make it difficult to be competitive in performance. It also inhibits its usage in OSINT work. [1]: https://intel.github.io/hyperscan/dev-reference/api_files.ht... [2]: https://github.com/p-ranav/hypergrep/blob/ee85b713aa84e0050a... [3]: https://github.com/p-ranav/hypergrep/blob/ee85b713aa84e0050a... [4]: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#why... [5]: https://github.com/BurntSushi/rebar/blob/96c6779b7e1cdd850b8...
In Ugrep – a more powerful, fast, user-friendly, compatible grep
I'm the author of ripgrep and its regex engine. Your claim is true to a first approximation. But greps are line oriented, and that means there are optimizations that can be done that are hard to do in a general regex library. You can read more about that here: https://blog.burntsushi.net/ripgrep/#anatomy-of-a-grep (greps are more than simple CLI wrappers around a regex engine). If you read my commentary in the ripgrep discussion above, you'll note that it isn't just about the benchmarks themselves being accurate, but the model they represent. Nevertheless, I linked the hypergrep benchmarks not because of Hyperscan, but because they were done by someone who isn't the author of either ripgrep or ugrep. As for regex benchmarks, you'll want to check out rebar: https://github.com/BurntSushi/rebar You can see my full thoughts around benchmark d
Read the rest
esign and philosophy if you read the rebar documentation. Be warned though, you'll need some time. There is a fork of ripgrep with Hyperscan support: https://sr.ht/~pierrenn/ripgrep/ Hyperscan also has some preculiarities on how it reports matches. You won't notice it in basic usage, but it will appear when using something like the -o/--only-matching flag. For example, Hyperscan will report matches of a, b and c for the regex \w+, where as a normal grep will just report a match of abc. (And this makes sense given the design and motivation for Hyperscan.) Hypergrep goes to some pain to paper over this, but IIRC the logic is not fully correct. I'm on mobile, otherwise I would link to the reddit thread where I had a convo about this with the hypergrep author.
In Ugrep – a more powerful, fast, user-friendly, compatible grep
Did you know that not all of those use the same definition of what a "word" character is? Regex engines differ on the inclusion of things like \p{Join_Control}, \p{Mark} and \p{Connector_Puncuation}. Although in the case of \p{Connector_Punctuation}, regex engines will usually at least include underscore. See: https://github.com/BurntSushi/rebar/blob/f9a4f5c9efda069e798... And then there's \p{Letter}. It can be spelled in a lot of ways: \pL, \p{L}, \p{Letter}, \p{gc=Letter}, \p{gc:Letter}, \p{LeTtEr}. All equivalent. Very few regex engines support all of them. Several support \p{L} but not \pL. See: https://github.com/BurntSushi/rebar/blob/f9a4f5c9efda069e798...
In Ugrep – a more powerful, fast, user-friendly, compatible grep
Rust has had a stable SIMD vector API[1] for a long time. But, it's architecture specific. The portable API[2] isn't stable yet, but you probably can't use the portable API for some of the more exotic uses of SIMD anyway. Indeed, that's true in .NET's case too[3]. So .NET has to rely on "manually vectorized with intrisics for each individual platform" as well. Rust does all this SIMD too. It just isn't in the standard library. But the regex crate does it. Indeed, this is where .NET got its SIMD approach for multiple substring search from in the first place[4]. ;-) You're right that Rust's standard library is conservatively vectorized though[5]. The main thing blocking this isn't the lack of SIMD availability. It's more about how the standard library is internally structured, and the fact that things like substring search are not actually
Read the rest
defined in `std` directly, but rather, in `core`. There are plans to fix this[6]. With all that said, I wouldn't necessarily disagree with the broader point in this thread. .NET's regex engine performance is quite impressive[7]. (And I still have to merge the PR updating to .NET 8.) I'm really only replying here to correct more narrow points. [1]: https://doc.rust-lang.org/std/arch/index.html [2]: https://doc.rust-lang.org/std/simd/index.html [3]: https://github.com/dotnet/runtime/blob/72fae0073b35a404f03c3... [4]: https://github.com/dotnet/runtime/pull/88394#issuecomment-16... [5]: https://github.com/BurntSushi/memchr#why-is-the-standard-lib... [6]: https://github.com/rust-lang/rfcs/pull/3469 [7]: https://github.com/BurntSushi/rebar#summary
In Ask HN: What's the fastest programming language with a large standard library?
Note that I also imported regex-redux into rebar: https://github.com/BurntSushi/rebar/tree/master/record/all/2... It isn't exactly equivalent to the benchmark game's version. The biggest difference is that it doesn't permit parallelism. For more details on the benchmark model: https://github.com/BurntSushi/rebar/blob/master/MODELS.md#re... .NET does well overall in rebar's benchmarks, but less so in regex-redux (in both the benchmark game and in rebar's version of it). Not sure why. I haven't investigated. Of course, this is assuming you aren't using .NET's default interpreter engine. You've got to switch to the no-backtracking or compiled modes to get decent perf.
In .NET 8
BenchmarksGame is actually an interesting if unfortunate case for C#. Many tests there are at odds with the areas of optimizations that get most engineering effort investment in .NET: high level and/or application logic and high-performance primitives like Span, Vector, etc. (with the exception of perhaps Regex-Redux, but even there I'm not sure why the results are so low, because C# has one of the fastest[0] regex engines that only loses to Intel Hyperscan, Rust one and PCRE2). Probably the best way to go about this is to just give it a try with a sample project and see if the RAM usage is to your liking - you can get far with plain 'dotnet new console --aot' and standard library (once you're done, .NET's variant of go/cargo build is 'dotnet publish -o {output folder}'). Overall, .NET tends to have higher memory footprint than Go because
Read the rest
its GC is designed to sustain much higher allocation rate and heaps spiking to much higher use without severely throttling performance (which Go does). But there are interesting developments with the new GC mode called DATAS. I made a very simple example project some time ago to showcase .NET's concurrency and publishing experience to someone else which shows that it now rivals Go where it's the strongest[1]. [0] https://github.com/BurntSushi/rebar#summary-of-search-time-b... [1] https://github.com/neon-sunset/http-bench
In .NET 8
Before getting to your actual question, it might help to look at a regex benchmark that compares engines (perhaps JITs are not the fastest in all cases!): https://github.com/BurntSushi/rebar In particular, the `regex-lite` engine is strictly just the PikeVM without any frills. No prefilters or literal optimizations. No other engines. Just the PikeVM. As to your question, the PikeVM is, essentially, an NFA simulation. The PikeVM just refers to the layering of capture state on top of the NFA simulation. But you can peel back the capture state and you're still left with a slow NFA simulation. I mention this because you seem to compare the PikeVM with "big graph structures with NFAs/DFAs." But the PikeVM is using a big NFA graph structure. At a very high level, the time complexity of a Thompson NFA simulation and a DFA hints strongly at the a
Read the rest
nswer to your question: searching with a Thompson NFA has worst case O(m*n) time while a DFA has worst case O(n) time, where m is proportional to the size of the regex and n is proportional to the size of the haystack. That is, for each character of the haystack, the Thompson NFA is potentially doing up to `m` amount of work. And indeed, in practice, it really does need to do some work for each character. A Thompson NFA simulation needs to keep track of every state it is simultaneously in at any given point. And in order to compute the transition function, you need to compute it for every state you're in. The epsilon transitions that are added as part of the Thompson NFA construction (and are, crucially, what make building a Thompson NFA so fast) exacerbate this. So what happens is that you wind up chasing epsilon transitions over and over for each character. A DFA pre-computes these epsilon closures during powerset construction. Of course, that takes worst case O(2^m) time, which is why real DFAs aren't really used in general purpose engines. Instead, lazy DFAs are used. As for things like V8, they are backtrackers. They don't need to keep track of every state they're simultaneously in because they don't mind taking a very long time to complete some searches. But in practice, this can make them much faster for some inputs. Feel free to ask more questions. I'll stop here.
In Translations of Russ Cox's Thompson NFA C Program to Rust
A simple loop over lines using Python's `re` package is unlikely to be as fast as an optimized grep. If your inputs are small, the difference might be negligible. But if they're large, the difference might be enormous. And that's not even accounting for just regex engine throughput on its own. ripgrep for example uses rust/regex. Compare its performance with python/re: https://github.com/BurntSushi/rebar#summary-of-search-time-b...
In Shshsh is a bridge connects Python and shell
Yes, I use regexes in vim all the time. Never use look-around or really much regex features beyond the basics. 99% of all searches are easily just literals. I also use regexes with ripgrep of course. ripgrep has features that standard grep doesn't have that typically mitigate the need for look-around, such as the `-r/--replace` flag. (I can't remember I when I last used `-P/--pcre2` in ripgrep for a reason that wasn't testing it.) Otherwise the rest of my regex use is in programs for a variety of tasks. Note that I'm the author of Rust's regex crate and of this regex benchmark: https://github.com/BurntSushi/rebar
In Debugging catastrophic backtracking for regular expressions in Python
It is much faster for CPU-bound work. In my experience, 2x-5x is the most common range I get. There are many reasons besides just memory allocation and LLVM that Rust code ends up faster. Let me just focus on one very commonly used data structure: hash maps. Rust hash maps: * Use SwissTable because they didn't make a short-sighted promise about delete-while-iterating behavior that makes Go incompatible with SwissTable's tombstone design and makes Go never shrink maps once created. * Let you customize the hash function to whatever performs best for your key type & target hardware. * Let you find & insert/update in one hash table lookup, where Go always requires at least two, and its compiler doesn't optimize even the most common idioms. * Let you reference a key/value in-place instead of having to copy it out each time. Go won't allow this
Read the rest
because the reference may become stale, but Rust can enforce this with lifetimes. * Let you clone a map without rehashing every key to a new seed. I generally measure at least 15x speedup from this alone, unlocking very useful design patterns like "clone a map and apply a few temporary updates for a one-off operation like validation or simulation" with no extra code complexity. Go gives you no better option than slowly rehashing the entire map. And that's just hash maps. How about Go's regex engine being one of the slowest in the world while Rust's regex crate being one of the fastest: https://github.com/mariomka/regex-benchmark#optimized https://github.com/BurntSushi/rebar#summary-of-search-time-b... Further, Go refusing to have macros means that many libraries use reflection instead, which often makes those parts of the Go program perform no better than Python and in some cases worse. Rust can just generate all of that at compile time with macros, and optimize them with LLVM like any other code. Some Go libraries go to enormous lengths to reduce reflection overhead, but that's hard to justify for most things, and hard to maintain even once done. The legendary https://github.com/segmentio/encoding seems to be abandoned now and progress on Go JSON in general seems to have died with https://github.com/go-json-experiment/json . Remember that next time someone parrots the myth that Go is ideal for HTTP JSON APIs. Many people claiming their projects are IO-bound are just assuming that's the case because most of the time is spent in their input reader. If they actually measured they'd see it's not even saturating a 100Mbps link, let alone 1-100Gbps, so by definition it is not IO-bound. Even if they didn't need more throughput than that, they still could have put those cycles to better use or at worst saved energy. Isn't that what people like to say about Go vs Python, that Go saves energy? Sure, but it still burns a lot more energy than it would if it had macros. Maybe someone's recipe website doesn't care but world-scale datacenter applications do. Rust can use state-of-the-art memory allocators like mimalloc, while Go is still stuck on an old fork of tcmalloc, and not just tcmalloc in its original C, but transpiled to Go so it optimizes much less than LLVM would optimize it. (Many people benchmarking them forget to even try substitute allocators in Rust, so they're actually underestimating just how much faster Rust is) Oh yeah, aside, tcmalloc was designed for C++ code with lots of temporary allocations that are allocated & freed by the same thread so they're very cheap when cached per thread/core. Go almost always frees things on a separate GC thread, completely undermining this design goal. So tcmalloc isn't even a good fit for Go, yet there's no other choice available. People don't even discuss it any more because what is there to discuss, Go and all of its users are just stuck with it. Finally, even Go Generics have failed to improve performance, and in many cases can make it unimaginably worse through -- I kid you not -- global lock contention hidden behind innocent type assertion syntax: https://planetscale.com/blog/generics-can-make-your-go-code-... It's not even close. There are many reasons Go is a lot slower than Rust and many of them are likely to remain forever. Most of them have not seen meaningful progress in a decade or more, and I'm not holding my breath for the next decade either.
In Rust vs. Go in 2023

Showing the 15 most recent of 22 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.