BUburntsushi·Mar 24, 2017
If I understand your idea right, then this is actually being done! https://github.com/google/fancy-regex
In The 1960's elegance behind Go's regexpgoogle
fancy-regex
Rust2017–2017
google / fancy-regex · Archived
fancy-regex
Discover this repository through its Hacker News conversations.
2Distinct people
4Mentions
0Explicit recommenders
165GitHub stars
What to know before exploring
What it is
A public description has not been verified.
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by week4 dated mentions
Observed mentions: 2017-02-08 20:22:51 UTC to 2017-03-24 23:21:37 UTC. Weeks start Monday, UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Feb 6 | 2017-02-06 00:00:00 UTC | 2017-02-13 00:00:00 UTC | 3 | 0 | 3 |
| Feb 13 | 2017-02-13 00:00:00 UTC | 2017-02-20 00:00:00 UTC | 0 | 0 | 0 |
| Feb 20 | 2017-02-20 00:00:00 UTC | 2017-02-27 00:00:00 UTC | 0 | 0 | 0 |
| Feb 27 | 2017-02-27 00:00:00 UTC | 2017-03-06 00:00:00 UTC | 0 | 0 | 0 |
| Mar 6 | 2017-03-06 00:00:00 UTC | 2017-03-13 00:00:00 UTC | 0 | 0 | 0 |
| Mar 13 | 2017-03-13 00:00:00 UTC | 2017-03-20 00:00:00 UTC | 0 | 0 | 0 |
| Mar 20 | 2017-03-20 00:00:00 UTC | 2017-03-27 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
4 sourcesSo I ran the sqlite3.c benchmark on my machine (a comparable "somewhat powerful" machine) and it took 6.7s with my engine vs 10.9s with the new VSCode one. Both are doing tokenization+theme matching but the machines and exact grammars are not necessarily the same. I'm using ST3's C grammar but they are using a different one. It could be that I ran it on a faster computer with an easier grammar, or it could be the opposite. It's close enough I'm not willing to claim my engine is substantially faster.
For context Sublime Text 3 takes 2 seconds with the same grammar and same file on the same computer due to using a better custom regex engine written specifically for highlighting.
Given what alexdima mentioned in a different comment about spending most of the time in the regex engine, I'm not sure that my engine would be substantially faster
Read the rest
under exactly identical conditions since I'm also bottlenecked by Oniguruma.
However, maybe after I port my engine to https://github.com/google/fancy-regex I'll be substantially faster. And if I do it is likely they could also benefit from a fancy-regex port.
It's harder than it sounds if you want to support many languages. The Sublime syntaxes repo I use has 34,000 lines of grammars whereas my engine is only 3000 lines of code. If you count all the tmLanguage files for nice languages available online it's probably hundreds of thousands of lines, and that's in a pretty dense format. The whole point of using tmLanguage files is that people don't care about how fast other languages are if there is no highlighting for their language.
I could get way better performance by rewriting all those grammars using compiled parsers in Rust (like Xi has as an option https://github.com/google/xi-editor/blob/master/rust/lang/Ca...) but it would take an absurd amount of effort.
The speed of highlighting tmLanguage files is limited mostly by the regex library. My code spends 50% of its time in Oniguruma. You ca
Read the rest
n improve that a bit by using a fancy regex engine, which is how Sublime is faster than my engine, but this evening I'm going to try to port my library to a faster engine based on Rust's regex crate. But that library (https://github.com/google/fancy-regex) is brand new and almost untested. It's the only open source library that is faster than Oniguruma while supporting all the right features. In the future VSCode may be able to gain some speed by switching to it, but they have much more overhead over the regex library to eat away first than I do.
Yes it would be possible to write a C wrapper for my Rust library and link that from Node, however I expect you might lose a lot of performance to data conversion.
I have yet to do the regex engine port, I'm doing that later this evening. I'm porting it from Oniguruma to https://github.com/google/fancy-regex which accelerates common types of regexes using the awesome Rust regex crate which is super fast.
In Optimizations in Syntax Highlighting