EKekidd·Sep 27, 2023
This is a great series!
I noticed that the author was using https://github.com/hydro-project/rust-sitter as a parser. Which is based on https://tree-sitter.github.io/tree-sitter/. I've been hearing about Tree-sitter a lot recently, so I dug into it.
Tree-sitter is a tool for generating fast, incremental parsers. In particular, the algorithm is suited towards writing "language servers" for IDEs, which re-parse code incrementally as the user works. These kinds of incremental parsers have historically been a huge problem. It looks like Tree-sitter is an enormous practical advance in this area.
And discovering that there's a way to use Tree-sitter from Rust is fantastic. From the post:
#[rust_sitter::language]
pub enum EvalExpr {
Number(
#[rust_sitter::leaf(
pattern = r"(\d+|0x[0-9a-fA-F]+)",
Read the rest
transform = parse_int
)]
u64
),
Symbol(
#[rust_sitter::leaf(
pattern = r"(([a-zA-Z0-9_@#.]+!)?[a-zA-Z0-9_@#.]+)",
transform = parse_sym
)]
String
),
// ...
Getting easy access to fast, incremental parsing is a huge win. And Tree-Sitter has support for being used from a huge list of languages, not just Rust.