regex

> One such engine is rust's https://github.com/rust-lang/regex It can process arbitrary deep nestings? I'm not familiar with it, but I had a look and I can't see anything in there that would allow it to do it. Can you point to syntax that allows it? In general regex's that run in linear time are based on DFA's (as opposed to NFA's - which is what most use). A DFA is part of the Chomsky hierarchy of languages that has DFA's at the bottom (aka regex's, least powerful - can't handle recursive structures), a DFA coupled with an infinite stack for memory (aka LR parsers, can't handle context sensitive grammars, also runs in very close to linear time), and finally a DFA coupled with an infinite tape (aka Turing Machine, can do any sort of computation). In each of these cases the DFA (regex) is essentially a computer program and all they are d

Yes, regex engines that are designed to do so run in linear time on the size of the regex (as well as linear time with respect to the length of the input). One such engine is rust's https://github.com/rust-lang/regex

I agree, this could be a nice project! If anyone wants to work on it, this is the place to start: https://github.com/rust-lang/regex/blob/master/src/literal/t... I'll get to it myself eventually if someone else doesn't, but it will likely be a while.

That work was all in the context of searching for a binary string with a mask. I didn't try too long to optimize it, since performance was quickly satisfactory, so it's not just possible, but rather very likely that my implementation isn't particularly good. I'll keep your advice about SSE 4.2 in mind should I revisit that code. https://github.com/rust-lang/regex/blob/master/src/literal/t... might be of interest to casual readers (not necessarily you, considering you were probably very involved in developing that algorithm ;)

Yes, ripgrep (or specifically, the Rust regex crate) takes inspiration from Hyperscan for its SIMD matching. https://github.com/rust-lang/regex/commit/203c509df9e1dcba61... Both libraries also rely on a number of techniques from existing published research. Hyperscan is ruthlessly optimized for x86 platforms, but has dropped support for non-x86 platforms. Rust's regex crate is designed more as a general purpose, portable regular expression implementation, suitable for wide scale use.

Rust's regex library also doesn't have this issue [1], and is easily usable by other languages [2]. If you're concerned about this issue in your code and your not using go it's probably easier to use this than the go one. [1] https://docs.rs/regex/1.1.0/regex/ [2] - C (and thus everything with a FFI): https://github.com/rust-lang/regex/tree/master/regex-capi - Go (heh): https://github.com/BurntSushi/rure-go - Possibly somewhat out of date Python: https://github.com/davidblewett/rure-python