hyperscan

I think of WAFs as an extra safety net. Defense in depth. The author complained about the performance cost of WAFs in general, but not all WAFs have be structured like ModSecurity. They could for example be based on something like https://github.com/intel/hyperscan and perf is at a very different level.

Another example of old timey Unix code just breaking things in minor point releases. See https://abi-laboratory.pro/index.php?view=changelog&l=glibc&... and https://github.com/intel/hyperscan/issues/359.

It’s a bespoke scanning setup designed to deal with GitHub’s scale, minimise false positives, and scan fast enough to be in the `git push` request/response cycle. Under the hood it’s using Intel’s hyperscan as the regex engine. https://github.com/intel/hyperscan

The main difference is that binwalk goes through a file linearly, searching for patterns like magic bytes, and tries to extract everything it finds. The problems with this: - very noisy, finding a lot of false positives (license code, format inside another format, etc). - very slow, trying to extract irrelevant things - imprecise, because it finds patterns in the middle of a file, where it's actually not relevant on the first level of extraction unblob solves these problems by being smarter about the file formats, recognizing them by their specification, for example unpacks format header structs and carves out files based the information in the header (size, offset). See a simple example for NTFS [1]. We also went to great lengths preventing unnecessary work by skipping formats inside another [2]. We are using hyperscan [3] instead of gr