libdeflate

I think this may not be a very high bar. zippy in Nim claims to be about 1.5x to 2.0x faster than zlib: https://github.com/guzba/zippy I think there are also faster zlib's around in C than the standard install one, such as https://github.com/ebiggers/libdeflate (EDIT: also mentioned elsethread https://news.ycombinator.com/item?id=43381768 by mananaysiempre) zlib itself seems pretty antiquated/outdated these days, but it does remain popular, even as a basis for newer parallel-friendly formats such as https://www.htslib.org/doc/bgzip.html

Zlib-ng is between a couple and multiple times away from the state of the art[1], it’s just that nobody has yet done the (hard) work of adjusting libdeflate[2] to a richer API than “complete buffer in, complete buffer out”. [1] https://github.com/zlib-ng/zlib-ng/issues/1486 [2] https://github.com/ebiggers/libdeflate

Not actually true if gzip refers to libdeflate[1] or isa-l[2] instead of zlib. Does seem sus here, though,—I couldn’t quickly find which decompression code is used in zboot, but I’d expect it to be closer to zlib. Perhaps their implementation of Zstandard is simpler than the reference one? [1] https://github.com/ebiggers/libdeflate [2] https://github.com/intel/isa-l

> The QOI benchmarks shows it only lagging a small bit behind PNG in compression ratio, while performing much faster (multiples) in both encoding and decoding. Presumably that’s with stock libpng, which uses zlib. I wonder if anyone tried patching it to use the substantially faster (on x86) libdeflate[1] instead? It doesn’t do streaming, but you shouldn’t really need that for PNG textures. [1] https://github.com/ebiggers/libdeflate

Note that libdeflate has used essentially the same method since 2016 (https://github.com/ebiggers/libdeflate/blob/v0.4/lib/adler32...), though I recently switched it to use a slightly different method (https://github.com/ebiggers/libdeflate/blob/v1.12/lib/x86/ad...) that performs more consistently across different families of x86 CPUs.

Going away from Zlib proper but staying with its format, there’s libdeflate[1], an implementation that’s substantially faster on modern machines but can’t stream (although I see no reason why it couldn’t, probably just for simplicity due to its origins as a baseline fast compressor[2]), and of course Zopfli[3], a glacially slow compressor that’s somewhat (and occasionally a lot) better. [1] https://github.com/ebiggers/libdeflate [2] https://encode.su/threads/2482-libdeflate-a-new-optimized-DE... [3] https://github.com/google/zopfli