zeekstd

If seeking to frames is good enough for random seeking, this can be done with zeekstd already. The cli sets a custom window log (ZSTD_c_windowLog) on the compression context when creating binary patches[1], I regularly use it with a window size above 1G. [1] https://github.com/rorosen/zeekstd/blob/main/cli/src/compres...

For whatever it’s worth, zeekstd seems to come with a CLI tool: https://github.com/rorosen/zeekstd/tree/main/cli

Hello, I would like to share a Rust implementation of the Zstandard seekable format I've been working on. Regular zstd compressed files consist of a single frame, meaning you have to start decompression at the beginning. The seekable format splits compressed data into a series of independent frames, each compressed individually, so that decompression of a section in the middle of an archive only requires zstd to decompress at most a frame's worth of extra data, instead of the entire archive. I started working with the seekable format because I wanted to resume downloads of big zstd compressed files that are decompressed and written to disk on the fly. At first I created and used bindings to the C functions that are available upstream[1], however, I stumbled over the first segfault rather quickly (it's now fixed) and found out that the fun