zpaqlpy

ZPAQ is the name of the tool but ZPAQ is also the name of the container format that gets used. ZPAQ embeds the decompression algorithm in the archive. One could store zstd-compressed blocks in ZPAQ archives as soon as a zpaql decompressor exists (e.g., for brotli there is a slow one implemented in a python subset and compiled to zpaql https://github.com/pothos/zpaqlpy). I don't know exactly whether other formats are better for seeking and streaming, but since the baseline is tar, ZPAQ (in the 2.0 spec) is already better as it supports deduplication and files can even be updated append-only, and the compression is not an afterthought wrapped around it but well integrated.

If that is your taste, you should definitely take a look at zpaq [1] which uses an embedded bytecode with a not-too-long specification [2]. It is tuned for compression algorithms but any general algorithm can be programed; for example there is a project that turns zpaq into (slower) Brotli [3]. [1] http://mattmahoney.net/dc/zpaq.html [2] http://mattmahoney.net/dc/zpaq206.pdf [3] https://github.com/pothos/zpaqlpy

Yes, there are two examples for pi here: http://mattmahoney.net/dc/zpaqutil.html and one where the input does not exactly have to be pi, but can contain it: https://github.com/pothos/zpaqlpy/blob/master/test/mixedpi2.... But of course for real world use cases one would choose a general compression algorithm or write a specific one for a certain type of data you want to handle (because it will perform better than a general algorithm).