MirrorEncode

> If you want to play around with lossless compression, try out my "BYOM" (Bring Your Own Model) library for arithmetic encoding: https://github.com/comperical/MirrorEncode I believe this concept is widely known in high-end compressors. (Its weakness is also widely known: decompression cannot be made much faster than compression.)

I'm pretty sure all PNG is doing here is exploiting the fact that some characters are much more common than others. It's implicitly creating a unigram character model and using that to encode. There may also be a tiny bit of benefit related to the fact that in English if you see a letter, you're slightly more likely to see that letter again immediately after, as in words like "battle", "sonnet", "thee", etc. I'm surprised that PNG does better than ZIP; I'd be willing to be that ZIP does better if you scale up to the full Shakespeare corpus. ZIP will start to utilize word probabilities if you give it enough text (e.g. it will learn that "the" is more common than "eth"). > Image compression algorithms are, generally, pretty good at finding patterns in images and squashing them down. This really isn't true in practice. You could imagine an I

Why not use ML to build a statistical model of your data, and then use that model to compress it? You will likely get better codelengths for your specific dataset than off-the-shelf algorithms can achieve. And if you figure out a good model that describes the fluctuations in the cryptocurrency prices very well, then you can use that model pretty directly to make money via automated trading. I've got an easy-to-use library for arithmetic encoding in Java, it would be easy to port to other languages: https://github.com/comperical/MirrorEncode