ryg_rans

One way to beat this in terms of coding efficiency would be to use something like rANS (explanation [1], code [2]). You would have a more complicated decoder with rANS though, even if you removed the frequency table lookup (because always 1/3) and you'd only produce one output at a time instead of 5 doing this with a lookup table. It might be possible to use a combination of both techniques with e.g. a 16 bit accumulator, multiplying by 243 to get the value to lookup into bits 23-16 and then bias and shift the incoming 8 bits into the accumulator. [1] https://fgiesen.wordpress.com/2014/02/18/rans-with-static-pr... [2] https://github.com/rygorous/ryg_rans

You don't seem to have a public repository for the code described in your posts (presumably a part of Chessbook?), but you might be using a textbook arithmetic coding algorithm which maintains both lower and upper bounds and does a renormalization by comparing topmost bits. If it's the case rANS would be much simpler to write and more efficient [1]. If your AC code is already well-optimized, there is not much reason to switch though. [1] Fabian Giesen's sample implementation is already as good as is, and also contains a good alias table implementation if you want multi-symbol inputs: https://github.com/rygorous/ryg_rans/blob/master/rans_byte.h

Yeah read about Kraken a while ago and found the encode.ru thread you pointed. Its unfortunate that we can't compare that side by side. Those guys seem another of those compression geniuses. Overall the world of compression is moving very fast and well these days. But even with that Yann might be the first. Yan's work on FSE is old [0] and if I'm reading this correctly his work on FSE is a mix of his own work and Jarek Duda. But it seems one of his failed attempt challenged Jarek to invent another version of ANS called rANS [1] and as you can see in the encode.ru comments from the RAD game tools that they seem to be using rANS at least for Kraken. Regardless these are impressive works and these people are bouncing ideas of each others, being challenged and inspired which is a very good thing for the rest of us :). [0] https://fastcompres