base65536

For fun there are other variants of base64 in a similar spirit [1][2] full unicode. [3] see other links in that repo... Not a stream cipher, just encoding but could be used in conjunction with a stream cipher to add compression. It could go turtles all the way down. [1] - https://github.com/qntm/base2048 [2] - https://github.com/qntm/base32768 [3] - https://github.com/qntm/base65536

There's a family of encodings optimized for fitting the most information possible into an Unicode string of a given length, e.g. for gimmicks like fitting the most possible binary data into tweets. For example: https://github.com/qntm/base65536 For short messages in the mobile phone (i.e. GSM/3GPP) sense, which was my first association for "short message compression", it doubt that it works better than just sending binary messages with the appropriate header, but if that's not an option, it might just beat a custom alphabet based on the 7-bit GSM charset [1] (since that allows 100% of possible 7-bit characters to be used, whereas UTF-16 probably has at least some reserved codepoints that might be causing problems). [1] https://en.wikipedia.org/wiki/GSM_03.38

Base65536? https://github.com/qntm/base65536

base65536, and look who the author is :-D https://github.com/qntm/base65536

> For transmitting data through Twitter, Base65536 is now considered obsolete; see Base2048. Source: https://github.com/qntm/base65536

Sorry, I edited the post concurrently with your comment - it now points to Base2048, the link I meant to post, which actually should work - rather than https://github.com/qntm/base65536 (which I think you're commenting on).

There's always base65536 https://github.com/qntm/base65536

https://github.com/qntm/base65536#why > Why? > Erm. > I wanted people to be able to share HATETRIS replays via Twitter. My brain is melting.

Since they naively count unicode characters, https://github.com/qntm/base65536 is the best data encoding mechanism. You can fit 15 bits per character, or a total of 510 bytes per key and value.

One of my toy projects: https://github.com/thanatos/baseunicode ; there's probably a lot that could be done better, and the alphabet it uses needs to be greatly expanded. There are few other implementations of the same idea out there, too. Here's another one: https://github.com/qntm/base65536 ; since it uses a power-of-two sized alphabet, it avoids some of the work mine has to do around partial output bytes. That link also links to some other ideas & implementations The exact metric I cared about at the time was compressing for screen-space, not for actual byte-for-byte space. I wrote it after spending some time in a situation where doing a proper scp was just a PITA, unfortunately, but copy/pasting into a terminal is almost always a thing that works. But I was also dealing with screen/tmux, which makes scrolling difficult, so I wanted th

There is also Hatetris[1], with the same premise. Which I discovered because, for Hatetris, he wrote a nice twitter-oriented compact binary-to-text encoding, base65536[2]. [1] https://qntm.org/hatetris [2] https://github.com/qntm/base65536

Already exists: https://github.com/qntm/base65536 It's actually pretty useful for compressing data in Unicode-aware environments, like Twitter. Which makes me wonder if Unicode support is universal enough now that an encoding like this could replace MIME/base64 in email.

Here's a serious attempt at encoding arbitrary data into Unicode characters: https://github.com/qntm/base65536