lepton

Dropbox's Lepton [1][2], an additional lossless compression layer that operates on top of JPEG, would make for a smoother transition. If a browser natively supported Lepton, you could serve the compressed version over the wire, and still get the bit-exact original .jpg out at the end, if the user wanted to save the file. Of course, browser support doesn't exist yet. But browser implementors, consider the gains. This isn't promoting an incompatible format, it's getting bandwidth savings while being fully backwards compatible. [1] https://blogs.dropbox.com/tech/2016/07/lepton-image-compress... [2] https://github.com/dropbox/lepton

Portability notes for the Lepton implementation (https://github.com/dropbox/lepton): * Implemented in C++ (-std=c++0x and -std=c++11 work, -std=c++98 doesn't work). * Needs a recent g++ to compile (g++-4.8 works, g++-4.4 doesn't work). * Runs on Linux and Windows. * Runs on i386 (-m32) and amd64 (-m64) architectures. Doesn't work on other architectures, because it uses SSE4.1 instructions. * Can be compiled without autotools (http://ptspts.blogspot.ch/2016/07/how-to-compile-lepton-jpeg...).

I would be interested why C++ was chosen for https://github.com/dropbox/lepton instead of Rust? Given the recent usage of Rust for the implementation of Brotli compression (https://blogs.dropbox.com/tech/2016/06/lossless-compression-...) and that it's used for data storage (http://www.wired.com/2016/03/epic-story-dropboxs-exodus-amaz...) this somewhat surprises me. The reasons given for Rust as stated in https://blogs.dropbox.com/tech/2016/06/lossless-compression-... would seem valid here too: > For Dropbox, any decompressor must exhibit three properties: > > 1. it must be safe and secure, even against bytes crafted by modified or hostile clients, > 2. it must be deterministic—the same bytes must result in the same output, > 3. it must be fast.

Are we able to take the EXE and run it on JPEGs from a Windows machine without issue? (I haven't tried it myself, but probably will using the instructions I saw on Github a moment ago and report back...it looks like it should work though): https://github.com/dropbox/lepton/releases However I am wondering about the two different EXEs available on the page (one has an avx prefix that I need to try and figure out). If anyone has info on that that'd be useful. I could see this potentially being useful for schools and businesses doing quite a bit of digital scanning so I'm going to try running some tests using it and some images I think we have available somewhere.

Hi GrantS: That's pretty close to how it works. We always use the new prediction, even where it's worse than JPEG though (very rarely), to stay consistent. As for having the mega-model that predicts all images better: well it turns out with the lepton model out you only lose a few tenths of a percent by training the model from scratch on each images individually. We have a test case for training a global model in the archive (it's https://github.com/dropbox/lepton/blob/master/src/lepton/tes... ) That trains the "perfect" lepton model on the current image then uses that same model to compress the image (It's not meant to be a fair test, but it gives us a best-case scenario for potential gains from a model that has been trained from a lot of images) and in this case it doesn't gain much, even in a controlled situation like the test suite.