DAdaniel_rh·Aug 1, 2017
I'm the author of https://github.com/dropbox/rust-brotli and I certainly ran into the issues mentioned in the article, and the initial version of my brotli decoder and encoder were each almost 10x slower. I also worked around each and every one of them, and the result is something that performs at 80-90% of the speed of the original on average, and some files compress faster with rust than with the original brotli codec.
allocator: I ran into the same situation and abstracted it with a generic allocator: https://github.com/dropbox/rust-alloc-no-stdlib it's an ugly solution, but it works and can allow for significant perf improvements down the line by bundling all same types together
benchmarks: I simply made a test that printed the time
primitive types: it's easy to write generic functions that do this
macro system: I found to be amazing
Read the rest
, but I never used it to compact data types
borrow checker: split_at_mut was super helpful...also core::cmp::replace was useful to taking away pieces of a structure and manipulating them, then putting them back
Though it wasn't all roses: to try to get SSE vectorization I had to convert this 6 line short string matching function
https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d...
into this multi hundred line monster with macros and so forth
https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d...
but in the end it was as fast as hand-tuned intrinsics in C
I also found myself scared by dependencies, including onto the std library, so I tried to get everything to remain within the core library. This should allow something as low-level as a codec to be used in kernel space or in another place where a custom allocator is needed.
I also found that "rewriting in safe rust from C" was made easy by corrode https://github.com/jameysharp/corrode
since C and rust can be interface compatible, it's easy to go one file at a time and turn it into working rust, then safe rust.