MAMaulingMonkey·May 21, 2025
> Is this so inefficient?
It can be. If you have large buffers (tuned for throughput) that end up fulfilling lots of small requests for whatever reason, for example. And there's always the occasional article when someone rediscovers that replacing malloc + memset with calloc can have massive performance savings thanks to zeroing by the OS only occuring on first page fault (if it ever occurs), instead of an O(N) operation on the whole buffer up front.
Which, if in the wrong loop, can quickly balloon from O(N) to O(scary).
https://github.com/PSeitz/lz4_flex/issues/147
https://github.com/rust-lang/rust/issues/117545
If I'm reading that log-log plot right, that looks like a significantly worse than 100x slowdown on 1GB data sets. Avoiding init isn't the only solution, of course, but it was a solution.
> then it seems preferable to re-use b
Read the rest
uffers
Buffer reuse may be an option, but in code with complicated buffer ownership (e.g. transfering between threads, with the thread of origination not necessarily sticking around, etc.), one of the sanest methods of re-use may be to return said buffer to the allocator, or even OS.
> and pay the initialization once at startup.
Possibly a great option for long lived processes, possibly a terrible one for something you spawn via xargs.