tcmalloc

yeah that’s a common mistake when evaluating tcmalloc. gperftools tcmalloc diverged quite a while ago. doesn’t have a lot of the fancier features of modern tcmalloc [0]. [0]: https://github.com/google/tcmalloc/blob/master/docs/gperftoo...

no article, sorry, grabbed the numbers from an old PR. to confirm, you were using tcmalloc from https://github.com/google/tcmalloc and not from https://github.com/gperftools/gperftools, right? the latter is a lot worse iiuc.

modern tcmalloc uses per CPU caches via rseq [0]. We use async rust with multithreaded tokio executors (sometimes multiple in the same application). so relatively high thread counts. [0]: https://github.com/google/tcmalloc/blob/master/docs/design.m...

We evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily written in Rust and the allocators were linked in statically (except for glibc). Unfortunately I didn't capture much context on the allocation patterns. I think in general the apps allocate and deallocate at a higher rate than most Rust apps (or more than I'd like at least). Our results from July 2025: rows are <allocator>: <RSS>, <time spent for allocator operations> app1: glibc: 215,580 KB, 133 ms mimalloc 2.1.7: 144,092 KB, 91 ms mimalloc 2.2.4: 173,240 KB, 280 ms tcmalloc: 138,496 KB, 96 ms jemalloc: 147,408 KB, 92 ms app2, bench1 glibc: 1,165,000 KB, 1.4 s mimalloc 2.1.7: 1,072,000 KB, 5.1 s mimalloc 2.2.4: tcmalloc: 1,023,000 KB, 530 ms app2, bench2 gli

TCMalloc has an absurd amount of bookkeeping and stats, but you have to understand the implementation deeply to make sense of the stats. https://github.com/google/tcmalloc/blob/master/docs/stats.md

The reason I hedged and said "... or some automatic system ..." was because they use a machine-learned forecast of the memory requirements of every container and use that as the soft limit for the container when it starts. You can read about that at [1]. But what I was getting at is that using less than the configured amount of memory does not lead to more containers able to be scheduled on a given machine, nor does it lead to lower economic chargeback. Machines are scheduled and operators are charged by the configured limit, not the usage. Giving memory back to the operating system is antithetical to the nature of caching allocators ("caching" is right there in the name of "tcmalloc"). The whole point of a caching allocator is that if you needed the memory once, you'll probably need it again, and most likely right now. At most what these

Thanks for sharing the insight! As I observed when I was at Google: tcmalloc wasn't a dedicated team but a project driven by server performance optimization engineers aiming to improve performance of important internal servers. Extracting it to github.com/google/tcmalloc was complex due to intricate dependencies (https://abseil.io/blog/20200212-tcmalloc ). As internal performance priorities demanded more focus, less time was available for maintaining the CMake build system. Maintaining the repo could at best be described as a community contribution activity. > Meta’s needs stopped aligning well with those of external uses some time ago, and they are better off doing their own thing. I think Google's diverged from the external uses even long ago:) (For a long time google3 and gperftools's tcmalloc implementations were so different.)

The post was written two days ago, but the allocator versions are between three and six years old. Nowadays, there are at least two different versions of tcmalloc: the one from https://github.com/google/tcmalloc/ is newer (it has rseq support, for example), and the version bundled in https://github.com/gperftools/gperftools.

> * Switching from libc malloc to tcmalloc (dating myself a little bit) If you think of tcmalloc as an old crusty allocator, you've probably only seen the gperftools version of it. This is the version Google now uses internally: https://github.com/google/tcmalloc It's worth a fresh look. In particular, it supports per-CPU caches as an alternative to per-thread caches. Those are fantastic if you have a lot more threads than CPUs. I haven't checked if it's been adapted for the latest upstream kernel API, but there's also the idea of "vcpu"-based caches: basically rather than a physical cpu id, it's an (optionally per-numa-node-based) dense id assigned to active threads, so that it still works well if you have a small cpu allocation for this process on a many-core machine.

The default setting of max_ptes_none is also problematic. On a stock kernel, it's 511. TCMalloc's docs recommend using max_ptes_none set to 0 for this reason: https://github.com/google/tcmalloc/blob/master/docs/tuning.m... (Disclosure: I work on TCMalloc and authored the above doc.)

The easiest way to exploit THP, by far, is to link your program against TCMalloc and forget about it. Literally free money. Highly recommended. https://github.com/google/tcmalloc

You are talking about the old tcmalloc from 2006 that should never be used anymore. There is also a new tcmaloc: https://github.com/google/tcmalloc/ If you are in doubt, you should simply use what ClickHouse is using.

This is factually false, and I'm wondering why you'd say that. Some of the more common malloc implementations out there are dlmalloc, jemalloc, and tcmalloc. Here is where they call munmap. https://github.com/ennorehling/dlmalloc/blob/71296436f979a35... https://github.com/jemalloc/jemalloc/blob/c0c9783ec9289e6d1d... Note that not all allocators work this way, you can see that tcmalloc will instead MADV_DONTNEED. (You can find munmap in tcmalloc, but it's not being used to free memory.) https://github.com/google/tcmalloc/blob/48808681a9f38e974b49...

Not true. mmap is commonly used in malloc implementations. Look at this man page for jemalloc. http://jemalloc.net/jemalloc.3.html "Traditionally, allocators have used sbrk(2) to obtain memory, which is suboptimal for several reasons, including race conditions, increased fragmentation, and artificial limitations on maximum usable memory. If sbrk(2) is supported by the operating system, this allocator uses both mmap(2) and sbrk(2), in that order of preference; otherwise only mmap(2) is used." Also, Google's tcmalloc uses mmap to get system memory: https://github.com/google/tcmalloc/blob/master/docs/design.m... "An allocation for k pages is satisfied by looking in the kth free list. If that free list is empty, we look in the next free list, and so forth. Eventually, we look in the last free list if necessary. If that fails, we fetch memory

cache alignment also gets stupid complicated when you add in what kind of malloc you use, c.f. google's tcmalloc: https://github.com/google/tcmalloc