twemcache

Some software that Twitter has put out: [1] Heron, a realtime, distributed, fault-tolerant stream processing engine - https://github.com/twitter/heron [2] Finagle, a fault tolerant, protocol-agnostic RPC system - https://github.com/twitter/finagle/ [3] FlockDB, a distributed, fault-tolerant graph database - https://github.com/twitter/flockdb [4] Gizzard, a flexible sharding framework for creating eventually-consistent distributed datastores - https://github.com/twitter/gizzard [5] Twemcache, a Twitter fork of Memcached - https://github.com/twitter/twemcache [6] Twemproxy, a fast, light-weight proxy for memcached and redis - https://github.com/twitter/twemproxy

Twitter puts out a LOT of software, and open-sources a good portion. I don't know if this is smart in the sense of being overstaffed relative to their revenue, but they do it regardless. Some examples: [1] Fabric, an SDK for mobile apps - https://docs.fabric.io/android/fabric/overview.html [2] Heron, a realtime, distributed, fault-tolerant stream processing engine - https://github.com/twitter/heron [3] Finagle, a fault tolerant, protocol-agnostic RPC system - https://github.com/twitter/finagle/ [4] FlockDB, a distributed, fault-tolerant graph database - https://github.com/twitter/flockdb [5] Ruby implementation of the ICU (International Components for Unicode - https://github.com/twitter/twitter-cldr-rb [6] Clockwork Raven, Human-Powered Data Analysis with Mechanical Turk - https://github.com/twitter/clockworkraven [7] Gizzard, a flexible

This is not an issue for fatcache. slab item chunk size introduces internal fragmentation (because items sizes usually don't match "chunk" sizes unless you use slab profile, -z option, to match them up). The kind of external fragmentation you described is due to the eviction strategy memcached uses, which can be avoided by using slab level eviction strategies (Twemcache and latest Memcached both have this support). fatcache does slab level eviction based on write timestamps, which is the equivalent of LRC eviction in Twemcache, and you can read about the mechanism here: https://github.com/twitter/twemcache/wiki/Eviction-Strategie...

The Memcached maintainer's reaction to this sudden, large code drop of a fork and the way it was communicated to the community: https://github.com/twitter/twemcache/issues/2