napkin-math

When I began my career in programming I remember reading Jeff Dean's "Latency Numbers Every Programmer Should Know" [1], and getting inspired. Alas, it's been over a decade, and I never fully internalized all the numbers. Until recently. I discovered Simon Eskildsen's talk [2] and github repo [3] on programmer napkin math. I liked his reasoning, and decided to _actually_ internalize the numbers now. So I made baserates.dev. It's a spaced repetition program that teaches you the napkin math from Simon, enhanced with a few things from Jeff Dean's list. The main changes: I included CPU instructions, like L1 Cache reads. I also included some nooby context, like how to pronounce the character μ, and a reminder on what is bigger: nano seconds, or micro seconds. The app automatically saves your progress, and doesn't require auth. Most of the log

Yep! Other than random reads (~p99=200ms on larger ranges), it's essential to get good download performance of a single file. A single (range) request can "only" drive ~500 MB/s, so you need multiple offsets. https://github.com/sirupsen/napkin-math

Not everything you think you know is right. https://github.com/sirupsen/napkin-math

And napkin-math: https://github.com/sirupsen/napkin-math

Most production storage systems/databases built on top of S3 spend a significant amount of effort building an SSD/memory caching tier to make them performant enough for production (e.g. on top of RocksDB). But it's not easy to keep it in sync with blob... Even with the cache, the cold query latency lower-bound to S3 is subject to ~50ms roundtrips [0]. To build a performant system, you have to tightly control roundtrips. S3 Express changes that equation dramatically, as S3 Express approaches HDD random read speeds (single-digit ms), so we can build production systems that don't need an SSD cache—just the zero-copy, deserialized in-memory cache. Many systems will probably continue to have an SSD cache (~100 us random reads), but now MVPs can be built without it, and cold query latency goes down dramatically. That's a big deal We're currentl

Comparison on Clickhouse, also runs in about 30-40ms, however there's no indexing being used and this is a full-table scan. create table if not exists test_table ( id UInt64, text1 String, text2 String, int1000 UInt64, int100 UInt64, int10 UInt64, int10_2 UInt64 ) engine = MergeTree() order by (id) ; insert into test_table with repeat('b', 1024) as one_kib, repeat('b', 255) as bytes_255 select number as id, one_kib, bytes_255, rand() % 1000 as int1000, rand() % 100 as int100, rand() % 10 as int10, rand() % 10 as int10_2 from numbers(10e6) ; > select count(*) from test_table where int1000 = 1 and int100 = 1; ┌─count()─┐ │ 9949 │ └─────────┘

`madvise(2)` doesn't matter _that_ much in my experience with [1] on modern Linux kernels. SSD just can't read _quite_ as quickly as memory in my testing. Sure, SSD will be able to re-read a lot into ram, analogous to how memory reading will be able to rapidly prefetch into L1. I get ~30 GiB/s for threaded sequential memory reads, but ~4 GiB/s for SSD. However, I think the SSD number is single-threaded and not even with io_uring—so I need to regenerate those numbers. It's possible it could be 2-4x better. [1]: https://github.com/sirupsen/napkin-math

Yes, sequential I/O bandwidth is closing the gap to memory. [1] The I/O pattern to watch out for, and the biggest reason why e.g. databases do careful caching to memory, is that _random_ I/O is still dreadfully slow. I/O bandwidth is brilliant, but latency is still disappointing compared to memory. Not to mention, in typical Cloud workloads, IOPS are far more expensive than memory. [1]: https://github.com/sirupsen/napkin-math