leveldb

Porting LevelDB[1] to Seastar[2], for internal metadata storage in Redpanda[3]. Before you ask why can’t something off the shelf be used, seastar has unique constraints around its runtime and its memory allocator that means we can’t reuse an existing library. 1: https://github.com/google/leveldb 2: https://github.com/scylladb/seastar 3: https://github.com/redpanda-data/redpanda/pull/28351

A product written by some notorious rock stars at Google, and with a reputation for corrupting data... [1] - "...LevelDB has history of database corruption bugs.[15][16][17][18][19][20] A study from 2014 has found that, on older (non-checksummed) file systems, the database could become corrupted after a crash or power failure.[21]..." https://github.com/google/leveldb/issues?utf8=%E2%9C%93&q=co... https://bugs.chromium.org/p/chromium/issues/detail?id=261623 https://forum.syncthing.net/t/panic-leveldb-table-corruption... https://github.com/google/leveldb/issues/333 https://ethereum.stackexchange.com/questions/1159/corruption... [1] - https://en.wikipedia.org/wiki/LevelDB

Google didnt, thus Chrome started replacing sqlite with https://github.com/google/leveldb

LevelDB by Jeff and Sanjay. https://github.com/google/leveldb

leveldb https://github.com/google/leveldb

I don't know of any rule of English grammar that would lead to this interpretation. If you do, you should immediately write to the maintainers of these websites: https://redis.com/nosql/key-value-databases/ https://www.mongodb.com/databases/key-value-database https://aws.amazon.com/nosql/key-value/ https://etcd.io/docs/v3.4/learning/why/ https://riak.com/products/riak-kv/ https://rocksdb.org/ https://github.com/google/leveldb http://www.memcached.org/

LevelDB: https://github.com/google/leveldb

+1000 https://github.com/google/leveldb Jeff Dean and Sanjay Ghemawat are amazing engineers and this code is (/was?) nice.

You seem to be describing leveldb: https://github.com/google/leveldb

You can check out redis's implementation: https://github.com/redis/redis/blob/unstable/src/t_zset.c Or leveldb's: https://github.com/google/leveldb/blob/master/db/skiplist.h

This is the "secret sauce" behind LevelDB: https://github.com/google/leveldb#performance

If you just do an occasional key/value lookup, you don't need 1.2 MiB of WebAssembly. [1] That might already exceed your total database size. I'd solve it via sharding: divide the database into N pieces via range- or hash-sharding. [1] Choose an N that's large enough for each piece to be reasonably small. When you look up a key, fetch the shard of interest. You can put each piece into separate files (a little simpler to code, and most static servers will use pre-gzipped files for "Content-Encoding: gzip requests" easily, but you waste more disk space due to internal fragmentation) or one file (with range serving and an index of the byte range offset for each piece). The format for each piece can be anything, eg json (simple) or an sstable-like format (more efficient). [3] [1] Content-Length of https://phiresky.github.io/youtube-sponsorshi

To avoid some confusion, this is only the "distributed" part. The actual store is handled by the https://github.com/google/leveldb library.