thomasjungblut

go-sstables

Go2022–2025

thomasjungblut / go-sstables

go-sstables

Go library for protobuf compatible sstables, a skiplist, a recordio format and other database building blocks like a write-ahead log. Ships now with an embedded key-value store.

Explore on GitHub Read Hacker News discussions

3Distinct people

6Mentions

1Explicit recommenders

369GitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Go library for protobuf compatible sstables, a skiplist, a recordio format and other database building blocks like a write-ahead log. Ships now with an embedded key-value store.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

I highly recommend people comfortable with Go checkout the building blocks at https://github.com/thomasjungblut/go-sstables This codebase shows how SSTables, WAL, memtables,…

Xeoncross used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

blevesearch/bleve · linked in 2 of the same discussions
meilisearch/meilisearch · linked in 2 of the same discussions
mosuka/phalanx · linked in 2 of the same discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year6 dated mentions

Observed mentions: 2022-02-10 01:26:15 UTC to 2025-10-09 20:56:03 UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Comments
2022	2022-01-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	4	4
2023	2023-01-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	0	0
2024	2024-01-01 00:00:00 UTC	2025-01-01 00:00:00 UTC	0	0
2025	2025-01-01 00:00:00 UTC	2026-01-01 00:00:00 UTC	2	2

What Hacker News says

6 sources

Xeoncross·Oct 9, 2025

I really liked the README, that was a good use of AI. If you're interested in the idea of writing a database, I recommend you checkout https://github.com/thomasjungblut/go-sstables which includes sstables, a skiplist, a recordio format and other database building blocks like a write-ahead log. Also https://github.com/BurntSushi/fst which has a great Blog post explaining it's compression (and been ported to Go) which is really helpful for autocomplete/typeahead when recommending searches to users or doing spelling correction for search inputs.

In Show HN: I wrote a full text search engine in Go

twalla·Feb 26, 2025

Someone asked about resources and I ran into this while evaluating embedded db options for a golang project - it's a collection of db components implemented in golang: https://github.com/thomasjungblut/go-sstables

In Show HN: A Database Written in Golang

Xeoncross·Aug 25, 2022

I've never worked on a project that encompasses as many computer science algorithms as a search engine. There are a lot of topics you can lookup in "Information Storage and Retrieval": - Tries (patricia, radix, etc...) - Trees (b-trees, b+trees, merkle trees, log-structured merge-tree, etc..) - Consensus (raft, paxos, etc..) - Block storage (disk block size optimizations, mmap files, delta storage, etc..) - Probabilistic filters (hyperloloog, bloom filters, etc...) - Binary Search (sstables, sorted inverted indexes, roaring bitmaps) - Ranking (pagerank, tf/idf, bm25, etc...) - NLP (stemming, POS tagging, subject identification, sentiment analysis etc...) - HTML (document parsing/lexing) - Images (exif extraction, removal, resizing / proxying, etc...) - Queues (SQS, NATS, Apollo, etc...) - Clustering (k-means, density, hierarchical, gaussi

Read the rest

an distributions, etc...) - Rate limiting (leaky bucket, windowed, etc...) - Compression - Applied linear algebra - Text processing (unicode-normalization, slugify, sanitation, lossless and lossy hashing like metaphone and document fingerprinting) - etc... I'm sure there is plenty more I've missed. There are lots of generic structures involved like hashes, linked-lists, skip-lists, heaps and priority queues and this is just to get 2000's level basic tech. If you are comfortable with Go or Rust you should look at the latest projects in this space: - https://github.com/quickwit-oss/tantivy - https://github.com/valeriansaliou/sonic - https://github.com/mosuka/phalanx - https://github.com/meilisearch/MeiliSearch - https://github.com/blevesearch/bleve - https://github.com/thomasjungblut/go-sstables A lot of people new to this space mistakenly think you can just throw elastic search or postgres fulltext search in front of terabytes of records and have something decent. The problem is that search with good rankings often requires custom storage so calculations can be sharded among multiple nodes and you can do layered ranking without passing huge blobs of results between systems. Source: I'm currently working on the third version of my search engine and I've been studying this for 10 years.

In Ask HN: I want to dive into how to make search engines

Xeoncross·Aug 23, 2022Recommendation phrase

I highly recommend people comfortable with Go checkout the building blocks at https://github.com/thomasjungblut/go-sstables This codebase shows how SSTables, WAL, memtables, recordio, skiplists, segment files, and other storage engine components work in a digestible way. Includes a demo database showing how it all comes together to make a RocksDB / LevelDB competitor (not really).

In What's the big deal about embedded key-value databases?

Xeoncross·Jul 23, 2022

There isn't a one-size-fits all approach, but I've never worked on a project that encompasses as many computer science algorithms as a search engine. - Tries (patricia, radix, etc...) - Trees (b-trees, b+trees, merkle trees, log-structured merge-tree, etc..) - Consensus (raft, paxos, etc..) - Block storage (disk block size optimizations, mmap files, delta storage, etc..) - Probabilistic filters (hyperloloog, bloom filters, etc...) - Binary Search (sstables, sorted inverted indexes) - Ranking (pagerank, tf/idf, bm25, etc...) - NLP (stemming, POS tagging, subject identification, etc...) - HTML (document parsing/lexing) - Images (exif extraction, removal, resizing / proxying, etc...) - Queues (SQS, NATS, Apollo, etc...) - Clustering (k-means, density, hierarchical, gaussian distributions, etc...) - Rate limiting (leaky bucket, windowed, etc.

Read the rest

..) - text processing (unicode-normalization, slugify, sanitation, lossless and lossy hashing like metaphone and document fingerprinting) - etc... I'm sure there is plenty more I've missed. There are lots of generic structures involved like hashes, linked-lists, skip-lists, heaps and priority queues and this is just to get 2000's level basic tech. - https://github.com/quickwit-oss/tantivy - https://github.com/valeriansaliou/sonic - https://github.com/mosuka/phalanx - https://github.com/meilisearch/MeiliSearch - https://github.com/blevesearch/bleve - https://github.com/thomasjungblut/go-sstables A lot of people new to this space mistakenly think you can just throw elastic search or postgres fulltext search in front of terabytes of records and have something decent. That might work for something small like a curated collection of a few hundred sites.

In Writing your own search engine is hard (2004)

tjungblut·Feb 10, 2022

If you're interested to play around with those concepts mentioned in the blog, I built a small go library here: https://github.com/thomasjungblut/go-sstables

In Understanding LSM trees: What powers write-heavy databases (2020)