Very cool! In a similar vein Distributed Services with Go [0] works through SST creating a KV store. I found it helpful for working with BadgerDB [1].
[0] https://pragprog.com/titles/tjgo/distributed-services-with-g...
[1] https://github.com/dgraph-io/badger
In What's the big deal about embedded key-value databases?dgraph-io
badger
Go2017–2022
dgraph-io / badger
badger
Fast key-value DB in Go.
25Distinct people
30Mentions
0Explicit recommenders
15.8KGitHub stars
What to know before exploring
What it is
Fast key-value DB in Go.
GitHub
Where it is used
- Usenet Express — Usenet Express - Serving over 300TB of data with Badger. Source
- BBVA — BBVA - Raft backend implementation using BadgerDB for Hashicorp raft. Source
- 8 more documented examples
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:03:16 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year30 dated mentions
Observed mentions: 2017-05-14 17:08:41 UTC to 2022-08-23 23:52:42 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 3 | 1 | 2 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 10 | 1 | 9 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 4 | 1 | 3 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 4 | 0 | 4 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 7 | 0 | 7 |
From the project README
An excerpt from the repository, fetched Nov 13, 2026.
Read the project’s introduction
BadgerDB
BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go. It is the
underlying database for Dgraph, a fast, distributed graph
database. It's meant to be a performant alternative to non-Go-based key-value stores like RocksDB.
Project Status
Badger is stable and is being used to serve data sets worth hundreds of terabytes. Badger supports
concurrent ACID transactions with serializable snapshot isolation (SSI) guarantees. A Jepsen-style
bank test runs nightly for 8h, with --race flag and ensures the maintenance of transactional
guarantees. Badger has also been tested to work with filesystem level anomalies, to ensure
persistence and consistency. Badger is being used by a number of projects which includes Dgraph,
Jaeger Tracing, UsenetExpress, and many more.
The list of projects using Badger can be found here.
Please consult the Changelog for more detailed information on releases.
Note: Badger is built with go 1.23 and we refrain from bumping this version to minimize downstream
effects of those using Badger in applications built with older versions of Go.
Table of Contents
BadgerDB
Project Status
Table of Contents
Getting Started
Installing
Installing Badger Command Line Tool
Choosing a version
Badger Documentation
Resources
Blog Posts
Design
Comparisons
Benchmarks
Projects Using Badger
Contributing
Contact
Getting Started
Installing
To start using Badger, install Go 1.23 or above. Badger v3 and above needs go modules. From your
project, run the following command
go get github.com/dgraph-io/badger/v4
This will retrieve the library.
Installing Badger Command Line Tool
Badger provides a CLI tool which can perform certain operations like offline backup/restore. To
install the Badger CLI, retrieve the repository and checkout the desired version. Then run
cd badger
go install .
This will install the badger command line utility into your $GOBIN path.
Badger Documentation
Badger Documentation is available at https://badger.dgraph.io
Resources
Blog Posts
Introducing Badger: A fast key-value store written natively in Go
Make Badger crash resilient with ALICE
Badger vs LMDB vs BoltDB: Benchmarking key-value databases in Go
Concurrent ACID Transactions in Badger
Design
Badger was written with these design goals in mind:
Write a key-value database in pure Go.
Use latest research to build the fastest KV database for data sets spanning terabytes.
Optimize for SSDs.
Badger’s design is based on a paper titled WiscKey: Separating Keys from Values in SSD-conscious
Storage.
Comparisons
Feature
Badger
RocksDB
BoltDB
Design
LSM tree with value log
LSM tree only
B+ tree
High Read throughput
Yes
No
Yes
High Write throughput
Yes
Yes
No
Designed for SSDs
Yes (with latest research 1)
Not specifically 2
No
Embeddable
Yes
Yes
Yes
Sorted KV access
Yes
Yes
Yes
Pure Go (no Cgo)
Yes
No
Yes
Transactions
Yes, ACID, concurrent with SSI3
Yes (but non-ACID)
Yes, ACID
Snapshots
Yes
Yes
Yes
TTL support
Yes
Yes
No
3D access (key-value-version)
Yes4
No
No
1 The WISCKEY paper (on which Badger is based) saw big wins with separating
values from keys, significantly reducing the write amplification compared to a typical LSM tree.
2 RocksDB is an SSD optimized version of LevelDB, which was designed specifically for
rotating disks. As such RocksDB's design isn't aimed at SSDs.
3 SSI: Serializable Snapshot Isolation. For more details, see the blog post
Concurrent ACID Transactions in Badger
4 Badger provides direct access to value versions via its Iterator API. Users can also
specify how many versions to keep per key via Options.
Benchmarks
We have run comprehensive benchmarks against RocksDB, Bolt and LMDB. The benchmarking code, and the
detailed logs for the benchmarks can be found in the badger-bench repo. More explanation,
including graphs can be found the blog posts (linked above).
Projects Using Badger
Below is a list of known projects that use Badger:
Dgraph - Distributed graph database.
Jaeger - Distributed tracing platform.
go-ipfs - Go client for the InterPlanetary File System (IPFS),
a new hypermedia distribution protocol.
Riot - An open-source, distributed search engine.
emitter - Scalable, low latency, distributed pub/sub
broker with message storage, uses MQTT, gossip and badger.
OctoSQL - Query tool that allows you to join, analyse and
transform data from multiple databases using SQL.
Dkron - Distributed, fault tolerant job scheduling system.
smallstep/certificates - Step-ca is an online
certificate authority for secure, automated certificate management.
Sandglass - distributed, horizontally scalable,
persistent, time sorted message queue.
TalariaDB - Grab's Distributed, low latency time-series
database.
Sloop - Salesforce's Kubernetes History Visualization
Project.
Usenet Express - Serving over 300TB of data with Badger.
gorush - A push notification server written in Go.
0-stor - Single device object store.
Dispatch Protocol - Blockchain protocol for distributed
application data analytics.
GarageMQ - AMQP server written in Go.
RedixDB - A real-time persistent key-value store with the
same redis protocol.
BBVA - Raft backend implementation using BadgerDB for
Hashicorp raft.
Fantom - aBFT Consensus platform for
distributed applications.
decred - An open, progressive, and self-funding
cryptocurrency with a system of community-based governance integrated into its blockchain.
OpenNetSys - Create useful dApps in any software language.
HoneyTrap - An extensible and opensource system for
running, monitoring and managing honeypots.
Insolar - Enterprise-ready blockchain platform.
IoTeX - The next generation of the decentralized
network for IoT powered by scalability- and privacy-centric blockchains.
go-sessions - The sessions manager for Go net/http and
fasthttp.
Babble - BFT Consensus platform for distributed
applications.
Tormenta - Embedded object-persistence layer / simple JSON
database for Go projects.
BadgerHold - An embeddable N
Read the full README on GitHubWhat Hacker News says
30 sourcesCool.
Curious why you chose sqlite instead of something like badger [https://github.com/dgraph-io/badger] given you expose it as a key value database, which badger is.
In Show HN: Kvass, a personal key-value storeThe litestream project was created by https://github.com/benbjohnson who wrote https://github.com/boltdb/bolt (a key value store) which has been instrumental (from my point of view) in the Go community as one of the original choices for an embedded database as it was not only fast, but had transactions with stable snapshots.
It was used by https://github.com/blevesearch/bleve, https://github.com/etcd-io/etcd, and number of other projects.
These days, https://github.com/dgraph-io/badger is often favored because of it's improved write throughput.
In A database for 2022Out of curiosity, why write https://github.com/cockroachdb/pebble and not just use https://github.com/dgraph-io/badger?
In CockroachDB: The Resilient Geo-Distributed SQL DatabaseOn the contrary, it sounds like a cover-up. Looking into the Github Issues, every-time someone does demonstrate data loss, the admins ignore it, argue it isn't data loss, or in one case, claim that the person who showed data loss is a disgruntled ex-employee (???)
https://github.com/dgraph-io/badger/issues/570#issuecomment-...
https://discuss.dgraph.io/t/bounty-challenge-prove-that-badg...
Given that the former CEO is here right now accusing people of shilling while he scrubs unpleasant details from his past posts, I'd say this just appears to be a dead, toxic startup.
In Quitting Dgraph LabsI think all you need to know about this CEO/CTO, Dgraph as a product, and the underlying key-value database Badger, is that when people report data loss bugs where the database fails to return data written to it, they'll claim it's not data loss if the data still happens to reside on the disk somewhere, and *then delete the bug report*.
https://github.com/dgraph-io/badger/issues/578 titled "Badger unable to return data after value log GC"
vs
https://github.com/dgraph-io/badger/issues/570#issue-3621168...
> However, if the bug only causes Badger to not return data, and the data is fully recoverable by an easy fix to the code -- that's technically not a data loss (See Badger unable to return data after value log GC #578).
Never let this person store your data.
In Quitting Dgraph LabsBadger has performed exceptionally well for us. Preferred over LevelDB/similar solutions.
https://github.com/dgraph-io/badger
In Quitting Dgraph LabsBadgerDB? https://github.com/dgraph-io/badger
In FerretDB: A truly open-source MongoDB alternativeWe're using Nats (https://nats.io/) for event streaming and pub/sub. The sessions are aggregated in memory, then they go to Badger (https://github.com/dgraph-io/badger), and finally the daily aggregates are easily distributable and immutable JSON files.
We're setting a session identifier to sessionStorage and those id's are wiped out once the session is processed and leaves the server memory. We worked together with a privacy specialist to ensure this is not violating GDPR. Details on what/how we collect data is here:
https://volument.com/privacy/
In Show HN: Volument – Our take on website analyticsHow does this compare to Badger[0], another similar in nature key-value store in Go?
What were the trade-offs which made it necessary to create something new instead of adapting what exists?
[0]: https://github.com/dgraph-io/badger
In Pebble: A RocksDB Inspired Key-Value Store Written in GoBadger supports concurrent ACID.
https://github.com/dgraph-io/badger
In Graviton Database: ZFS for key-value storesComparison to Badger? Badger is also go-native and, for me, has been exceptional at scale and for read-heavy workloads on SSD.
Ref: https://github.com/dgraph-io/badger
In Graviton Database: ZFS for key-value stores(author of Badger) Keeping things simple. If you can break it (decrypt data without knowing the master key), we'll be happy to pay you a bounty. We can extend it from data loss to also include "breaking encryption."
https://github.com/dgraph-io/badger/issues/601
In BadgerDB Implemented AES Encryption> I particularly like your idea that — if I understand it correctly — data can be stored independently of the log itself.
Not quite what I was getting at, though I see what you mean. (You might be interested in Badger₀, and the WiscKey paper₁ it's based on.)
[0] https://github.com/dgraph-io/badger
[1] https://www.usenix.org/system/files/conference/fast16/fast16...
To me, a "log" is just a directory of files (= log segments), where only one file (the newest segment) is ever opened for writing (specifically appending), and those appends are only ever of whole, valid records. Given a "log", you can treat everything except for the newest file/segment in it as an object (i.e. object-store backed local cache file, with atomic insert/replace/delete), rather than a file (= inode containing buffer of extents) per se.
Mind you, in Kafka's design, a
Read the rest
ll the old segments get "compacted" together, meaning that they get converted into a single on-disk hash table of the newest value for each key. That's cool and all, but the compacted-log hash table doesn't have object-like semantics; it suffers constant random writes every time new segments get "compacted into" it.
IMHO, rather than this kind of compaction, I'd much rather just have streaming compaction, sort of like LevelDB's level-based compaction. Ideally, every time a broker finalized a segment that caused it to now have 2^N contiguous historical segments of level N in a log, it would queue up a streaming-compaction job into another topic; and then some arbitrary node would work that job by streaming through the relevant segments with a de-duping merge sort, producing a new combined segment of level N+1 and then purging the level-N source segments from the object store.
Congratulations on the funding, I've been following the project for a while (your tech blog is great) and am especially fond of the spin-off project Badger[1]. I'd like to ask a few questions and make a few "content requests" for the tech blog if I may, since I see @campoy is here and @mrjn will probably stop by later =)
Some things I'd love to read about on your blog
1 - I'd love to read more on the local information retrieval strategies you use for Badger, especially if you have use cases that are comparable to sequential reads (something that Kafka does, but with a structure optimized for magnetic drives) or ISAM style indexation (traditional SQL databases), and how you leverage the metadata byte to speed things up (this has been touched on only superficially on the blog afaik) along with other specialized features/operations Badger su
Read the rest
pports;
2 - Some insights on how Dgraph's data model influenced the design of Badger, and what parts you decided to explicitly generalize for other use cases, and what use case constraints have you purposefully decided on (aside from the obvious ones like SSD optimized, etc);
3 - Any progress you might have made regarding why some queries in Badger take longer than in RocksDB. That so far has been the biggest cliffhanger in the tech blog, I've been waiting for the sequel to that one for some time now :)
It's difficult to find good resources where people go in depth on these database engineering topics, but your blog is a very good one and a joy to read. Thanks for not simplifying things and keeping it very technical!
Some questions relevant to the post and Dgraph itself
4 - When I (and I assume many others) see "Enterprise Features" on databases that is kind of a turn off. That usually correlates to keeping separate codebases that are being manually kept in sync (which has problems on it's own), and it sometimes feels like it's gatekeeping some pretty crucial features (e.g. better backups) which make people either accept a more vulnerable architectural situation, or makes it unsuitable for a few use cases for lack of such features. I do understand and respect the need to make money to keep Dgraph being awesome, but have you considered something like the CockroachDB approach (selling support + mandating a license to sell the DB as a managed service)? If you did, why did you turn it down and what would make you reconsider it? On a second note from a maintenance perspective, how are you planning to handle the "feature flagging" and "repo syncing" of community and enterprise editions?
5 - As someone who's considered using Dgraph in the past, something that I found interesting is that graph databases in some use cases alleviate some of the same data modelling woes that Datomic does[2][3]. How would you diff them apart and what would you say they differently excel at?
6 - Graph databases sit in a spectrum of use cases, where some people want a system to work similar to a traditional RDBMS but with more flexible/powerful data modelling capabilities, and some people might want to load up big graph-heavy datasets for analytics purposes (i.e. loading a giant product catalog to map product relationships and either do a bunch of batch queries to feed a secondary dataset or keep it live for realtime queries on a recommender system). How would you place Dgraph and it's strengths/weaknesses in this spectrum? What alternative solutions would you recommend for the places where Dgraph would be unsuited for?
Sorry for writing too many questions (which will probably take up a lot of time to answer) but many of these things (especially on the second half) are not explicitly said anywhere, and are valuable knowledge for making an informed decision on a database product.
[1] https://github.com/dgraph-io/badger
[2] https://augustl.com/blog/2018/datomic_look_at_all_the_things...
[3] https://augustl.com/blog/2018/datomic_look_at_all_the_things...
Showing the 15 most recent of 30 sources. Enable JavaScript to search and browse every indexed mention.