etcd

> https://github.com/etcd-io/etcd/issues/9771 > stale bot marked this as completed (by fucking closing it) Ah, yes, what would a Kubernetes-adjacent project be without a fucking stale bot to close issues willy nilly

It isn't a mere warning. It is strongly recommended as the upper limit. https://www.perfectscale.io/blog/etcd-8gb https://github.com/etcd-io/etcd/issues/9771 And yes, I agree not allowing a pluggable replacement is really stupid.

Look at the code for the watcher client[1] and lease management[2]. [1]: https://github.com/etcd-io/etcd/blob/main/client/v3/watch.go [2]: https://github.com/etcd-io/etcd/blob/main/client/v3/lease.go

I was curious and dug into the Go client code. You linked to the definition of KV – the easiest way to create one is with NewKV [1], which internally creates a RetryKV [2] wrapper around the Client you give it. RetryKV implements the KV methods by delegating to the underlying client. But before it delegates an immutable request (e.g., range), it sets the request retry policy to repeatable [3]. Retries are implemented with a gRPC interceptor, which checks the retry policy when deciding whether a request should be retried [4]. The Jepsen writeup says a client can retry a request when “the client can prove the first request could never execute, or that the request is idempotent”. In my (cold) read of the code, the Go client stays within those bounds. For non-idempotent requests, the Go client only retries when it knows the request was never

could you be more specific on what's so hard to follow? it's quite literally just the implementation of the GRPC interface [1]. [1] https://github.com/etcd-io/etcd/blob/main/client/v3/kv.go#L3...

Thanks! Could you please point me to a reference for (1) etcd/wal actually does do preallocations (https://github.com/etcd-io/etcd/blob/24e05998c68f481af2bd567...) Yet to implement max buffer age! Any references for this would be bomb! Is mmap() really needed here? Came across a similar project that does this? Really gotta dig deep here! https://github.com/jhunters/bigqueue

Did I miss it or is there no call to os.File.Sync(), i.e. fsync, anywhere? Since you mention etcd/wal: https://github.com/etcd-io/etcd/blob/v3.3.27/wal/wal.go#L671 https://github.com/etcd-io/etcd/blob/v3.3.27/pkg/fileutil/sy...

This is essentially do-it-yourself etcd. https://github.com/etcd-io/etcd

Reminded me of etcd, which is exactly this - Raft-based key-value store written in Go. https://github.com/etcd-io/etcd

I am not sure neither. But this might overcome the etcd's soft storage limit of 8GB? [1] [1] https://github.com/etcd-io/etcd/issues/9771

this happened very recently Etcd v3.5.[0-2] is not recommended for production => https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQ... https://github.com/etcd-io/etcd/issues/13766#issuecomment-10...

The litestream project was created by https://github.com/benbjohnson who wrote https://github.com/boltdb/bolt (a key value store) which has been instrumental (from my point of view) in the Go community as one of the original choices for an embedded database as it was not only fast, but had transactions with stable snapshots. It was used by https://github.com/blevesearch/bleve, https://github.com/etcd-io/etcd, and number of other projects. These days, https://github.com/dgraph-io/badger is often favored because of it's improved write throughput.

Example where etcd’s logger was calculating log size and suffering from a performance issue as a result: https://github.com/etcd-io/etcd/issues/12835